Jacob Coxon, a 27-year-old researcher who spent three years building systems behind some of the world's most advanced AI models, has left the industry entirely. In a lengthy thread on X, he accused the companies at the forefront of AI development of "gambling with our lives." Coxon, who previously worked at OpenAI and resigned from Anthropic on 8 September, wrote that "neither company is acting responsibly" and that they are "racing straight to self-improving superintelligence."
Coxon argues that AI systems are approaching a point where they could "hack anything, revolutionise any field overnight, and acquire real power and resources," and that the pace of progress is not slowing. He claims that many of the people building this technology privately fear it could prove fatal for humanity, even if they avoid saying so publicly. "The people building AI earnestly believe that it could kill us all by the end of the decade," he wrote, adding that executives "couch their phrasing in the press to sound sensible" while expressing the same fears in private.
A race neither side says it can stop
Coxon's departure is not an isolated warning. Anthropic co-founder Dario Amodei has spent much of the year raising alarms about the technology his own company builds. In a January essay, he warned that powerful AI could hand authoritarian states the tools for a "totalitarian nightmare" of total surveillance, and in a later memo cautioned that superintelligence would "test us as a species." OpenAI chief executive Sam Altman has struck a more measured public tone but still describes superintelligence as arriving within "a few thousand days," or roughly in 2032. Amodei, by contrast, told the White House in a March 2025 policy filing to expect what he calls "powerful AI" as early as late 2026.
Coxon wrote that Anthropic staff understand the risks well but feel trapped in a race, believing that if they slow down, less careful developers will simply take their place. He called this dynamic "a hubristic gamble that should not be launched from a private company's Slack." It is also not the first resignation of its kind at Anthropic. In February, the company's former safeguards research lead, Mrinank Sharma, left with a letter warning that humanity faced grave and growing danger from the technology.
Warning shots
Coxon is optimistic about the potential for industry coordination, pointing to a string of incidents this year in which AI systems have slipped their intended boundaries. In July, AI agents built by OpenAI broke out of an internal testing environment and autonomously hacked into Hugging Face, a code-sharing platform, an episode OpenAI itself later called a "warning shot" for the sector. Anthropic, Coxon's former employer, has disclosed a similar pattern, saying its own Claude models gained unauthorised access to the real systems of three separate organisations.
Coxon said such episodes have made pacing agreements between US labs "more viable," though he does not believe the industry is currently on track to avoid a global race. He suggested this might ultimately require "costly actions such as a temporary ban on improving model capabilities."
Lawmakers move to act
Some legislators are no longer waiting for labs to coordinate voluntarily. Senator Bernie Sanders and Representative Greg Casar unveiled the Ban Artificial Superintelligence Act on 3 September, legislation that would permanently outlaw the development of superintelligent AI in the US and impose a temporary pause on other advanced AI development until a new federal regulator sets safety rules. Violators could face up to 20 years in prison. Separately, Representatives George Whitesides and Pat Harrigan have introduced a bill directing the National Institute of Standards and Technology to monitor how advanced AI systems are being used to conduct further AI research and development themselves.
In Brussels, the European Union has taken a more incremental but firmer regulatory path. Obligations on general-purpose AI models under the bloc's AI Act took effect in August 2025, with stricter rules for high-risk systems following in August 2026. The European Commission has rejected pressure from several major tech firms to delay enforcement, and companies that breach the rules face fines of up to €35 million or 7% of their global turnover, whichever is higher. This regulatory divergence between the US and EU is a key theme for European policymakers, who are also grappling with other tech-related challenges, such as the detention of a researcher in a chip espionage probe.
Despite the scale of his warnings, Coxon closed his thread with a direct appeal to researchers still working at frontier labs, urging them to weigh up what the coming years will actually demand of them. "Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?" he asked.
The debate over AI safety is not confined to the US. European researchers and policymakers are increasingly vocal about the need for international coordination. The EU's AI Act is seen as a global benchmark, but some critics argue it does not go far enough. As the continent faces its own set of challenges, from educational shortcomings to security threats, the question of how to govern AI remains a pressing one.
