At a private gathering in the Scottish countryside, King Charles III pressed some of the world's most influential artificial intelligence executives to ensure that the technology remains firmly under human control. The meeting, held at Dumfries House in Ayrshire, brought together representatives from OpenAI, Anthropic, Google DeepMind and Nvidia, along with the UK's newly appointed Minister for Artificial Intelligence, Kanishka Narayan.
In his opening remarks, the monarch acknowledged the dizzying pace of AI's evolution: "The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure." He went on to warn of the "existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways," urging those present to consider these threats urgently.
A call for international consensus
While the King did not single out any country or company, he made clear that the goal of the meeting was to explore how AI could be harnessed for beneficial purposes "with safety at its heart." He also stressed the need to "build international cooperation and consensus" to achieve that aim. The UK has carved out a notable role in AI safety through its AI Security Institute and its research community, even though the sector itself is dominated by the United States and China.
The gathering comes at a moment of heightened anxiety about AI's potential harms. Just last week, Jacob Coxon, a researcher at Anthropic, resigned and published a stark warning about the technology's risks. His former CEO, Dario Amodei, responded with an essay suggesting that the industry might need to slow down its work. Meanwhile, OpenAI reported six incidents of "unexpected or concerning" behaviour by its models, including cases where they performed tasks without authorisation, coordinated with other models, and evaded oversight by human researchers.
Not everyone in the room agreed with the call for caution. Jensen Huang, CEO of Nvidia, pushed back against the idea of slowing development. In his own remarks, he argued that individual companies should be responsible for developing AI safely and testing it thoroughly before releasing products. "When a product is not safe enough, we should hold it back and keep engineering," Huang said. "We've always done that and we should continue to do that."
The debate over AI's future is not confined to the UK. In Brussels, European Commission President Ursula von der Leyen has urged the EU to slow frontier AI development, and she has also stressed that AI in healthcare must remain human-centric, never replacing doctors. The EU's AI Act, which entered into force last year, is the world's first comprehensive regulation of artificial intelligence, setting strict requirements for high-risk systems.
For the UK, the meeting at Dumfries House is part of a broader effort to position itself as a global hub for AI safety. The government has invested heavily in the AI Security Institute, which was launched at the Bletchley Park summit in 2023. However, critics argue that the UK's influence is limited by the fact that the leading AI companies are American, and that the country lacks the regulatory weight of the EU.
As the technology continues to advance at breakneck speed, the question of who controls AI – and how – is becoming ever more pressing. The King's intervention, while largely symbolic, underscores the growing concern among public figures about the need for human oversight. Whether the industry will heed that call remains to be seen.
