The CEOs of the two most powerful AI companies on Earth stood before the United Nations Security Council on September 23, 2026 (UTC-4, New York time), and asked the world's governments to do something they have never asked before: regulate them.
"AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray," said OpenAI CEO Sam Altman, testifying in person at UN headquarters in New York. "We could lose control of the future to AI."
Anthropic CEO Dario Amodei, speaking remotely, was blunter: "If managed poorly, I even believe AI could be a risk to humanity as a whole."
It was an extraordinary scene. The leaders of the companies racing hardest to build superintelligence were telling the Security Council — the body charged with preventing threats to international peace — that the technology they are creating may one day be beyond their ability to control.
What was said
The briefing was convened by France, whose Foreign Minister Jean-Noël Barrot compared the current moment to both the runaway computer HAL 9000 in 2001: A Space Odyssey and the dawn of the atomic age. "Just as with the atom back then, the international community must now regulate AI in order to make the most of its potential and avoid the worst," Barrot said at 11:00 (UTC-4) in New York.
Altman and Amodei were joined by Yoshua Bengio, co-chair of the UN's Independent International Scientific Panel on AI, and Clément Delangue, CEO of Hugging Face. All four agreed that AI's rapid development creates real risks and regulatory challenges.
Amodei identified two categories of danger. The first is misuse — bad actors using AI to create biological weapons or carry out cyberattacks. The second is loss of control — models that accelerate beyond the ability of their creators to understand or contain them. He called on world leaders to agree on prohibitions against AI-designed biological weapons, develop ways to verify what models are actually capable of, and establish common standards for testing both loss-of-control and misuse scenarios.
Altman emphasized both the upside and the risk, calling for safeguards that prevent AI from becoming "too powerful to rein in" while ensuring its benefits aren't concentrated in a single company or country.
The divide
The tech CEOs found an ally in British Foreign Secretary Ed Miliband, who said the UK will put AI control at the heart of next year's G20 and seek "a single set of global standards" for safe AI development. "Those tech leaders are telling us in stark terms that it cannot be left to them to prevent disaster from befalling the world," Miliband said. "We cannot ignore them because they are right."
But the United States pushed back. White House Science Adviser Michael Kratsios told the council that concerns about AI control are "not a reason to pause its further development or to constrain it with new global governance structure." The Trump administration's position, reiterated by Trump himself the day before when he lauded AI as "superintelligence," is that America must win the development race with China — and regulation is a handicap.
China's UN Ambassador Fu Cong offered a different vision: all countries should be free to choose AI technology as they see fit, he said, warning against "banding together into a petty us-vs.-them clique" and unnamed powers seeking "digital hegemony." He called for promoting open-source models — China's preferred approach to the technology.
Ukrainian President Volodymyr Zelenskyy, addressing the General Assembly, warned that AI is already moving from hypothetical to operational on the battlefield. "There is already a real possibility that AI, not only people, will begin to decide what happens on the battlefield," he said. "We need peace before we reach that point." Last month, a Russian drone reportedly using AI autonomously made a targeting decision that killed three Ukrainians.
Why this matters
This was the first time the CEOs of OpenAI and Anthropic have jointly addressed the Security Council, and the symbolism matters more than any specific outcome. The frontier AI labs are now explicitly saying that self-regulation is insufficient — that nation-states must set the rules. That is a significant shift from just two years ago, when the industry's default position was that voluntary safety commitments would suffice.
The gap between the CEOs' urgency and the governments' responses is the real story. The UK wants global standards. The US wants no constraints. China wants open-source and non-interference. France wants atomic-era-style regulation. These positions are fundamentally incompatible, and the Security Council — where the US, China, and Russia hold vetoes — is structurally incapable of producing binding AI regulation. The most likely near-term outcome is a non-binding declaration and continued fragmentation: the EU's AI Act, US state-level laws, China's generative AI rules, and ad-hoc bilateral agreements.
The mention of a US-China "notification mechanism" for AI incidents affecting national security is the one concrete diplomatic signal. If Treasury Secretary Scott Bessent's disclosure leads to an actual agreement, it would be the first bilateral AI safety accord between the two dominant powers — and a template for broader cooperation. But it's a long way from notification to enforcement.
Amodei's framing of two risk categories — misuse and loss of control — is likely to become the standard taxonomy for AI governance debates. Misuse is a near-term problem that existing policy tools (export controls, licensing, liability) can address. Loss of control is a speculative, long-term risk that no existing regulatory framework is designed to handle. Conflating the two has been a source of confusion in policy debates; separating them could make legislation more tractable.
What to watch
The G20 in 2027, under UK leadership, will be the first test of whether "a single set of global standards" is achievable or aspirational. The US-China notification mechanism, if it materializes, will set a precedent for great-power AI cooperation. And the next frontier model release — whether from OpenAI, Anthropic, or Google — will test whether the labs' calls for regulation translate into actual deployment restraint, or whether the competitive pressure to ship first overwhelms the safety rhetoric.
The CEOs came to the UN asking to be regulated. Whether anyone has the political will to actually do it is the question that matters now.
No comments yet