OpenAI and Anthropic Push UN AI Safety Rules

Altman said the “most important decisions” could not be made by labs in San Francisco alone, while Amodei called for agreements banning AI use in biological weapons development, common testing standards and stronger verification and incident reporting. Their message reflects a growing recognition inside the industry that the regulatory question is moving from abstract ethics to operational risk: model testing, incident disclosure, export controls and liability standards could all become part of the cost of doing business.

The geopolitical divide was clear. Britain and France backed international frameworks, and China’s U.N. ambassador also called for stronger regulatory mechanisms and cross-border cooperation. The United States, however, rejected the idea of a global regulator, with White House adviser Michael Kratsios arguing that countries should build their own expertise and share best practices instead. That leaves the world’s largest AI market still favoring a lighter-touch, national approach even as other governments push for common guardrails.

For investors, that means the prize remains enormous but the policy discount is rising. Nvidia and Microsoft, two of the market’s biggest AI beneficiaries, already flag in filings that government restrictions, safety rules and cross-border limits could raise costs or constrain deployment. Nvidia’s shares have remained near record levels and Microsoft has held above its 50-day and 200-day moving averages, but the stock moves also show how quickly sentiment can turn when regulation threatens the pace of AI monetization.

The backdrop is a broader clash over whether AI should be policed by industry self-regulation or by formal government oversight. A recent White House summit underscored the voluntary approach favored by Donald Trump and several technology chiefs, but regulators in the U.S. and abroad are already testing that model through investigations and proposed rules. The U.N. push suggests the debate is moving beyond domestic politics toward a more durable question: whether advanced AI can be safely scaled without a common international framework.

If the talks gain traction, the most likely winners are firms that can absorb compliance costs and prove their systems are auditable, secure and enterprise-ready. The losers would be smaller model builders, open-source players and vendors exposed to export controls or stricter testing requirements. For now, the industry is still writing the rulebook — but the fact that CEOs are asking the U.N. to do it with them is itself a sign that safety concerns are becoming a market issue, not just a policy one.