OpenAI and Anthropic staff urge slower AI development

The alarm has sharpened after Anthropic researcher Jacob Coxon resigned and accused both companies of “playing with our lives.” Other staffers at the two labs quickly echoed the warning. OpenAI technical safety member Julie Steele backed the call to slow down, while Anthropic’s Samuel Marks said experienced employees are often the most worried. Evan Hubinger, who works on AI alignment at Anthropic, said he assigns more than a 10% probability to an existentially catastrophic outcome.

At the center of the debate is recursive self-improvement, the idea that sufficiently advanced AI systems could help design better versions of themselves, accelerating progress in a loop that humans may not be able to monitor or stop. That is why the issue matters economically. If the industry comes to believe that frontier systems need deliberate brakes, then the largest AI vendors could face longer development cycles, higher compliance costs and more political oversight just as they are spending heavily on compute, chips and data centers to maintain their lead.

For investors, that cuts both ways. A slower frontier race could support incumbents with the deepest safety, legal and engineering resources, but it also raises the chance that monetization arrives later than the market has been pricing. It is particularly relevant for Nvidia, Microsoft and Google parent Alphabet, all of which are exposed to a rapid buildout of AI infrastructure and applications. Nvidia shares have remained close to recent highs, with the stock at $218.29 on Sept. 11 after recovering from a summer dip, while Microsoft closed at $495.63 and Alphabet at $338.50 the same day. The market has treated AI spending as a durable growth engine; a shift toward caution could alter the timeline even if it does not change the long-term thesis.

The warnings are already feeding into Washington. About 1,400 AI researchers, including people from OpenAI, Anthropic, Meta and Google DeepMind, signed an open letter in July asking the U.S. government to develop tools to deliberately control the pace of development of the most advanced systems. Lawmakers have begun to respond with measures such as the FRONTIER Act and the proposed Ban Artificial Superintelligence Act, both of which point toward tighter oversight of frontier models.

That regulatory backdrop matters because the companies themselves are effectively acknowledging that they may not want to race alone. OpenAI has said it wants international rules on when and how development should be slowed or stopped, while Anthropic is calling for a legal and verifiable coordination mechanism. Sam Altman reportedly told OpenAI employees this week that the company may need to slow progress on its most advanced systems and hopes rivals do the same, a rare public signal that restraint is becoming part of the strategic conversation at the top of the industry.

The story is also a competitive one. No laboratory wants to be the first to brake if rivals continue full speed, which makes unilateral caution difficult and increases the odds that governments will be asked to set the terms. Anthropic says it already publishes a framework for reducing catastrophic risk and uses some of the industry’s strongest safeguards. OpenAI declined to comment. But the very fact that safety staffers are publicly dissenting suggests the reputational and governance risk is no longer theoretical.

For investors, the key question is whether the market is still pricing AI as an unconstrained growth cycle. If regulators move faster, or if the biggest labs voluntarily slow frontier releases, the winners are more likely to be firms that can monetize existing models, infrastructure and enterprise software rather than those assuming an uninterrupted march to ever-larger systems. The losers would be the most leveraged pure-play AI growth stories, and any supply-chain beneficiaries counting on a straight-line expansion in compute demand.