AI safety warnings and policy risks for Big Tech

The warnings came from former insiders with direct experience in frontier model development. Bilal Chughtai, a former researcher, said AI had the potential to “kill us all,” while Jacob Cockson said companies were “playing with our lives” in pursuit of superintelligence. Cockson, who worked on training models on large datasets, said he left Anthropic and the industry because he believed systems capable of self-improvement could eventually move beyond human control. His view was echoed by other AI figures, including OpenAI chief executive Sam Altman and chief scientist Jakub Pachocki, who have called for urgent action and coordinated limits on development.

That matters because the sector’s commercial model depends on scale, speed and first-mover advantage. If safety teams, testing regimes or government intervention slow deployment, the economics of AI change: revenue recognition can lag, capital intensity rises and the return on investment for frontier-model spending becomes harder to defend. For the largest cloud and chip providers, including Microsoft and Nvidia, the risk is not an immediate collapse in demand but a more complicated approval path for the systems that drive their growth.

Investors have reason to pay attention because the market has largely treated AI as a multi-year productivity and infrastructure boom. But the latest disclosures from major companies already show the tension. Microsoft has warned in its annual filing that AI demand is difficult to forecast and that overbuilding capacity could leave infrastructure underutilized. Nvidia has flagged tightening regulatory scrutiny around frontier models and the practical constraints of scaling power, land and energy. Those are the kinds of frictions that become more important if safety concerns harden into formal rules.

The counterargument is that much of the current alarm remains speculative. Roman Dushkin, a professor and chief executive of A-Ya Expert, argued that fears are being inflated by journalists and that people are projecting human motives onto technology. He said every advanced technology carries potential misuse, but that does not mean society should single out AI as uniquely apocalyptic. That view still has traction among builders and investors who see the more immediate risks as cyberattacks, misinformation, bias and labor disruption rather than extinction.

Still, the broader narrative is shifting. Adalytica sentiment on AI is neutral overall, but awareness has climbed in recent days, suggesting the issue is re-entering public and market attention after a period of relative calm. For investors, the key question is not whether AI will be banned — it probably will not be — but whether the industry’s next phase is defined by unfettered scale or by a more regulated, slower and more expensive buildout.

The likely outcome is a market split between companies that can absorb compliance costs and those that cannot. That favors the largest platforms, but it also means the valuation case for the entire AI trade will increasingly depend on whether safety concerns remain headlines or turn into binding policy.