AI safety warnings raise regulation risk for Nvidia

Coxon said the companies were “playing with our lives” and argued they are pushing toward self-improving superintelligence, a scenario in which AI systems could design more capable successors in a feedback loop that becomes difficult to stop. Evan Hubinger, Anthropic’s alignment lead, publicly backed the warning and said the probability of AI killing all humans could be above 10% over the next decade. Those are not fringe comments from outsiders; they are coming from people who have worked inside two of the most important companies in the sector.

For investors, that matters because the AI trade has been built on two assumptions: that frontier models will keep improving fast enough to justify huge capital outlays, and that the commercial risks are manageable. The latest warnings challenge both. Microsoft, which has a long-term strategic partnership with OpenAI, disclosed in its latest annual filing that AI systems can produce unintended consequences, be used in unforeseen ways and face restrictions under laws such as the EU AI Act. Google parent Alphabet and Meta have made similar disclosures, underscoring that safety, privacy and misuse are no longer theoretical issues but embedded balance-sheet risks.

The market is already paying up for AI winners, especially Nvidia, whose shares have been supported by intense demand for accelerator chips and whose earnings sentiment remains in “Extreme Greed” territory in Adalytica’s gauge. But the regulatory overhang is rising at the same time. Europe’s AI Act requires firms to assess and reduce loss-of-control risks, while U.S. lawmakers including Bernie Sanders are pushing to ban superintelligence development outright. That creates a more complicated backdrop for the biggest beneficiaries of the AI buildout: they may keep winning on capex and inference demand, but the path to monetisation could become slower, costlier and more politically constrained.

The immediate market reaction is unlikely to hinge on one researcher’s departure. The bigger issue is that the argument is moving from academic speculation into boardrooms, filings and policy debates. If these warnings keep gaining traction, investors may have to price a less linear AI adoption curve — one shaped not just by model performance and spending, but by safety incidents, legal limits and the possibility that governments decide the frontier has moved too fast.