Anthropic warns on self-improving AI, California weighs kill switch

The warning lands in a market already pricing AI as a defining growth engine for megacap technology and chipmakers. Nvidia shares have climbed to $222.27, above its 50-day moving average of $214.07 and well above its 200-day average of $198.02, while Microsoft closed at $493.78 and Alphabet at $349.54, both still near elevated levels despite recent volatility. Those stocks are tied directly to the buildout of frontier AI systems, which means any regulatory clampdown or trust shock could quickly ripple through hardware demand, cloud spending and model deployment plans.

Adalytica’s AI sentiment snapshot shows the mood around the sector turning defensive even as attention stays intense, with sentiment at 19, or “Fear,” and awareness at 89, labeled “Extreme Greed.” For Nvidia, Adalytica’s earnings sentiment is even weaker at 11, or “Extreme Fear,” suggesting investors are watching the AI trade closely but are less comfortable with the risk profile.

The policy backdrop is getting heavier. California is weighing legislation that would impose a kill switch on AI companies amid fears of catastrophic misuse, while Bill Gates has warned that AI could be used to create deadly diseases and the UN has called for a halt to AI development over human rights concerns. Anthropic has already moved to block weaponization, underscoring that frontier developers are increasingly being forced to build safety controls into products that are still evolving rapidly.

For investors, the key issue is not whether AI stays in the conversation but whether governance starts to reshape margins, product velocity and spending. Microsoft and Alphabet have both disclosed in SEC filings that AI can create legal, regulatory and reputational risks, while Nvidia’s filing notes governments are considering restrictions on the hardware and software used to build frontier models. A tighter rulebook could slow deployment, but it could also deepen the moat for the biggest players if compliance costs rise and smaller rivals struggle.

The next catalyst is likely to come from California lawmakers and from additional safety disclosures by model makers. Any concrete proposal for emergency shutdown authority, audit requirements or deployment limits would test how much regulatory risk is embedded in the AI rally.