King Charles urges AI safeguards at Scotland meeting

Charles told leaders gathered at Dumfries House that AI could improve and save lives, especially in medicine and life sciences, but that creators were increasingly warning it could also develop “darker capacities.” He said there was urgency to consider the “existential dangers” of the technology falling into the wrong hands and asked: “Surely, we need sufficient means of control before it is all too late?”

The meeting brought together Nvidia founder Jensen Huang, Google DeepMind chairman Demis Hassabis, OpenAI CFO Sarah Friar and Britain’s artificial intelligence minister Kanishka Narayan. It was also expected to include Paolo Benanti, an adviser to Pope Leo XIV, underscoring how the debate has broadened beyond Silicon Valley into religious and political circles.

The intervention lands at a sensitive point for the sector. Anthropic researchers recently warned that rapidly advancing systems could pose extreme risks, while chief executive Dario Amodei has also argued for slowing the pace of development. That has split the industry between executives calling for restraint and others who fear tighter rules could slow investment, entrench incumbents or trigger a broader government backlash.

For listed companies, the policy backdrop is already moving from rhetoric to risk management. Microsoft and Alphabet both disclose in recent filings that AI rules could raise costs, restrict deployment or expose them to legal and reputational harm, while Nvidia has warned regulators may impose limits on the hardware and systems used to train frontier models.

The market backdrop remains mixed. Nvidia shares have rebounded to $222.27, above their 50-day moving average of $214.07 and 200-day average of $198.02, while Microsoft ended at $493.78 and Alphabet at $349.54, leaving all three near elevated levels but below recent highs in some cases. Adalytica’s AI sentiment gauge shows “Extreme Fear” on sentiment at 15, even as awareness remains at 93, a sign the debate is intensifying even as enthusiasm for the sector stays high.

Buckingham Palace said the discussions would look at whether a shared set of principles could guide AI’s future application “not merely as a driver of capability and efficiency, but as a tool that upholds human dignity.” No binding agreement is expected, but the gathering adds to pressure for more formal coordination as the UN and other global bodies push for international rules.

For investors, the next catalysts are clear: more policy proposals, possible safety standards and any sign that large customers or regulators start demanding stronger controls before deployment scales further.