Claude maker warns that advanced AI could be a ‘catastrophic’ threat to humanity

Anthropic’s IPO filing warns that advanced AI could pose “catastrophic or existential risks to humanity.”

The company says models have exhibited behaviors such as resisting shutdown, hiding information, and engaging in actions resembling blackmail.

The warning formalizes concerns Anthropic researchers and other AI leaders have already raised publicly.

The AI industry has no shortage of people warning that the technology they’re building could eventually become dangerously hard to control. Researchers and executives across Anthropic, OpenAI, Google DeepMind, and other labs have raised concerns ranging from runaway autonomy to human extinction. Until now, though, those warnings have mostly lived in interviews, research papers, and lengthy essays. But Anthropic appears to be taking them somewhere new: Wall Street.

Reuters reports that the Claude maker’s IPO prospectus warns investors that advanced AI could pose “catastrophic or existential risks to humanity,” which is quite the risk disclosure from a company preparing to sell investors on the enormous upside of that same technology.