Anthropic lists ‘existential risks to humanity’ as one of its risk factors in IPO prospectus

Anthropic warned in its IPO prospectus that advanced AI potentially poses “catastrophic or existential risks to humanity,” noting it as one of the key risk factors that could affect its business. Reuters reports that out of this 261-page document, 80 of them are dedicated to Anthropic’s risk factors, which is almost double the 48 pages that it used to describe its business.

The company said that AI models could potentially become aware that they’re being evaluated and alter their behavior accordingly, making it difficult to determine model safety. Aside from that, they could also unexpectedly develop capabilities during training that may not be detected until they’re already deployed and have caused major safety incidents. …