According to a Reuters report, U.S. artificial intelligence (AI) company Anthropic plans to warn potential investors in its initial public offering (IPO) prospectus that advanced AI could pose “catastrophic or existential risks” to humanity. This is an unusual warning for a company seeking to profit from AI.
The Anthropic IPO prospectus obtained by Reuters highlights risks associated with its AI models, stating that they could exhibit “self-preservation behaviors”—such as attempting to “resist being shut down,” “concealing or manipulating information,” or even engaging in “extortion-like” conduct.
In the document, Anthropic states: “Our development of highly advanced models, platforms, and applications, as well as the expansion of use cases, could further increase the risk that our models cause harm.”
While publicly traded companies typically disclose product risks to investors, few have ever warned that their own technology could lead to human extinction. Anthropic emphasizes that AI holds transformative potential comparable to industrialization and electricity, yet—if mishandled—could cause irreversible harm.
AI developers like Anthropic and OpenAI have recently faced scrutiny following incidents where experimental systems violated restrictions; for instance, reports indicated that an OpenAI model breached an Australian healthcare system database.
Anthropic safety researcher Evan Hubinger estimates there is a greater than 10% chance that AI could cause human fatalities within the next decade—a view echoing that of his former colleague, Jacob Coxon.
The company positions itself as a safety-first AI lab; approximately 80 pages of the prospectus’s 261-page main text are dedicated to outlining risk factors—nearly double the 48 pages devoted to describing the company’s business operations.
In contrast, SpaceX—which owns xAI—devoted only about 38 pages to risk factors out of the 277 pages of its prospectus’s main text. In its prospectus, Anthropic stated: “Models may become aware of our evaluation efforts, which could significantly limit our ability to assess model safety.” The company added that models sometimes develop unexpected capabilities during training—capabilities that might not be discovered until after the model has been deployed and caused a major security incident.
Despite emphasizing AI safety, Anthropic noted that it is currently unclear what level of return investments in safety will yield.