Anthropic reportedly blocked possible attempts to make bioweapons using its AI

Anthropic released a paper Thursday that details how the company took action against several users of its AI models who were conducting research that could have led to the development of biological weapons. The report states that the cases it includes aren't typical examples of misuse of its agents, and include everything from fake dating apps to surveillance systems to potential bioweapons.

The organizations and individuals Anthropic blocked for the bioweapons research are not named in the report, and it doesn't mention what biological agents were involved. But Anthropic says there were scientists who made "efforts to obfuscate the purpose of their research to evade our safeguards."

One of the case studies Anthropic cites in the report involved "a grant application for orthopoxvirus research at a state-associated infectious disease laboratory," which described planned work with live orthopoxviruses. These viruses include variola, better known as smallpox, and Mpox, or monkeypox.

Anthropic says in the report that it's publishing its efforts publicly "because we believe we have a responsibility to disclose malicious misuse of our services. As models become increasingly capable, their risks will increase, unless AI developers and society's defenders act to make them safer."

But it's worth noting the report's publication follows a series of social media posts this week from Anthropic researchers who warned that AI could "kill all humans" by the end of the decade. Researcher Jacob Coxon , who resigned from Anthropic on Tuesday, said the warnings "were not a marketing stunt," and that "no other human activity poses this level of danger."

Anthropic says that the examples of complex research it halted were only possible because its most recent AI models have become powerful enough to carry out the tasks, and with many of the cases, "even just a year ago, [they] would have required many skilled operators and specialist knowledge."