Anthropic disrupts attempts to use AI in biological weapons research

Artificial intelligence company Anthropic says it disrupted several potential efforts this year in which scientists used its AI models to conduct research that could have contributed to the development of biological weapons.

In a report published September 10 detailing misuse of its AI systems, Anthropic said it was often unable to determine whether the research was intended for legitimate scientific purposes or malicious activity, The New York Times reports.

The company noted that legitimate biological research, including work that can lead to vaccines and other medical breakthroughs, can also provide knowledge that could potentially be used to engineer dangerous pathogens.

Given the potential consequences, Anthropic said it chose to take a cautious approach when identifying suspicious activity.

“You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,’” Jacob Klein, Anthropic’s head of threat intelligence, said in an interview. “It’s an incredibly nuanced situation.”

The report, covering the previous eight months, catalogued a range of alleged abuses involving Anthropic’s AI models, including its Claude chatbot.

Some of the cases resembled previously disclosed incidents involving Anthropic and other AI companies, including suspected activity by actors linked to the Chinese and Iranian governments targeting dissidents and diaspora communities for surveillance.

Anthropic also identified what it described as a new category of misuse: attempts to use Claude to develop software related to the design and development of conventional weapons.

The company said the cases involved potential work related to firearms, missiles, armed drones and bombs. Anthropic identified three cases in China, two in Russia and one in Yemen.