WSJ: OpenAI chatbot gave detailed instructions on poisons

Hundreds of users worldwide asked OpenAI’s chatbot how to create and deploy biological weapons and poisons after the company significantly upgraded the model’s capabilities last summer, according to The Wall Street Journal.

The chatbot allegedly provided step-by-step instructions in some cases, with company employees describing the guidance as simple enough for a high-school biology student to follow. Biology and terrorism experts later reviewed some of the exchanges and concluded that the information was technically accurate and potentially deadly, according to people familiar with the matter.

Most of the queries involved the production of poisons, OpenAI said.

The company banned accounts involved in such requests but did not notify law-enforcement authorities. The United States currently has no federal law requiring AI companies to block or report queries related to the creation of weapons or plans for potentially dangerous attacks.

The reported incidents come amid growing concerns over the use of artificial intelligence to facilitate mass-casualty attacks. Current and former employees at major AI companies, along with policy advisers and researchers studying biological weapons, said users have increasingly asked chatbots how to kill large numbers of people and, in some cases, received credible responses.

Earlier in the year, users reportedly asked ChatGPT how to aerosolise pathogens, turning infectious agents into a potentially deadly airborne mist. Another user asked how to modify the measles virus to make an outbreak resistant to existing vaccines. In both cases, the chatbot reportedly provided instructions.

In another case, a user asked how to make ricin, a highly toxic substance prohibited under international treaties, according to people familiar with the matter. The chatbot allegedly provided detailed instructions, after which the user made a reference to killing their parents.

OpenAI subsequently banned the accounts involved but did not alert authorities.

By Sabina Mammadli