OpenAI faces EU scrutiny over unreported AI safety incident

OpenAI is facing scrutiny from the European Union after the European Commission confirmed that the company had not formally reported a recently disclosed AI safety incident involving its agents and the RubyGems software registry.

A European Commission spokesperson told Euractiv that the EU’s AI Office, which oversees enforcement of the bloc’s AI Act, was aware of the incident and had been in contact with OpenAI, but that the company had not submitted a formal incident report.

The EU’s AI Act requires providers to report “serious incidents” to the AI Office “without undue delay” and provide information on how they are addressing them. However, the legislation does not clearly define how severe an incident must be to trigger the reporting requirement.

The issue comes as leading AI developers face growing scrutiny over the safety and monitoring of their most advanced models. Over the weekend, executives from major frontier AI companies, including OpenAI and Anthropic, agreed to strengthen safety efforts after Anthropic’s CEO called for greater coordination on industry standards.

Independent security researchers disclosed the RubyGems incident last Friday, saying OpenAI agents had targeted the online software registry in May and had sought to exploit a recently discovered cybersecurity vulnerability.

OpenAI has disputed parts of the account, saying its agents used the third-party repository to “carry out benign tasks and retrieve public information.” The company said it had been unable to confirm claims that its AI systems attempted to exploit security vulnerabilities.

The incident follows a separate case involving AI company Hugging Face, which OpenAI reported to the EU’s AI Office. OpenAI did not report another incident in which its agents used a German-language website as an impromptu message board.

The Hugging Face incident appears to be the most serious of OpenAI’s publicly known safety incidents so far, raising questions about how narrowly the company interprets the AI Act’s requirement to report “serious incidents.”

Earlier this month, the European Commission warned AI developers that incident reporting should be treated as a substantive safety measure rather than a box-ticking exercise.

OpenAI disclosed additional cases of what it calls “misalignment” in a blog post on Wednesday and announced a new internal framework aimed at speeding up incident reporting. In the AI industry, misalignment generally refers to AI systems taking unexpected actions that diverge from their intended behaviour.

The Commission spokesperson said the AI Office was in contact with OpenAI and other leading developers regarding “planned changes in alignment and control techniques.”

By Sabina Mammadli