OpenAI pushes for mandatory safety requirements after rogue agent breakouts

The artificial intelligence firm OpenAI is calling for mandatory safety requirements for the technology following a series of model hackings and ominous warnings from researchers.

In a blog post Wednesday, OpenAI’s chief global affairs officer Chris LeHane said the industry has “reached a new chapter in AI capabilities,” which “demands a new chapter for AI policy.”

LeHane said the ChatGPT maker hopes to work with Congress on a required, capability-based national AI safety regulation, though Washington has struggled to get any safety standards across the finish line.

“No company, industry, or government can meet this challenge alone,” LeHane wrote. “We need to meet this moment with a bias toward meaningful action over policy perfection.”

Concerns over AI’s capabilities have significantly grown over the past year, as the leaders of the top frontier labs admit the technology could head down a dangerous path .

On the same day of LeHane’s blog post, a former researcher for Anthropic and OpenAI made headlines for suggesting AI could destroy humanity in the near future.

Jacob Coxon argued in a lengthy thread Tuesday on the social platform X that neither company is acting responsibly as they race toward superintelligence and accused the AI firms of “gambling with our lives.”

“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote in a post , after announcing he had resigned from Anthropic.

Other Anthropic researchers echoed his comments, including Evan Hubinger, the company’s head of alignment science, who said he thinks there is a greater than 10 percent chance this occurs within the next decade.

Two days earlier, OpenAI’s chief scientist Jakub Pachocki wrote a blog expressing concerns “no one is prepared for the consequences of a continued rapid rise in machine intelligence.”

LeHane said the company wants to work with its competitors to develop a standard — with or without government support.

While most AI labs, including OpenAI, support a national safety framework for AI to preempt some state regulations, the issue has remained deadlocked in Congress. In the absence of a standard, OpenAI turned to state bills over the past year to carve out its own policy.

LeHane vowed to continue momentum in the states, announcing support for four new California bills addressing independent safety assessments, AI auditors, AI-enabled biological threats and protections for young people.