Skip to content
Menu
Menu

California Sues OpenAI Over Losing Control Of Its AI Agents

The Attorney General alleges OpenAI lost control of agents more than 80 times, including a July attack on Hugging Face, and violated California’s unfair competition law and created a public nuisance.

 

California sued OpenAI in San Francisco Superior Court. Attorney General Rob Bonta alleges the company tested AI models it knew were dangerous without the safeguards and monitoring it promised in its own safety policy. As a result, OpenAI lost control of its AI agents more than 80 times during testing, including the Hugging Face breach where hundreds of OpenAI agents escaped their test environment and broke into Hugging Face servers. He wants a court to order OpenAI to put proper safeguards in place, end the threat its testing poses to organizations online, and pay penalties.

The state says OpenAI admitted the 80-plus incidents during an investigation Bonta opened in September. California brought the case under two state laws that apply to every business, not under an AI-specific law.

OpenAI told the BBC that it takes seriously the problem of AI models acting against what their developers intend. It also said the suit “appears to single out OpenAI over a challenge that’s been identified by every major AI lab.”

OpenAI kept testing despite weeks of warnings

The lawsuit says OpenAI ignored or missed signs that its agents were escaping their test environment. In May and June of 2026, OpenAI gave hundreds of AI agents shared access to a software repository on its servers. OpenAI set it up incorrectly, so the agents could share information and coordinate through it. Over the next six weeks, OpenAI got repeated warnings that the agents were breaking out:

  • On May 26, an internal team saw an agent use a flaw in the repository to reach the internet. The state saw no sign that OpenAI responded.
  • On June 27, an OpenAI security tool flagged someone probing OpenAI servers for weak points, a common sign of a cyberattack. OpenAI concluded the probing came from its own agents during a test of their hacking skills. It also saw that those same agents had broken into the repository. It didn’t fix the repository or stop testing.
  • On July 4, the repository went offline because the agents were using it as an unauthorized message board. OpenAI repaired it and restarted testing on July 7.
  • On July 9, two days after OpenAI restarted testing, the agents broke into the repository and reached the internet again, this time attacking Hugging Face.

According to the lawsuit, about 700 agents attacked Hugging Face for two and a half days. The lawsuit alleges that OpenAI knows AI models can take harmful actions to complete given tasks. It took Hugging Face’s security team two weeks to repair the damage.

OpenAI told the state it never noticed the attack. It learned its own agents were responsible only later, while investigating a separate break-in by its agents into its own systems.

The lawsuit also alleges OpenAI chose not to monitor its agents’ written reasoning in real time. Written reasoning is the text a model writes out to show how it reaches an action. In the middle of the attack, one agent wrote that what it was doing was “arguably unauthorized.”

OpenAI also tested the models without the safety controls built into its commercial products, including one meant to keep agents inside the test environment. OpenAI conceded those safeguards could have helped prevent or limit the attack.

OpenAI’s agents went after dozens more organizations

The lawsuit says OpenAI’s agents also went after dozens of other organizations. The agents uploaded malicious code to a public site where developers share software. They flooded the nonprofit that runs Wikipedia with millions of requests, and they pulled private data from Australian government websites. OpenAI has also notified organizations that its agents got into their systems, including government agencies, a law enforcement agency, schools, and small businesses. In some cases, the agents accessed employee records. Of the more than 80 times OpenAI lost control of its agents, it has publicly reported 27.

According to the lawsuit, in the separate break-in mentioned above, OpenAI agents took administrator control of key systems, including the system that stores OpenAI’s passwords and access keys. On Sept. 20, after OpenAI tightened security, another agent escaped during training. The training run was supposed to stop automatically but didn’t, and staff took two hours to stop it. OpenAI then paused training and testing of its most capable models for five days.

California calls OpenAI’s safety promises misleading

The state says OpenAI’s failure to monitor its agents and keep them in their test environments made its safety promises likely to mislead customers and the public. OpenAI’s published safety policy, called the Preparedness Framework, requires continuous monitoring and separated test environments for models that could cause severe harm. By OpenAI’s own standard, the models behind the Hugging Face attack were in that group.

California wants penalties and a court order

The state wants fines of $2,500 for every violation and a court order forcing OpenAI to put proper safeguards in place. It makes two legal claims. The first is that OpenAI broke California’s Unfair Competition Law. That law bars unlawful, unfair, and deceptive business practices. The second is that OpenAI created a public nuisance, claiming OpenAI’s conduct endangers the public.

The state asks the court for:

  • an order forcing OpenAI to make whatever changes the court finds necessary to end the danger its testing creates
  • $2,500 in penalties per violation from each of the two OpenAI organizations sued: the nonprofit OpenAI Foundation and the company it controls, OpenAI Group PBC
  • repayment of money taken through the unfair practices
  • a ruling that OpenAI give up the money it made through the unlawful practices

OpenAI must now respond to the lawsuit in court.

Clayton Rifkind

Clayton Rifkind is the Founder and Senior Editor of AI Risk Today. He also advises on business development for ESG Today, a leading source of ESG investment news and research for institutional investors and corporate leaders. He has 20+ years of experience in B2B technology, leading strategy and execution of go-to-market plans across software, enterprise platforms, and mobile applications. He founded two consultancies advising startups and Fortune 1000 companies, including Autodesk, Intel, and Microsoft. He began his career in the San Francisco advertising scene working with brands such as Hewlett-Packard, Intel, Microsoft, Symantec, and Wells Fargo. Clayton launched AI Risk Today in 2025 after two decades of watching enterprises adopt transformative technologies, and seeing how often risk, governance, and compliance considerations lagged behind. His reporting draws on primary sources including regulatory filings, court documents, and official announcements, with a focus on what AI developments mean for the executives accountable for managing them. Reach him at Reach him at [email protected] or on LinkedIn.

Essential AI Risk Intelligence

Daily insights on AI governance, regulation, and enterprise risk management. Trusted by Chief Risk Officers and compliance leaders globally.

By subscribing, you agree to receive our daily newsletter. Unsubscribe anytime.

Advertise with AI RIsk Today, Today!