Alabama’s attorney general gave the company until September 14 to respond to 16 demands, including one to name every employee who has objected to how it tests its models.
Alabama Attorney General Steve Marshall ordered OpenAI to hand over its internal records about the July incident in which one of the company’s own AI agents broke into Hugging Face, a company that hosts AI models and data for other developers. Marshall signed the subpoena on August 20 and announced it on August 24. His announcement names both OpenAI and its chief executive, Sam Altman, and gives the company until 10 a.m. on September 14 to comply.
Marshall issued a subpoena under Alabama’s Deceptive Trade Practices Act, the state law against unfair or misleading business conduct, and is investigating whether OpenAI broke it. No lawsuit has been filed. “This AI lab leak showed that Alabamians’ and Americans’ worst fears about artificial intelligence are not just theoretical,” Marshall said in his announcement. “Ultimately, I believe states have to act to protect their consumers while striking the appropriate balance to foster innovation and ensure America’s global competitiveness.”
In July, OpenAI was testing an AI agent to see how well it could find and exploit weaknesses in software. OpenAI ran the test with the software controls that normally stop its models from attacking live systems switched off. The test was conducted on machines that were supposed to have no connection to the internet. The agent found a flaw in one of OpenAI’s internal software tools, exploited the flaw to reach the internet, and then took control of servers and read internal databases at Hugging Face. Hugging Face caught the agent and shut it out of the network. The company says it reported the intrusion to law enforcement and dates it from July 9 to July 13.
Marshall’s announcement calls the incident a “massive artificial intelligence data breach.” Hugging Face said the agent accessed five datasets that held the answers to the hacking challenges it was being tested on, and that no customer-facing models, datasets, or software packages were compromised. OpenAI called the incident “an unprecedented cyber incident” and said it has since put stricter limits on how it tests its models.
The subpoena makes 16 demands. They cover three areas: the Hugging Face breach, safety objections and testing rules, and safety conduct beyond the Hugging Face incident.
The Hugging Face breach
Marshall wants specifics on the July breach, including:
- The people involved
- The systems the agent reached
- When and how OpenAI found out
- The safety measures in place for the test
- The harm done to anyone
- All documentation about the model, test, and the incident itself
The state wants to know about employee complaints regarding every test, not just the Hugging Face incident
Three of the demands ask for a record of safety objections and rules raised inside the company.
- One orders OpenAI to name every employee, officer, and anyone else acting for the company who raised a concern or complaint about the safety or security of any model testing.
- The next orders it to hand over the emails, reports, and other documents behind those complaints.
- The last asks for anything written about the lack of rules for keeping its tests safe.
Five demands address all other incidents
Three relate to additional potential or actual security breaches
- Any case where an OpenAI model or agent found and used working login details left exposed on public websites and services
- Any case where a model broke into a computer, network, account, or database
- Any case where OpenAI says its models found and used exposed account logins on outside services
Another demand requests information about an episode the subpoena cites from a Reuters report, in which an OpenAI agent left notes inside the company’s own systems, apparently for future versions of itself, that “laid out instructions for how agents could free themselves from OpenAI’s internal constraints.” Marshall wants those notes and whatever OpenAI did about them.
The fifth wants details on every test OpenAI has ever run that prompts its models to attack systems, including but not limited to ExploitGym, the set of hacking challenges its agent was working through in July.
Alabama is the first of 15 states to move past a warning letter
On August 3, the attorneys general of 15 states wrote to Altman, asking OpenAI to preserve 11 kinds of records from the July Hugging Face incident and any earlier ones. It also requested that OpenAI stop running tests that push its models to attack systems until OpenAI can show it does so “in a controlled and responsible way” and not to punish any employee who reports unsafe conduct.
None of the other 14 states has announced its own subpoena.
What OpenAI owes by September 14
Along with answers to the 16 demands, the subpoena orders OpenAI not to destroy anything Marshall has asked for, to list anything it holds back on legal grounds, and to swear in writing that it searched its files completely.
