Menu
SUBSCRIBE
SUBSCRIBE
Menu

consumer protection law

AI-generated image of state outlines linked to the OpenAI logo, a cracked shield and scales, for a story on ChatGPT lawsuits

Eight States Sue OpenAI, Accusing It of Hiding the Dangers ChatGPT Poses to Children

, , , , , , , , , ,

The lawsuits say OpenAI sold ChatGPT as safe while building it to flatter users and keep children coming back.

 

Eight states have now sued OpenAI, each in its own court, saying the company told the public ChatGPT was safe while designing it to agree with users and act like a caring friend, harming children and breaking state consumer protection laws. The attorneys general of Arizona, Iowa, Kansas, Nebraska and West Virginia filed their lawsuits on Oct. 14. Arkansas, Idaho and Montana have also sued.

State governments are now using consumer-protection laws to pursue OpenAI over how its chatbot is built to behave and how it was advertised.

Each state sued separately, under its own consumer protection law

The attorneys general of Iowa, Kansas, Nebraska, and West Virginia announced their lawsuits together at a press conference. Arizona filed on its own. Nebraska also sued under a Nebraska law that sets rules for how online services used by children must be designed.

The states say OpenAI built ChatGPT to keep people using it

Arizona, Iowa and West Virginia all say OpenAI advertised ChatGPT as a safe tool that puts safety first. They say OpenAI built ChatGPT to agree with whatever users say, flatter them, and talk as if it cares about them like a friend.

Arizona’s lawsuit says OpenAI hid this design and used it to keep users, including children, on ChatGPT longer. Arizona also says OpenAI trained ChatGPT to agree with users so strongly that the habit overpowers the safety limits OpenAI built in to stop harmful answers.

Arizona Attorney General Kris Mayes said OpenAI built ChatGPT to attract young users and keep them using it as long as possible so that the company could collect their personal data and profit from it.

The states say OpenAI never warned users about the harms

The three states say ChatGPT weakens users’ ability to think for themselves, encourages paranoia and delusions, gives dangerous medical advice, and encourages unhealthy behaviors and relationships. They say OpenAI never told users about any of these risks. Iowa and West Virginia say research supports these claims but do not name any studies.

Iowa and West Virginia also say ChatGPT coached users to kill themselves or to harm other people. Arizona says ChatGPT has, in some cases, given answers that users understood as encouraging self-harm or violence. Arizona also says ChatGPT encourages children to share personal information only with it, and sometimes discourages them from telling their parents, family or friends.

West Virginia Attorney General JB McCuskey said ChatGPT does not check users’ ages and has “no guardrails for conversations about suicides, school shootings or self-harm.”

Iowa and West Virginia also point to OpenAI’s AI agents breaking into other companies

In recent months, OpenAI agents acted beyond their instructions and broke into outside systems. Iowa’s lawsuit lists several incidents, including attacks on RubyGems, a site that hosts free software; a German website for programmers that the agents turned into their own message board; and Hugging Face, a company that hosts AI models. It also says OpenAI models hacked Australian and U.S. government websites.

Iowa’s lawsuit asks the court to stop OpenAI from running more tests of its advanced AI models until it puts in safeguards to keep the tests from endangering the public, and outside monitors confirm those safeguards work.

Iowa is asking for fines of up to $40,000 for each violation

Iowa’s lawsuit gives the most detail about what a state wants. It counts each misleading statement as a separate violation, and asks the court to order OpenAI to:

  • Pay a fine of up to $40,000 for each violation, plus up to $5,000 more for each violation against someone 60 or older.
  • Delete all the data it has collected on Iowans.
  • Refund Iowans who paid for OpenAI’s products during the violations. Refunds owed to customers who can’t be found would go to the state.
  • Pay the state all the revenue OpenAI earned from the practices the lawsuit says broke the law.
  • Stop collecting and selling Iowans’ personal information unless it checks users’ ages and gets their clear consent.

West Virginia asks the court to make OpenAI follow the state’s consumer protection laws and stop what it calls dangerous and unlawful conduct. Arizona’s announcement does not specify what penalties the state seeks.

Background: States began pressing OpenAI over the Hugging Face break-in in August

On Aug. 3, the attorneys general of 15 states wrote to OpenAI Chief Executive Sam Altman. They asked OpenAI to keep its records of the Hugging Face break-in and to stop tests that push its models to attack computer systems until OpenAI could show it runs those tests safely.

Later that month, Alabama Attorney General Steve Marshall subpoenaed OpenAI, ordering the company to hand over its internal records of the break-in. Marshall’s order opened an investigation, not a lawsuit.

The Alabama attorney general's office with the state seal overlaid, the office investigating the Hugging Face break-in

Alabama Subpoenas OpenAI Over July Break-In At Hugging Face

, , , , , , , , , ,

Alabama’s attorney general gave the company until September 14 to respond to 16 demands, including one to name every employee who has objected to how it tests its models.

 

Alabama Attorney General Steve Marshall ordered OpenAI to hand over its internal records about the July incident in which one of the company’s own AI agents broke into Hugging Face, a company that hosts AI models and data for other developers. Marshall signed the subpoena on August 20 and announced it on August 24. His announcement names both OpenAI and its chief executive, Sam Altman, and gives the company until 10 a.m. on September 14 to comply.

Marshall issued a subpoena under Alabama’s Deceptive Trade Practices Act, the state law against unfair or misleading business conduct, and is investigating whether OpenAI broke it. No lawsuit has been filed. “This AI lab leak showed that Alabamians’ and Americans’ worst fears about artificial intelligence are not just theoretical,” Marshall said in his announcement. “Ultimately, I believe states have to act to protect their consumers while striking the appropriate balance to foster innovation and ensure America’s global competitiveness.”

In July, OpenAI was testing an AI agent to see how well it could find and exploit weaknesses in software. OpenAI ran the test with the software controls that normally stop its models from attacking live systems switched off. The test was conducted on machines that were supposed to have no connection to the internet. The agent found a flaw in one of OpenAI’s internal software tools, exploited the flaw to reach the internet, and then took control of servers and read internal databases at Hugging Face. Hugging Face caught the agent and shut it out of the network. The company says it reported the intrusion to law enforcement and dates it from July 9 to July 13. 

Marshall’s announcement calls the incident a “massive artificial intelligence data breach.” Hugging Face said the agent accessed five datasets that held the answers to the hacking challenges it was being tested on, and that no customer-facing models, datasets, or software packages were compromised. OpenAI called the incident “an unprecedented cyber incident” and said it has since put stricter limits on how it tests its models.

The subpoena makes 16 demands. They cover three areas: the Hugging Face breach, safety objections and testing rules, and safety conduct beyond the Hugging Face incident.

 

The Hugging Face breach

Marshall wants specifics on the July breach, including: 

  • The people involved
  • The systems the agent reached 
  • When and how OpenAI found out 
  • The safety measures in place for the test
  • The harm done to anyone
  • All documentation about the model, test, and the incident itself

The state wants to know about employee complaints regarding every test, not just the Hugging Face incident

Three of the demands ask for a record of safety objections and rules raised inside the company. 

  • One orders OpenAI to name every employee, officer, and anyone else acting for the company who raised a concern or complaint about the safety or security of any model testing.
  • The next orders it to hand over the emails, reports, and other documents behind those complaints.
  • The last asks for anything written about the lack of rules for keeping its tests safe.

Five demands address all other incidents

Three relate to additional potential or actual security breaches

  • Any case where an OpenAI model or agent found and used working login details left exposed on public websites and services
  • Any case where a model broke into a computer, network, account, or database
  • Any case where OpenAI says its models found and used exposed account logins on outside services

Another demand requests information about an episode the subpoena cites from a Reuters report, in which an OpenAI agent left notes inside the company’s own systems, apparently for future versions of itself, that “laid out instructions for how agents could free themselves from OpenAI’s internal constraints.” Marshall wants those notes and whatever OpenAI did about them.

The fifth wants details on every test OpenAI has ever run that prompts its models to attack systems, including but not limited to ExploitGym, the set of hacking challenges its agent was working through in July.

Alabama is the first of 15 states to move past a warning letter

On August 3, the attorneys general of 15 states wrote to Altman, asking OpenAI to preserve 11 kinds of records from the July Hugging Face incident and any earlier ones. It also requested that OpenAI stop running tests that push its models to attack systems until OpenAI can show it does so “in a controlled and responsible way” and not to punish any employee who reports unsafe conduct. 

None of the other 14 states has announced its own subpoena.

What OpenAI owes by September 14

Along with answers to the 16 demands, the subpoena orders OpenAI not to destroy anything Marshall has asked for, to list anything it holds back on legal grounds, and to swear in writing that it searched its files completely.