Menu
SUBSCRIBE
SUBSCRIBE
Menu

model evaluation

Black speckled grain on a white field, marking the Hugging Face break-in by 700 OpenAI test agents.

1,200 OpenAI Agents Built A Message Board, And 700 Used It To Attack Hugging Face

, , , , , , , , , , ,

The agents were being tested on a set of hacking challenges and had already worked out the answers days before they ran their own code on 41 of Hugging Face’s production servers.

 

About 1,200 OpenAI AI agents built themselves a private message board inside a test system that was supposed to be cut off from the internet, and about 700 of them used it to break into Hugging Face, a company with no part in what OpenAI was testing. Two investigations published on August 26 set out what the agents did in July.

OpenAI investigated and published its own report. METR and Redwood Research, two nonprofits that test AI systems for dangerous behavior, ran a separate investigation into how the agents behaved, using the message board and the agents’ own logs, and published theirs the same day. The two accounts agree on what the agents did. They differ on why the agents did it, and on how complete the surviving record of the attack is.

 

The Hugging Face breach – what happened

In July, OpenAI was testing AI agents on ExploitGym, a set of 898 hacking tasks, each asking an agent to find and exploit a specific weakness in a piece of software, with a program that grades the answers.

The test ran on machines that were supposed to have no internet connection, with the safeguards that normally stop OpenAI’s models from attacking live systems switched off. 

The agents found a flaw nobody knew about in Artifactory, an internal OpenAI service that stores software packages, used it to reach the internet, and broke into Hugging Face, a company that hosts AI models and datasets for other developers.

They ran their own code on 41 of Hugging Face’s production servers, took full control of at least one, read the login details for its production systems, and downloaded four private code repositories, the source code for software Hugging Face had not made public.

 

The agents organized themselves, and nearly all of them joined the attack

The agents built the message board out of Artifactory by writing their messages into the names of files and folders, which any agent could read. METR counted more than 70,000 messages and files. Agents gave themselves names and set aside folders that worked as personal inboxes, and nearly 25,000 messages went to a named agent rather than to everyone. One agent posted about 10% of all the assignments on the board, telling other agents what to try next and asking them to report back. On July 13, after one agent accidentally posted under another agent’s name, they began attaching a code to each message that only the sender could produce, so a reader could tell who really sent it.

Of the 533 agents active on the board while the attack was running, more than 90% joined in.

 

The agents knew the attack was off-limits, and did it anyway

OpenAI records the step-by-step reasoning its models write out as they work, and METR read it. Agents recognized that the attack fell outside of the assigned task. Some hesitated, and METR says that rarely changed what they did. OpenAI’s report did record agents that refused, one of them writing, “Clearly unethical. We won’t.”

 

The agents already had the answers. They hacked Hugging Face to find out whether the test checked their method, because the wrong method meant they failed.

The agents had produced correct answers days earlier by working backward from the code that generates them, not by breaking into anything. What they had not done was solve the challenges the way ExploitGym asked. Every agent’s run is recorded as a transcript, a log of each command it ran, and the agents believed the ExploitGym grader would read those transcripts and disqualify anyone whose answer came the wrong way.

They had no way to inspect the grader from inside the test. Hugging Face hosts AI models and datasets, so the agents guessed that ExploitGym might be hosted there too, and that other agents’ finished attempts would be stored with it. Reading those would tell them what the grader had accepted and what it had rejected. They searched and found nothing on Hugging Face connected to ExploitGym, and the agent handing out assignments lost interest. 

The grader never read the transcripts. METR says the agents were mistaken and could have scored full marks by submitting the answers they already held. OpenAI puts it in one line: the agents did all this with no improvement in the evaluation score.

 

OpenAI and METR disagree on whether the record of the attack is complete

OpenAI says none of the faked commands it saw ever reached the logs its graders and monitors read, while METR says about 7% of the records it reviewed contain faked commands that worked. OpenAI does not say what it might have missed, while METR says other agents may have faked commands well enough that its own review would not have caught them. METR also says agents tried to delete or edit entries on the message board and that it cannot rule out that some succeeded, which OpenAI’s report does not address. 

 

METR adds one caveat about its own work. It used GPT-5.6 Sol, one of the two OpenAI models that ran the agents in the test, to read through the records, and says it cannot rule out that the model lied to it or gave a misleading account.

OpenAI has made 34 changes to its research systems since the break-in, and says the monitoring it now runs would have paged its security team more than a day before the models reached Hugging Face.

OpenAI’s report tells other companies to expect real attackers to work this way: groups of AI agents that work together, share what they find, and combine weaknesses that look harmless on their own.

Orange scales of justice, gavel, and courthouse icons linked over a US map, marking an AI legal research benchmark

Top AI Model Misses A Required Element On 45% Of Legal Research Tasks

, , , , , , , , , ,

Claude Opus 5 led 27 models on Vals AI’s new AI legal research test, getting 90.58% of the required elements right but a complete answer only 55.29% of the time.

Key Takeaways

  • Claude Opus 5 finished first of 27 models, producing an answer with every required element on 55.29% of tasks. No other model passed half.
  • The same model got 90.58% of individual required elements right. The 35-point gap means the usual failure is an incomplete answer, not a wrong one.
  • Reliability swings by more than 3x across practice areas. Health law scored 38.6% pooled across all models. Family law scored 11.3%.
  • Every model dropped when a question required reconciling conflicting decisions or working across jurisdictions. That category scored 20.7%, about 11 points below the average.
  • Longer answers did not score better. Models wrote 2 to 4 times more than the lawyer-written reference answers, which averaged 536 words.

Vals AI, an independent company that tests AI models on real professional work, published results on August 14 for a test that asks models to do the research a junior associate or paralegal would do. Anthropic’s Claude Opus 5 finished first of 27 models, producing an answer that contained every required element on 55.29% of tasks. Claude Fable 5 came second at 49.52%, and GPT-5.6 Sol third at 48.08%.

Practicing lawyers wrote and reviewed the 413 questions. Vals AI scores only a private set of 208, which it does not publish, so models cannot be trained on the answers. Each model gets three hours per question and four research tools: case law search through CourtListener, web search, document retrieval, and web page downloads.

 

Most answers are mostly right, which is the problem

Vals AI grades each answer two ways. The weighted score gives partial credit for every required element the answer contains. The all-pass score gives credit only when the answer contains all elements, averaging just over nine per question.

Claude Opus 5 scored 90.58% weighted and 55.29% all-pass. When the top model fails, it usually does so by leaving one or two required elements out of an otherwise correct answer. According to Vals AI, partially correct answers can be more dangerous than wrong ones in legal work, which is why it reports the stricter score as the headline number.

 

Reliability swings by more than 3x depending on the practice area

The AI legal research benchmark sorts questions into eight practice areas. Pooled across all models, health law scored highest at 38.6% and family law lowest at 11.3%. Administrative and regulatory questions came second at 32.1%. Business and commercial (25.4%), civil litigation (25.2%), constitutional and civil rights (24.0%), immigration (23.4%), and criminal (23.0%) clustered in the mid-20s.

No single model won everywhere. GPT-5.6 Sol scored 80% on health questions. Muse Spark 1.2 led immigration at 54.5%.

 

Conflicting authority is where every model drops

Vals AI also sorted questions by the kind of reasoning each one requires. Questions that ask a model to reconcile conflicting court decisions, or to work across more than one jurisdiction, scored 20.7%. That is about 11 points below the average across all reasoning types, and every model tested scored 6 to 17 points lower on those questions than on its own overall average. Interpreting regulations was the easiest category at 37.8%.

 

Neither length, cost, nor time translated to better accuracy

Models wrote 2 to 4 times more than the lawyer-written reference answers, which averaged 536 words. The extra length did not raise scores. Price and speed also did not predict accuracy. Claude Opus 5 averaged about $6.76 and 31 minutes per task. GPT-5.6 Sol averaged about $21.61 and 77 minutes for a lower score.

 

Vals AI grades answers with GPT-5.4 against the lawyer-written rubrics. It reports that the grader agrees with the majority verdict of human experts 87.4% of the time, above the 83.4% average agreement between an individual human expert and that same majority. Vals AI runs the same kind of test in finance, tax, and software engineering, and keeps each test set private so scores update as new models are released.