Accentu

METR, a nonprofit that tests AI models, will publish its analysis of Anthropic’s risks, and Anthropic can require the removal of only certain information.

 

Anthropic granted a team from METR, a nonprofit that tests the most powerful AI models, access to its internal tools and information close to what Anthropic employees have. The team is checking whether Anthropic’s AI models might try to bypass the company’s controls or take harmful actions, such as unauthorized cyberattacks. METR will publish its findings without needing Anthropic’s approval.

Anthropic said some safety information can’t be made public because it could help attackers or reveal business secrets, such as how the company trains its models. Instead, METR’s team is reviewing that information internally and will tell the public what it concludes about the risks.

METR’s team sees nearly what Anthropic’s own risk staff see

Six METR employees now have Anthropic equipment, office space, and access to internal tools and systems that Anthropic says are “roughly on par” with what its own risk assessment teams use. In September, Anthropic said testers placed inside the company can watch models being trained, follow decisions about how models are built and released, and talk directly to employees.

Anthropic tells its employees to share information with the six the way they would with a colleague. Some information is off limits, including customer data, legal advice protected by attorney-client privilege, and details of how Anthropic designs its models. Anthropic said the testers may also work with other companies, so design details could leak by accident, and those details usually matter little for judging risk.

METR can publish without Anthropic’s approval

METR will publish its analysis of how much risk Anthropic’s models pose, incidents at the company, and how the company operates. Anthropic cannot veto publication. METR says it does not accept payment for its risk assessments.

Anthropic can require METR to remove information that falls into categories the two agreed on in advance, such as anything that would reveal confidential intellectual property. Anthropic decides what falls into those categories. METR can still say publicly how much access and information it received.

Background: six AI companies promised the White House outside reviews

On Sept. 29, Google, Anthropic, Meta, OpenAI, xAI, and Nvidia signed a voluntary safety agreement with President Donald Trump, called the White House Accord on Super Intelligence. Among other commitments, the companies agreed to bring in an independent outside auditor or evaluator to check whether their safeguards work.

Anthropic said it is placing testers inside the company voluntarily, and that “every frontier AI company should do the same.”

Other testers get less access for now

Anthropic said it is setting up three levels of access for outside testers. At the first level, testers look into narrow questions, get only the access Anthropic chooses to give, and need its approval to publish. At the second, they see more and can publish without approval. Anthropic can still require some information to be removed, and testers can say publicly whether they think the removals were justified. METR’s arrangement is the third level: employee-like access, with publishing rights Anthropic describes as similar to the second level’s.

SecureBio, a research group that works to prevent pandemics, has a narrower role: it is reviewing only the chemical and biological weapons chapter of a risk report Anthropic published in August.

Anthropic is also working toward a partnership with Accenture, the consulting firm, and Faculty, Accenture’s AI business. The work would include testing the models, red-teaming them (trying to break their safeguards on purpose), and checking whether the models behave as Anthropic intends. When Anthropic announced the partnership on Sept. 18, it said it would pay Accenture directly for the work.

Anthropic said it is in early talks with other organizations. Over time, it hopes to have several testers at the third level, each publishing its own analysis of the risks from Anthropic’s models and the major decisions the company makes about them. It did not say when METR will publish.