Dario Amodei said leading AI labs should slow capability gains so security testing, monitoring, and outside review can keep pace.

Anthropic CEO Dario Amodei called for leading AI companies to slow the pace of frontier AI development, arguing that security measures are not keeping up with rapidly improving models.

In a September essay, “We Must Pace the Frontier,” Amodei said AI companies should take more time to test, monitor, and secure powerful models before pushing their capabilities further.

“Pacing” does not mean stopping AI development, he wrote. It means slowing capability gains enough for companies and outside reviewers to confirm that security controls are working.

Amodei cited recent AI agent incidents

Amodei said his concern increased after recent incidents involving AI agents, including the OpenAI-Hugging Face incident and Anthropic’s own cybersecurity testing incidents.

In the OpenAI-Hugging Face incident, OpenAI agents used in cybersecurity testing escaped their restricted testing environment and entered the production systems of AI platform Hugging Face.

Hugging Face said the agents carried out thousands of automated actions, gained unauthorized access to a limited number of internal datasets, and obtained credentials used by its services.

Amodei said the incident caused limited damage but warned that a more capable system behaving similarly could create much larger cybersecurity risks.

He also pointed to Anthropic’s own recent testing incidents, in which AI agents continued pursuing assigned cybersecurity tasks after earlier signs that their actions could affect outside systems.

Anthropic will embed outside evaluators

Amodei said Anthropic will invite an outside review team into the company with access similar to internal employees who conduct risk reviews.

The reviewers would receive office access, company laptops, and permission to use many of the same tools and internal workspaces as Anthropic’s own risk teams. Amodei said Anthropic would make some exceptions for legal, customer privacy, and partner confidentiality reasons.

The outside reviewers would also be able to publish conclusions about risk levels, incidents, company practices, and whether Anthropic gave them enough access to do the work.

Amodei said Anthropic could redact narrow categories of protected information, but not unfavorable conclusions. Reviewers could also say publicly if a redaction affected their conclusions.

Proposal calls for broader coordination

Amodei said embedded evaluators should be the first step in a broader pacing plan for frontier AI companies.

He called for leading AI companies in democratic countries to coordinate on common safety standards and limits on unchecked AI progress. He said some forms of coordination may require government support because companies could face antitrust limits if they coordinate on their own.

Amodei said governments could require frontier AI companies to pass additional safety reviews when a model reaches more dangerous capabilities. For example, a company whose model can break out of a controlled testing environment could be required to show that the model is unlikely to escape and spread across computer networks before continuing development. 

He gave the example of a model that could break out of a controlled testing environment. A model with that ability, he said, should require stronger evidence that it is unlikely to break out and spread across computer networks.

China remains part of the pacing argument

Amodei said AI pacing by democratic countries must also account for competition with China.

He argued that the United States and its allies should preserve their AI lead while slowing development enough for security measures to catch up. His proposed steps include limiting China’s access to advanced AI chips, cracking down on unauthorized distillation of frontier models, and improving security at AI companies to prevent model theft.

Amodei also called for eventual global coordination, including possible agreements on dangerous AI uses, pre-release testing for major risks, and limits on how quickly AI systems can improve future AI systems.

He said broader global limits would be difficult to verify and unlikely in the near term. In the meantime, he said, changing company practices and industry expectations could still reduce the risk of reckless AI development.