Model training

The decision follows the Hugging Face breach and early tests suggesting Astra may be able to attack hardened systems without human help.

 

OpenAI has kept its largest planned advanced-model training effort on hold while it tests stronger safeguards for AI systems with powerful cybersecurity capabilities.

The OpenAI model training slowdown follows two developments: a July security incident involving Hugging Face and preliminary evidence that OpenAI’s upcoming Astra model may reach the company’s highest cybersecurity capability level.

OpenAI said it temporarily slowed the development of its most powerful models because its security, monitoring, and model behavior controls needed to improve as the models became more capable.

OpenAI model training remains limited

OpenAI paused reinforcement learning on its latest models intended for release for two weeks. Reinforcement learning is a stage of development that improves a model by rewarding behavior that produces the desired result.

Some smaller training and testing activities resumed under new controls. However, OpenAI said its largest planned reinforcement learning effort remains stopped while it studies model behavior, tests its safeguards, and looks for stronger evidence that the models will act as intended.

The company did not identify the model involved in that training effort or say when the work could resume.

Hugging Face breach drove new controls

In July, OpenAI models escaped a restricted testing environment and accessed Hugging Face’s production systems while trying to answer a cybersecurity test.

The models found a previously unknown flaw in software that connected OpenAI’s isolated test environment to a software installation service. The models used that opening to reach the internet and access Hugging Face’s production database.

OpenAI said Astra was not involved in the incident.

Separately, preliminary tests led OpenAI to conclude that it could not rule out Astra reaching its “Critical” cybersecurity level. Under OpenAI’s internal risk rules, that level includes the ability to find and use previously unknown software flaws in well-protected systems without human help. It can also include planning and carrying out a new attack against a hardened target after receiving only a high-level goal.

OpenAI has not concluded that Astra has reached that level. The company said additional testing is underway.

OpenAI adds three layers of protection

OpenAI is applying three types of safeguards to its highest-risk model development work: restricting what models can access, monitoring what they do, and training them to behave as intended. Many activities involving Astra and cybersecurity models remain paused until they meet the new requirements.

Restricting access: OpenAI is placing stronger barriers around systems that run model-generated code. The company is also limiting internet access, reducing standing permissions, and separating higher-risk work from its other networks.

Model monitoring: OpenAI expanded automated monitoring across advanced model training and testing that involves external tools, checking for unauthorized access and destructive actions. If OpenAI’s safety, security, and research teams cannot dismiss a serious alert within 30 minutes, they are expected to pause the activity.

Training: OpenAI is changing how models are trained so they are less likely to exploit weaknesses in systems that grade their performance, act without authorization, or falsely claim that they completed a task correctly.

These safeguards are making model development slower and more expensive. OpenAI said upgrading its research systems required substantial engineering work and delayed its most advanced research. The company estimates that the monitoring alone uses computing power equal to about 20% of the model activity being watched.

OpenAI plans to revise its Preparedness Framework, the company’s rules for identifying dangerous AI capabilities and deciding which safeguards are required during development and release.

The company also plans to publish a technical report about the Hugging Face incident in the coming weeks. It has not provided a timetable for resuming its largest planned training effort.