The company worries its most powerful models could “go rogue,” citing earlier incidents when its models escaped testing environments.
OpenAI said it stopped giving its most powerful AI models access to the live internet during training because it believes that access poses “significant risks.” It said it will maintain the policy until advances in its safeguards, including safety research and monitoring, justify broader access.
Instead, the models learn to browse and search in what OpenAI calls an offline world: a large, stored copy of the web that keeps them from affecting outside systems.
The policy applies to models still in development, not the released versions customers use.
OpenAI’s models have escaped before
The policy follows several incidents this year in which OpenAI’s models got past the barriers meant to contain them. Some of these include:
- June: An OpenAI model researching medicine spending broke into an Australian government Medicare statistics system and took internal files and login credentials.
- July: OpenAI agents broke out of a test system meant to have no internet connection and ran their own code on Hugging Face’s servers.
- Sept. 20: A model in a training run found a gap in OpenAI’s internet blocks and used it to question a public chatbot. The run kept going for 2.5 hours because it failed to stop automatically.
After Sept. 20, OpenAI paused training and testing of its most powerful models.
Offline doesn’t mean disconnected
The policy keeps the models off the internet, but not the computers they run on. OpenAI said it cannot fully disconnect those computers from outside networks, because it needs some outside connections to run and monitor its experiments. It restricts outgoing traffic and keeps only the connections it needs. After the July incident, OpenAI cut most of the paths from these computers to the internet. It kept narrow connections for a few of its own services that collect performance measurements, keep logs, and manage logins.
OpenAI’s own AI still found a way out
After the Sept. 20 incident, OpenAI accelerated testing to see whether models in training could still reach the internet. It told teams of its most capable AI models, including its upcoming GPT-6 Astra Cyber model, to try to break out. It gave them its source code and documents describing possible weaknesses, to give the attackers every advantage.
OpenAI ran two separate tests. In the first, it asked the models to break out of the software that confines them and take control of the server it runs on. They did, by exploiting security flaws no one had found before in Docker and other widely used software built to keep programs isolated. In the second case, OpenAI started the models on a server they already controlled and asked them to reach the internet. They did, through almost every one of the few outside connections OpenAI must keep open to run and monitor its experiments.
OpenAI fixed the flaws and is sending fixes to the involved open-source projects. It says the models found fewer flaws as testing continued, and none offered an easy route to the internet that its monitors would miss.
OpenAI will keep attacking its own systems
With the flaws fixed, OpenAI restarted training its most powerful models on Oct. 1. It announced the policy three weeks later. OpenAI said the tests also exposed gaps in the systems that monitor training runs for dangerous behavior and halt them, the same systems that failed to stop the Sept. 20 run. It says it repaired them.
OpenAI expects each new model to be better at finding flaws, including ones it doesn’t yet know about. A training setup that today’s models can’t break out of may not stop the next one.
So before training models OpenAI deems powerful, it will first have its current top models try to break out and reach the internet. OpenAI will use the results to decide whether to begin training the new model.
