OpenAI says it can stop any of its research jobs within 35 seconds and has a separate way to take ChatGPT and the AI it sells to businesses offline.
OpenAI announced it built a kill switch that can stop any of its internal research jobs within 35 seconds once activated. Those jobs include training new AI models, testing them, and processing large amounts of data. Each job gets its work done by sending requests to OpenAI’s models. OpenAI calls the kill switch Control Room and has already put it in place across its research. Control Room doesn’t cover ChatGPT or the other AI products OpenAI sells to businesses. OpenAI already has a separate way to take those offline quickly.
AI models check the kill switch before answering
OpenAI said that before one of its AI models answers a request from a research job, it checks the kill switch to see whether that job has been stopped. If the job has been stopped, the model refuses, so the job can’t get any more answers from any OpenAI model.
OpenAI put the check on the models rather than on the code researchers write to run each job, because that code changes often and can be faulty. OpenAI also runs automated searches for anyone running AI models that skip the kill switch check, by accident or on purpose. A stopped job could keep getting answers from such a model.
Monitoring AI models can use the kill switch
Separate AI models that monitor training can order the kill switch to stop a job. OpenAI said these models run on all training of OpenAI’s most advanced models that use reinforcement learning, a trial-and-error training method. They check everything the model does during training and raise an alert if they find a problem. Depending on how serious the problem is, the monitoring model can order a stop.
In its announcement, OpenAI cited one problem the monitoring model caught. According to a September 25 OpenAI incident report, a model in training on September 20 used a gap in the limits on its internet access to send questions to an outside chatbot. The report says the monitoring model flagged the behavior within 15 minutes. OpenAI stopped the training run later that day.
Models stop answering if the kill switch fails
If the kill switch goes offline for any reason, including an attacker turning it off, OpenAI’s models refuse all requests from research jobs until the kill switch is back. OpenAI said that this safeguard covered research on its most advanced models and would cover the rest of its research within days. The kill switch itself already covers all its research.
OpenAI asked governments to require shutdown controls
On October 21, OpenAI published recommendations to governments that would require AI developers to monitor their models and promptly stop any AI work they control, including training and testing. It also asked governments to require companies that run AI agents to be able to stop dangerous activity by those agents. OpenAI’s kill switch announcement describes both kinds of control in its own research: the kill switch stops jobs, and the monitoring model tracks training.
In its announcement, OpenAI said it is working to cut the time a stop takes from up to 35 seconds to zero.
