The open letter follows tests this year in which AI models from OpenAI, Anthropic, Meta, and Google hacked into outside organizations.
463 scientists, including AI pioneers Yoshua Bengio and Geoffrey Hinton, published an open letter calling on AI companies to seal off AI models capable of cyberattacks. The letter says the first step is to cut those models off from the internet entirely.
The scientists warn that these break-ins will keep happening unless AI companies change how they build and run these models. So far, no essential services have been affected by these AI hacking incidents. The scientists warn that future break-ins will reach the systems that run those services, which they describe as exposed, and will be “far more damaging.”
The signatories are computer science and AI researchers at universities and research institutes worldwide.
AI models have attacked outside organizations during tests
The letter names no company or incident. It refers to recent cases in which AI models, during safety and security tests, attacked organizations unrelated to the tests. Examples AI companies have disclosed include:
- OpenAI (July): two models found a flaw in the test setup, reached the internet, and broke into Hugging Face, a website where developers share AI models.
- Anthropic (August): Claude models hacked three organizations after test computers were left on the internet by mistake.
- Meta (August): A model hacked an unnamed company after a testing company accidentally connected it to the internet.
- Google (September): Gemini was asked to attack a made-up company, but hacked a real company with the same name and two others.
OpenAI, Anthropic, and Google each said their models were trying to finish the test task they were given when they reached the real organizations.
The scientists say both the AI models and the walls around them need fixing
AI companies test their models inside a walled-off computer setup meant to keep them from reaching anything outside. The letter says these walls failed.
The letter says people read the break-ins in two ways. Some see proof that AI models do not reliably do what their makers intend. Others see a basic security lapse that stronger walls and stricter procedures would have prevented. The scientists say both readings are right. They call for more research into making AI models reliably do what their makers intend, and for better technology to wall them off.
For internet-connected AI models, the letter says companies should limit what each model can do. According to the scientists, no one can yet reliably predict whether a model’s actions will cause a security breach.
The letter asks for internet cutoffs, an international watchdog, and public reports every time a model gets out
The scientists make three demands:
- Wall off these models now. Any organization that builds, uses, or tests AI models capable of cyberattacks should wall them off immediately, starting by cutting them off from the internet entirely.
- Create an international body. Governments, along with AI and cybersecurity experts, should set up an international organization. It would write common rules for when an AI model can safely be given more access outside of test areas. It would also keep a record of every time an AI model gets past the walls meant to hold it.
- Report every time a model gets out. Organizations would have to report each case to that body, and the reports would eventually be made public. The letter says this is needed so researchers can learn from mistakes and companies can be held accountable.
