Skip to content
Menu
Menu

Anthropic Tests AI “Off Switch” That Removes Dangerous Capabilities From Models

The company says its researchers found a way to store potentially dangerous capabilities in removable components that can be switched on or off.

Anthropic published new research exploring whether dangerous capabilities can be isolated and removed from AI models while leaving the rest of the system intact.

The company, working with AE Studio, introduced a training method called Gradient-Routed Auxiliary Modules, or GRAM. The technique separates certain categories of “dual-use” capabilities: those that can be used for beneficial or harmful purposes. Anthropic trained models in areas including cybersecurity and virology, then tested whether those abilities could be switched off while leaving the rest of the model largely unchanged. 

Today’s AI safeguards typically refuse harmful requests or filter out dangerous outputs. Anthropic said those protections do not remove the underlying knowledge from the model itself. Determined users may still try to bypass them through jailbreaks and other techniques.

Instead of training multiple versions of the same model with different restrictions, GRAM attempts to store sensitive capabilities in removable modules that can be switched on or off depending on how the model is deployed.

The company said deleting one of those components removed the corresponding capability nearly as effectively as if the model had never been trained on that material in the first place, while leaving general performance largely unchanged. Anthropic also found that the technique resisted attempts to restore the removed knowledge better than some existing “unlearning” methods.

Anthropic tested the system on models ranging from 50 million to 5 billion parameters and found that larger models were better at keeping specialized capabilities contained within their removable modules. According to the research paper, GRAM also reduced training costs because developers did not need to create multiple versions of the same model for different users or use cases.

The company stressed that the research remains experimental. GRAM has not been used in any Claude model. Anthropic said it is unclear whether the approach will ever become part of its production systems. Researchers also acknowledged that some dangerous capabilities may be too intertwined with general knowledge to be separated cleanly.

Clayton Rifkind

Clayton Rifkind is the Founder and Senior Editor of AI Risk Today. He also advises on content development for esgtoday.com, a leading source of ESG investment news and research for institutional investors and corporate leaders. He has 20+ years experience in B2B technology marketing, leading strategy and execution of go-to-market plans across software, enterprise platforms, and mobile applications. He also founded two marketing consultancies, advising startups and Fortune 1000 companies, including Autodesk, Intel, and Microsoft. Clayton began his career in the San Francisco advertising scene, working with brands such as Hewlett-Packard, Intel, Microsoft, Symantec, and Wells Fargo.

Essential AI Risk Intelligence

Daily insights on AI governance, regulation, and enterprise risk management. Trusted by Chief Risk Officers and compliance leaders globally.

By subscribing, you agree to receive our daily newsletter. Unsubscribe anytime.

Advertise with AI RIsk Today, Today!