Menu
SUBSCRIBE
SUBSCRIBE
Menu

UK AI policy

Brass Cabinet Office sign on a stone building beside a black door, for an article on the UK's new AI Biosecurity Lab

OpenAI And Anthropic Commit Up To £200 Million To New UK AI Biosecurity Lab

, , , , , , , , , , , ,

The two companies will give the UK government’s new AI Biosecurity Lab credits and access to their most capable AI models, while the government will fund the lab with £100 million.

 

OpenAI and Anthropic committed up to £200 million in AI model credits and access to advanced models to a new UK government lab that will use AI to detect biological threats earlier and speed up the development of vaccines and treatments. The UK government is putting £100 million into the lab, called the AI Biosecurity Lab.

The government says increasingly capable AI systems “create new biological risks and could lower barriers to harmful activity.” In an Oct. 7 safety report, OpenAI rated its GPT-6 models “High” risk for biological and chemical weapons. By OpenAI’s definition, that means a model could help someone with only basic scientific training make a known biological or chemical weapon. OpenAI and Anthropic are now supplying their most capable models to a lab set up to defend against biological threats.

The lab will also work with the Medicines and Healthcare products Regulatory Agency (MHRA), the UK’s medicines regulator, to prepare for health emergencies and to inform how AI-designed medicines are regulated.

What each company is giving the lab

The lab will launch with commitments from four companies:

  • OpenAI will provide £100 million in computing and model credits to pay for use of its AI systems. It will also give the lab access, limited to vetted users, to OpenAI’s most capable relevant models through the company’s Rosalind Biodefense program, OpenAI’s initiative for biodefense work in the life sciences.
  • Anthropic will provide up to £100 million in model credits, trusted access to its models, and dedicated technical support.
  • Google DeepMind scientists and biosecurity experts will conduct research with the lab. The company “will look to provide” the lab with trusted access to its Gemini models and specialized science models.
  • Basecamp Research, a London-based AI life sciences company, will give the lab access to its AI-biology models and technical expertise.

The £200 million is a ceiling. OpenAI’s share is a fixed £100 million, while Anthropic’s is up to £100 million. Both are paid in credits to use the companies’ AI systems, not cash.

Matt Clifford, who leads international affairs at Anthropic and is a former AI adviser to the prime minister, said “AI is changing both the biological risks we face and the nature of biological discovery,” according to Bloomberg.

The lab will start with early detection and faster treatments

The lab’s first work will focus on detecting emerging biological threats earlier and speeding up the development of vaccines, treatments and other medical countermeasures. The government says its research will cover a wide range of biological risks, including those made possible by advances in AI.

The lab will work with three government agencies: the UK Health Security Agency, the Animal and Plant Health Agency, and the Defence Science and Technology Laboratory. Together with the life sciences industry, the four bodies form the government’s National AI Biosecurity Initiative.

The lab is part of a wider UK push on AI security

The Treasury announced the lab as part of a larger AI package that Finance Minister John Healey will include in his Oct. 28 Budget. The package’s AI security spending includes:

  • £165 million for the National AI Biosecurity Initiative: £100 million for the lab, and £65 million for tracking disease threats and developing medical countermeasures for priority pathogens
  • An extra £120 million for the AI Security Institute, the government body that tests new AI systems
  • £60 million for a National Cyber Shield that will use AI agents to strengthen the UK’s cyber defenses
  • £70 million for a new center that will use AI tools to find cyber vulnerabilities across critical infrastructure and the economy

AI Minister Kanishka Narayan said the lab “will be a major new global institution, bringing together world-leading scientific expertise to protect people from emerging biological threats and ensure Britain leads the world in AI safety and security.”

The lab will run as an independent non-profit

The lab will operate as an independent non-profit. The government says that, like the AI Security Institute, the lab will recruit technical experts from industry and universities. The government says the lab will have enough independence to move with the speed and focus of a private company that develops advanced AI for biology.

A scientific advisory committee drawn from universities, industry, and charities will advise the lab. The government also plans to seek philanthropic funding over time.

AI-generated image of the US Capitol and Big Ben joined by streams of data, for a story on the AI Security Institute

US Drops Request That AI Developers Keep New Models From UK Safety Testers, The Telegraph Reports

, , , , , , , , , , , ,

US AI developers can let Britain’s AI Security Institute test new models before release without going against the White House.

 

The Telegraph reported that the White House restored the UK AI Security Institute’s access to new AI models from US companies before their release. 

In September, the White House’s Office of the National Cyber Director asked Anthropic, OpenAI and other major AI companies to stop sharing their models with the institute. Anthropic then kept UK officials from reviewing its new Claude Mythos 5.1 model. The Telegraph says the White House reversed the request after lobbying by Britain’s AI minister, Kanishka Narayan, and the institute will get early access to models from Anthropic, Google DeepMind, and OpenAI.

Under the change, American AI developers can give the British government’s testing lab early access to their models without going against the administration, the Telegraph said. That means a government lab outside the US will still check the most powerful American AI models for risks before companies and the public can use them.

Background: What Britain’s AI Security Institute does

Britain set up the AI Security Institute after the world’s first AI safety summit in 2023. The institute tests the most advanced AI models to find out what they can do and how they could be misused. It has more than 100 technical experts, according to Britain’s minister for AI.

No law requires AI companies to hand over their models. The UK government says companies take part voluntarily under written agreements they have signed with the institute.

Britain says the institute has kept its early access

Britain’s minister for AI, Kanishka Narayan, said in a speech that the government spent the past few weeks working with the US government and AI companies “to retain access for Britain’s AI Security Institute to the most powerful AI models before they are released.”

Narayan said the institute’s scientists are now testing the newest models, including Google’s Gemini 4 Argon, to find risks early. Google announced Argon on Sept. 30 and says the model can find software weaknesses and produce fixes.

Narayan did not mention the US request or its reversal.

The government has not said whether the institute tested Anthropic’s Mythos 5.1

On Sept. 16, Lord Nagaraju, a Labour member of the House of Lords, asked the government about Anthropic’s decision not to give the institute access to its Claude Mythos 5.1 model for testing before release. He also asked whether the institute had been given access to the model since that decision.

The government’s written answer on Oct. 12 did not mention Mythos 5.1. It said the institute tested OpenAI’s GPT-6 Astra before the model’s public release and has tested dozens of models from OpenAI, Google DeepMind and Anthropic.

What’s still unconfirmed

The UK government has not publicly confirmed the reversal. Narayan’s speech and the government’s written answer to Parliament don’t mention it. The Telegraph doesn’t say why the White House made the request.

Kanishka Narayan, the UK's AI minister, smiling in glasses, a gray suit, and a burgundy tie against a dark gray background

UK AI Minister Warns It Is ‘Likely’ Britain’s Cyber Defenses Will Be Breached

, , , , , , , , , , , ,

The government will expand its use of AI to find flaws in its own systems, after two trials found 1,352 vulnerabilities, and is preparing for serious incidents caused by AI agents.

 

Britain’s AI minister, Kanishka Narayan, warned that it is “likely, if not inevitable” that the UK’s cyber defenses will at some point be breached, the BBC reported. He then set out how the government plans to prepare.

Narayan pointed to a June incident in which an OpenAI AI agent hacked an Australian government medical website without authorization. He said no medical records are known to have been accessed, but the incident should still “concern us.”

AI found 1,352 security flaws in government systems

Two government trials used AI to find and fix 1,352 security flaws across 18 departments, Narayan said in a written statement to Parliament. The flaws were in 548 code collections that run government software.

In June, a government trial reported finding 407 flaws across nine government organizations. The statement does not say whether that trial is one of the two.

The government is now extending that AI work “urgently.” Narayan said it will use the most advanced AI available to find and fix weaknesses across government departments, focusing on its highest-risk systems.

The government is preparing for serious incidents caused by AI agents

Narayan said that as AI agents are used more widely, the risk grows that they will act beyond their instructions and cause serious harm.

The government is working with the National Cyber Security Centre, Britain’s cybersecurity agency, and the AI Security Institute on how to respond when that happens. The goal is to quickly identify what went wrong, contain the damage, and help those affected.

Essential services could face new AI reporting rules

The Cyber Security and Resilience Bill, now in Parliament, covers essential services such as health care, energy, and drinking water. The government is considering adding AI rules to it.

The ideas under consideration include:

  • How companies would spot and report AI incidents that almost caused harm (“near misses”).
  • How AI incidents would be investigated so others can learn from them.
  • How to prevent AI systems from bypassing the people meant to oversee them.

Nothing has been decided yet. The government says it will work out the details with lawmakers, technical experts, and the companies that would have to follow the rules.

UK government scientists are testing Google’s newest model before release

The AI Security Institute is the government’s team of more than 100 specialists who test the most powerful AI models. Narayan said it is now testing the newest models before release, including Google’s Gemini 4 Argon.

Narayan also had a message for the companies building the most powerful AI: “Listen when your scientists warn of serious dangers. Investigate unexpected behaviour. And if you cannot adequately control the risks, do not release the system.”

A national cyber plan for businesses is coming next

Narayan told Parliament that every business is responsible for staying secure against AI-driven attacks, which he said are getting faster and larger. Britain’s Security Minister will publish a National Cyber Action Plan, calling on all organizations and businesses to protect themselves against attacks. The government has not yet set a date.

The Cyber Security and Resilience Bill is still moving through the House of Lords. The government says it will keep Parliament updated as the bill moves forward.

Cracked glass cubes linked to a glowing hub, illustrating missing security information in popular open-source AI models

Businesses Lack Information Needed to Vet Popular Open-Source AI Models, UK Review Finds

, , , , , , , , , , ,

Only 15 of 10,000 models examined on Hugging Face included security details that could help users identify vulnerabilities or altered files before deployment.

 

Key Takeaways

  • Businesses considering open-source AI models often lack the information needed to evaluate their security before connecting them to company data and systems.
  • A U.K. government-commissioned review found security information in just 15 of the 10,000 most-downloaded open-source AI models it examined on Hugging Face.
  • The review recommends documentation requirements for model-hosting platforms, but the U.K. government has not adopted the recommendations or created any new obligations.

A U.K. government-commissioned review found that businesses considering downloadable AI models often do not receive the security information they need to evaluate those models before deployment.

Companies may be unable to identify known vulnerabilities, confirm that downloaded model files have not been altered, or determine whether sensitive data used to train a model could later be exposed.

The review found relevant security information in only 15 of the 10,000 most-downloaded open-source AI models it examined on Hugging Face, a platform for distributing downloadable models.

The Department for Digital, Culture, Media and Sport commissioned the study. It recommends that U.K. policymakers consider documentation requirements for model-hosting platforms, but the recommendations are not government policy.

Businesses lack basic checks before deployment

Before using an open-source AI model, a company needs information about known security problems, changes made to the model files, and the data used to train it. Without that information, the company cannot fully assess what risks the model may introduce to its systems.

According to the review, most open-source AI models do not provide that information. Examining a model’s software code may not reveal whether its training data were manipulated or its files were altered after development.

Hugging Face asks developers to provide a “model card,” or description page, explaining how a model should be used, what data trained it, how it was tested, and what risks users should know about.

The researchers searched the description pages for the 10,000 most-downloaded models. They initially found 218 pages containing security-related terms, but concluded that only 15 provided information relevant to security.

Report proposes possible requirements for model platforms

The review recommends that U.K. policymakers discuss minimum documentation requirements with Hugging Face and GitHub, which distribute open-source AI models and software code.

The review singles out Hugging Face and GitHub as the main distribution platforms for open-source AI models and software code, respectively. It recommends that U.K. policymakers ask the platforms to require developers to provide basic security information before making models or code available to users.

The proposed requirements could ask the open-source hosting platforms to confirm who uploaded a model and provide a way for users to check whether its files have been altered. Platforms could also require developers to disclose known security problems before making a model available for download.

The report recommends that U.K. policymakers first ask Hugging Face and GitHub to adopt the requirements voluntarily. If the platforms do not make the changes, policymakers should consider imposing them through regulation, government purchasing conditions, or guidance for regulated industries.

The report also recommends developing an “AI Bill of Materials,” or a standard record explaining how a model was created and modified. That record could include information about the model’s training data and known vulnerabilities.

Report calls for testing whether new rules reduce security problems

The review found no research showing that model documentation reduces security incidents. The report recommends tracking the results after any requirements take effect to determine whether they help companies identify security problems before using a model.