Only 15 of 10,000 models examined on Hugging Face included security details that could help users identify vulnerabilities or altered files before deployment.
Key Takeaways
- Businesses considering open-source AI models often lack the information needed to evaluate their security before connecting them to company data and systems.
- A U.K. government-commissioned review found security information in just 15 of the 10,000 most-downloaded open-source AI models it examined on Hugging Face.
- The review recommends documentation requirements for model-hosting platforms, but the U.K. government has not adopted the recommendations or created any new obligations.
A U.K. government-commissioned review found that businesses considering downloadable AI models often do not receive the security information they need to evaluate those models before deployment.
Companies may be unable to identify known vulnerabilities, confirm that downloaded model files have not been altered, or determine whether sensitive data used to train a model could later be exposed.
The review found relevant security information in only 15 of the 10,000 most-downloaded open-source AI models it examined on Hugging Face, a platform for distributing downloadable models.
The Department for Digital, Culture, Media and Sport commissioned the study. It recommends that U.K. policymakers consider documentation requirements for model-hosting platforms, but the recommendations are not government policy.
Businesses lack basic checks before deployment
Before using an open-source AI model, a company needs information about known security problems, changes made to the model files, and the data used to train it. Without that information, the company cannot fully assess what risks the model may introduce to its systems.
According to the review, most open-source AI models do not provide that information. Examining a model’s software code may not reveal whether its training data were manipulated or its files were altered after development.
Hugging Face asks developers to provide a “model card,” or description page, explaining how a model should be used, what data trained it, how it was tested, and what risks users should know about.
The researchers searched the description pages for the 10,000 most-downloaded models. They initially found 218 pages containing security-related terms, but concluded that only 15 provided information relevant to security.
Report proposes possible requirements for model platforms
The review recommends that U.K. policymakers discuss minimum documentation requirements with Hugging Face and GitHub, which distribute open-source AI models and software code.
The review singles out Hugging Face and GitHub as the main distribution platforms for open-source AI models and software code, respectively. It recommends that U.K. policymakers ask the platforms to require developers to provide basic security information before making models or code available to users.
The proposed requirements could ask the open-source hosting platforms to confirm who uploaded a model and provide a way for users to check whether its files have been altered. Platforms could also require developers to disclose known security problems before making a model available for download.
The report recommends that U.K. policymakers first ask Hugging Face and GitHub to adopt the requirements voluntarily. If the platforms do not make the changes, policymakers should consider imposing them through regulation, government purchasing conditions, or guidance for regulated industries.
The report also recommends developing an “AI Bill of Materials,” or a standard record explaining how a model was created and modified. That record could include information about the model’s training data and known vulnerabilities.
Report calls for testing whether new rules reduce security problems
The review found no research showing that model documentation reduces security incidents. The report recommends tracking the results after any requirements take effect to determine whether they help companies identify security problems before using a model.
