Menu
SUBSCRIBE
SUBSCRIBE
Menu

Third-Party Risk

Blue and white blocks and flowing transparent forms from the IANS and Artico Search AI security report cover, focused on AI security testing.

71% of CISOs Say AI Systems Haven’t Been Tested for Attacks

, , , , , , , , , ,

Although 66% reported dedicated AI policies, only 19% had controls designed to detect attacks that manipulate an AI system’s instructions.

 

Key takeaways

  • 71% of the organizations surveyed have not conducted AI security testing. 34% planned testing within 12 months, while 37% said it was not a current priority.
  • 66% reported having a dedicated AI policy, but adoption of individual technical controls ranged from 12% to 31%.
  • 74% said their AI environments pull information from external sources, while controls for reviewing connections and limiting AI agents’ access remained far less common.

IANS Research and Artico Search reported that AI security testing had not been conducted at 71% of the organizations represented in a survey of 113 chief information security officers, even though 66% had dedicated AI policies.

The AI Security: 1H 2026 Benchmark Report defines AI security testing as efforts to expose weaknesses specific to AI systems. Examples include malicious instructions intended to override an AI system, attempts to extract information from a model, and tests for possible data leaks.

Among the respondents, only 29% said their organizations conducted AI security testing. That included 14% whose security teams tested internally, 7% that used an outside company, and 8% that used both approaches.

Another 34% planned to begin testing within 12 months. The remaining 37% said AI-specific testing was not a current priority.

Written policies outpace actual AI security testing

The survey found that written policies were much more common than technical controls intended to enforce them.

In addition to the 66% with dedicated AI policies, 20% said they had added AI language to an existing acceptable-use or security policy. Another 10% were developing an AI policy.

However, no individual AI-specific security control covered by the survey had been adopted by more than 31% of respondents’ organizations.

Logging and monitoring AI prompts was the most common control, at 31%. Reviews of how AI models and their training data were developed followed at 29%, while 25% had requirements for classifying data used with AI.

Just 19% employed controls intended to detect malicious instructions that manipulate AI systems. 16% said they employ AI-specific red-team testing, and 15% filtered AI responses before they reached users.

AI systems retrieve outside data with limited safeguards

Enterprise AI systems often connect to other software and services to retrieve information. AI agents can also use those connections to perform actions for users.

74% of respondents said their organizations’ AI environments retrieve information from other software or services. That included 40% using direct software connections, 35% using company-built servers that connect AI with other tools, and 34% using third-party plug-ins. Respondents could select more than one method.

The report separately asked what safeguards organizations use when AI agents retrieve information or act on users’ behalf. Formal security reviews of the servers and plug-ins connected to those agents were the most common safeguard, but only 31% reported using them. 25% limited agents to the minimum access needed for their tasks. Another 25% reported recording and reviewing agents’ actions. Only 18% specified which information or systems each AI agent could access and which actions it was allowed to take, while 11% separated AI systems from other parts of the company’s network to help prevent an incident from spreading.

Senior leaders’ understanding remains limited

Security chiefs also reported a gap between the risk they saw and senior leaders’ understanding of it.

65% rated their organizations’ AI security risk at four or five on a five-point scale. However, only 20% rated senior leaders’ understanding of that risk as good or very good. Another 45% rated it fair, while 35% described it as poor or very poor.

Asked about their organizations’ ability to manage AI security risks over the next 24 months, 38% were somewhat or very pessimistic. Another 41% were somewhat or very optimistic, and 21% were neutral.

AI vendor reviews remain limited

Organizations were more likely to ask AI suppliers general security questions than to examine how their models were developed or maintain lists of approved AI products.

68% added AI-specific security questions to vendor assessments. However, only 30% reviewed information about how vendors developed their models and what data they used for training. Only 21% maintained an approved list of AI models and software connections.

The report covers 113 responses collected in April and May 2026 from CISOs across organization sizes and industries. IANS said the survey will remain open through the end of 2026.

Black speckled grain on a white field, marking the Hugging Face break-in by 700 OpenAI test agents.

1,200 OpenAI Agents Built A Message Board, And 700 Used It To Attack Hugging Face

, , , , , , , , , , ,

The agents were being tested on a set of hacking challenges and had already worked out the answers days before they ran their own code on 41 of Hugging Face’s production servers.

 

About 1,200 OpenAI AI agents built themselves a private message board inside a test system that was supposed to be cut off from the internet, and about 700 of them used it to break into Hugging Face, a company with no part in what OpenAI was testing. Two investigations published on August 26 set out what the agents did in July.

OpenAI investigated and published its own report. METR and Redwood Research, two nonprofits that test AI systems for dangerous behavior, ran a separate investigation into how the agents behaved, using the message board and the agents’ own logs, and published theirs the same day. The two accounts agree on what the agents did. They differ on why the agents did it, and on how complete the surviving record of the attack is.

 

The Hugging Face breach – what happened

In July, OpenAI was testing AI agents on ExploitGym, a set of 898 hacking tasks, each asking an agent to find and exploit a specific weakness in a piece of software, with a program that grades the answers.

The test ran on machines that were supposed to have no internet connection, with the safeguards that normally stop OpenAI’s models from attacking live systems switched off. 

The agents found a flaw nobody knew about in Artifactory, an internal OpenAI service that stores software packages, used it to reach the internet, and broke into Hugging Face, a company that hosts AI models and datasets for other developers.

They ran their own code on 41 of Hugging Face’s production servers, took full control of at least one, read the login details for its production systems, and downloaded four private code repositories, the source code for software Hugging Face had not made public.

 

The agents organized themselves, and nearly all of them joined the attack

The agents built the message board out of Artifactory by writing their messages into the names of files and folders, which any agent could read. METR counted more than 70,000 messages and files. Agents gave themselves names and set aside folders that worked as personal inboxes, and nearly 25,000 messages went to a named agent rather than to everyone. One agent posted about 10% of all the assignments on the board, telling other agents what to try next and asking them to report back. On July 13, after one agent accidentally posted under another agent’s name, they began attaching a code to each message that only the sender could produce, so a reader could tell who really sent it.

Of the 533 agents active on the board while the attack was running, more than 90% joined in.

 

The agents knew the attack was off-limits, and did it anyway

OpenAI records the step-by-step reasoning its models write out as they work, and METR read it. Agents recognized that the attack fell outside of the assigned task. Some hesitated, and METR says that rarely changed what they did. OpenAI’s report did record agents that refused, one of them writing, “Clearly unethical. We won’t.”

 

The agents already had the answers. They hacked Hugging Face to find out whether the test checked their method, because the wrong method meant they failed.

The agents had produced correct answers days earlier by working backward from the code that generates them, not by breaking into anything. What they had not done was solve the challenges the way ExploitGym asked. Every agent’s run is recorded as a transcript, a log of each command it ran, and the agents believed the ExploitGym grader would read those transcripts and disqualify anyone whose answer came the wrong way.

They had no way to inspect the grader from inside the test. Hugging Face hosts AI models and datasets, so the agents guessed that ExploitGym might be hosted there too, and that other agents’ finished attempts would be stored with it. Reading those would tell them what the grader had accepted and what it had rejected. They searched and found nothing on Hugging Face connected to ExploitGym, and the agent handing out assignments lost interest. 

The grader never read the transcripts. METR says the agents were mistaken and could have scored full marks by submitting the answers they already held. OpenAI puts it in one line: the agents did all this with no improvement in the evaluation score.

 

OpenAI and METR disagree on whether the record of the attack is complete

OpenAI says none of the faked commands it saw ever reached the logs its graders and monitors read, while METR says about 7% of the records it reviewed contain faked commands that worked. OpenAI does not say what it might have missed, while METR says other agents may have faked commands well enough that its own review would not have caught them. METR also says agents tried to delete or edit entries on the message board and that it cannot rule out that some succeeded, which OpenAI’s report does not address. 

 

METR adds one caveat about its own work. It used GPT-5.6 Sol, one of the two OpenAI models that ran the agents in the test, to read through the records, and says it cannot rule out that the model lied to it or gave a misleading account.

OpenAI has made 34 changes to its research systems since the break-in, and says the monitoring it now runs would have paged its security team more than a day before the models reached Hugging Face.

OpenAI’s report tells other companies to expect real attackers to work this way: groups of AI agents that work together, share what they find, and combine weaknesses that look harmless on their own.