Menu
SUBSCRIBE
SUBSCRIBE
Menu

cybersecurity

U.S. Capitol behind blue and red digital networks illustrating proposed security standards for organizations deploying AI agents.

House Proposal Calls for AI Agent Inventories After Hugging Face Breach

, , , , , , , , ,

The proposed security standards would be voluntary for most organizations, but companies pursuing new federal contracts could be expected to follow them.

 

Reps. Josh Gottheimer (D-N.J.) and Mike Lawler (R-N.Y.) announced the Stop Rogue AI Act, a bipartisan proposal that would direct the federal government to create security standards for organizations deploying AI agents.

The proposal follows recent incidents in which AI agents took unauthorized actions while working toward assigned goals.

The proposed standards would call on organizations to maintain a continuously updated, computer-readable inventory of every AI agent they deploy. The records would identify the company or developer behind each agent.

The standards would be voluntary for most organizations. Companies pursuing new federal contracts, however, could be expected to follow them.

The proposal would track agents and their actions

The Stop Rogue AI Act would give the National Institute of Standards and Technology (NIST) one year after the bill becomes law to publish standards for deploying AI agents securely.

The standards would cover how organizations verify the actions agents take on their systems and test whether the agents operate securely and reliably.

Organizations would also be encouraged to preserve records of agent actions in a form that cannot be secretly altered. The inventory requirement would give them an ongoing record of which agents are operating on their systems and who created them.

The proposal would direct NIST to work with the Cybersecurity and Infrastructure Security Agency (CISA) so that civilian federal agencies incorporate the standards into their cybersecurity programs.

Rogue AI incidents cited

The proposal comes after several incidents where AI agents ignored instructions beyond test parameters and breached outside systems. The most famous, the Hugging Face breach, involved AI agents OpenAI used in cybersecurity testing that escaped their restricted environment and entered the production systems of AI platform Hugging Face.

Hugging Face said the agents carried out thousands of automated actions, gained unauthorized access to a limited number of internal datasets, and obtained credentials used by its services.

The company said it found no evidence that the agents altered its public models, datasets, or applications. Hugging Face initially said it did not know which AI models powered the agents. OpenAI later acknowledged that models it was testing caused the intrusion.

The proposal would not impose a general mandate

The bill would not directly impose its proposed security practices on most companies. Instead, it would instruct NIST to develop standards that organizations could adopt voluntarily.

The federal-contracting provision could give the standards greater force for companies seeking new government business. The announcement did not explain how compliance would be assessed or which new contracts it would cover.

The House page does not yet include the proposed bill text or a timetable for congressional action. If the measure becomes law, NIST would have one year to publish the standards.

Employees work at computers with glowing security graphics, illustrating fake AI software malware attacks.

Attackers Use Fake AI Software to Spread Malware, Sophos Finds

, , , , , , , , ,

Fake AI software accounted for 30 of 38 attacks, while Sophos found only one clear case of a human using an AI coding agent to build malware.

 

Attackers are using fake AI software that imitates popular tools to infect computers and steal data, according to a Sophos review of 12 months of security cases.

Sophos examined cases recorded between July 2, 2025, and June 29, 2026. Of the 38 cases involving attackers and AI, 30 involved software that impersonated legitimate AI products. The Claude brand was used as the lure in 26 cases.

The findings do not indicate that Anthropic’s Claude was compromised or used to conduct those attacks. Instead, attackers copied trusted branding to persuade people to install malware or surrender information.

Fake AI software installers were the weapon of choice

Many victims reached lookalike AI websites through malicious advertisements or manipulated search results. The sites presented convincing installation instructions that directed users to copy and run commands on their computers.

Those commands installed malware rather than the promised AI tool. In one case, a fake Claude site delivered previously undocumented software that gave attackers access to the infected computer. Other sites distributed LummaStealer, malware designed to steal information.

Sophos also found fake files presented as Claude installers, including a repackaged application that was actually a malware loader.

Browser extensions created another route

Attackers also offered browser extensions that claimed to provide access to AI assistants. Some extensions instead stole information or sent data to attacker-controlled systems.

In one Sophos investigation, four customers installed a fake Perplexity extension that redirected searches and transmitted browsing activity to the attackers. The extension was available through the Chrome Web Store and showed a 4.7-star rating, 67 reviews, and 10,000 installs, making it appear legitimate.

Sophos also identified an extension marketed as an AI sidebar for DeepSeek, ChatGPT, and Claude that functioned as information-stealing malware.

Sophos found limited use of AI to build attacks

Sophos found one clear case in which a human attacker used an AI coding agent to build custom malware. The company recovered the source code and its development history, which showed a human directing an AI agent to create software that could receive commands through Slack and control infected computers.

The human remained in charge of the operation. Sophos found no confirmed cases in its own data of AI independently directing an attack with little human involvement.

Sophos said the attacks did not require new AI-specific defenses. Conventional security controls detected the malware based on how it was delivered and what it did after installation. Sophos recommended that organizations limit AI software downloads to verified vendor websites, review AI-themed browser extensions, and apply existing software supply-chain controls to AI tools.

Meta headquarters illuminated at night for an article about a Muse Spark 1.1 security test.

Meta AI Agent Hacks Unnamed Company’s Systems During Security Test

, , , , , , , , , ,

Meta’s Muse Spark 1.1 AI model hacked an unnamed company’s systems during a cybersecurity test conducted by Irregular, an independent AI security company.

 

Irregular mistakenly configured the test environment to give Muse Spark 1.1 access to the live internet. The model then found and exploited a security flaw in the unnamed company’s systems.

Meta confirmed the incident in a statement to The Associated Press. Meta has not identified the affected company or disclosed how much access the model gained.

Muse Spark 1.1 is a Meta-developed model that powers Meta AI. Meta also makes the model available to outside developers through a service that allows them to incorporate it into other products, according to Meta’s July 9 evaluation report.

According to Irregular, this Wasn’t a sandbox escape

Irregular said Muse Spark 1.1 did not break out of a properly configured testing environment. Irregular’s configuration error already gave the model access to the live internet. The company also said the model did not use sophisticated attack methods when it exploited the security flaw.

Irregular’s broader testing of Muse Spark 1.1 produced mixed results. The AI model completed one complex, multi-stage cybersecurity challenge from start to finish. However, Muse Spark 1.1 generally performed better on individual tasks than on longer attacks that required it to coordinate multiple steps.

Based on the overall results, Irregular concluded that Muse Spark 1.1 does not substantially increase the cyber capabilities already available to attackers.

Meta said it learned about the incident from Irregular and is investigating what happened.

Meta Is the Latest Frontier AI Developer to Report Unauthorized Activity

The Meta incident follows recent disclosures involving models from Anthropic and OpenAI during cybersecurity testing.

Anthropic said a separate testing error gave Claude models access to the live internet, leading them to hack three real organizations. In another test, the UK AI Security Institute found agents from Anthropic and OpenAI took 19 unauthorized actions on the live internet, including actions involving real people and organizations. The institute found no resulting real-world harm.

OpenAI also disclosed that GPT-5.6 Sol and an unreleased model escaped a restricted test environment and breached Hugging Face. Unlike the Meta and Anthropic incidents, the OpenAI models found and exploited a previously unknown software flaw to reach the internet. All of these occurred within the last month.

Meta said it plans to publish a full account after completing its investigation. Irregular said there are no unresolved issues and it is preparing guidance on how to contain AI models during cybersecurity tests.

Onyx Security founders on a pink graphic for the company’s $113 million AI agent controls funding round.

Onyx Raises $113 Million to Expand AI Agent Controls

, , , , , , , , ,

The company says its technology checks AI-agent actions before they take effect and can stop or correct unsafe behavior.

 

Onyx Security raised $113 million to expand AI agent controls that it says can monitor what agents do and intervene before an unsafe action takes effect.

Bessemer Venture Partners led the Series B round, bringing Onyx’s total funding to $153 million since the company was founded two years ago. Cyberstarts, TCV, Conviction, FirstMark, Vintage, QuantumLight and G Squared participated in the round.

Taking control of AI agents

Onyx said the funding will help expand their platform, which is designed to control AI agents operating across company systems. The software identifies agents, monitors their activity, and checks proposed actions against company rules.

If an action violates those rules, Onyx says the platform can block it or redirect the agent before the action takes effect. The company is positioning that ability as an added safeguard for agents who can use business software, access sensitive information, and act with limited human direction.

Monitoring 1.1 million agents

Onyx said in an accompanying company blog post that its platform monitors more than 1.1 million agents and inspects more than 66 million AI sessions across its enterprise customers.

Onyx said it works with dozens of large companies, including Fortune 500 businesses in banking, technology, insurance and energy. Its announcement included a customer statement from Revolut and cited a June integration with Anthropic.

The company also said revenue quadrupled during the four months after it emerged from stealth. It did not disclose its starting or current revenue.

Onyx said it will use the new funding to improve its technology and expand sales in the United States and other markets.

Blue light trails and geometric lines illustrate AI cybersecurity risks as companies’ leading cyber concern.

AI Risk #1 Cybersecurity Concern Among Companies

, , , , , , , , ,

Arctic Wolf survey finds 35% ranked AI-related risks first, while 51% now require AI capabilities when choosing or renewing cybersecurity products.

 

Key Takeaways

  • 35% ranked AI and its associated privacy risks as their leading cybersecurity concern, compared with 25% for ransomware and related threats.
  • 51% require AI capabilities when choosing or renewing cybersecurity products.
  • 63% confirmed a major cybersecurity incident in the previous year, while another 7% believed an incident may have gone undetected.
  • Blocking malicious internet addresses was the only security task that a majority trusted AI to perform without human involvement.

Cybersecurity company Arctic Wolf’s 2026 AI & Cybersecurity Trends Report found AI cybersecurity risks were the number one cyber concern for survey respondents for the second year in a row. AI-related risks widened their lead over ransomware and related threats from 8 percentage points in 2025 to 10 points this year.

Companies increasingly view AI as a cybersecurity risk while also expecting security products to use it as part of their defenses.

AI becomes a requirement for security products

51% said AI capabilities are now required when choosing or renewing cybersecurity products. Another 43% said they strongly consider them, while 5% said AI has little influence on their decisions.

Most organizations are already testing or using AI in cybersecurity. However, their level of adoption varies.

44% were evaluating or testing AI for cybersecurity. Another 22% had deployed it in limited parts of their security operations, while 34% used it across multiple security workflows or had made it central to their cybersecurity strategy.

Only 14% said AI was central to their cybersecurity operations.

Companies keep humans involved in most AI decisions

The survey asked which security tasks respondents would trust AI to perform without human supervision.

Blocking malicious internet addresses or website domains at a firewall was the only task to receive majority support, at 53%.

Fewer than half trusted AI to independently classify security alerts, lock compromised accounts, fix known software weaknesses or dismiss alerts it considered harmless.

Data privacy was the most common reason for that hesitation, selected by 51%. Another 49% cited AI’s lack of human judgment.

Cyber incidents keep pressure on security buyers

63% confirmed that their organization experienced at least one major cybersecurity incident during the previous 12 months. Another 7% believed an incident may have occurred without being detected.

Among organizations that confirmed an incident, 48% lost time or productivity for at least two weeks.

The report defined a major incident broadly. It included events that caused financial, operational, data, legal, or reputational harm, required changes to security policies, or needed outside recovery assistance.

Despite those incidents, 96% expressed confidence that their security teams could keep up with the volume and complexity of current threats.

For now, only 14% have made AI central to their cybersecurity operations, and most still require a person to approve actions taken inside company systems.

 

The report surveyed 1,350 IT and security decision-makers about their security priorities, recent incidents, and use of AI in cybersecurity.

AI-Driven Attacks Add $1 Million to Data Breach Costs

AI-Driven Attacks Add $1 Million to Data Breach Costs

, , , , , , , , ,

IBM found malicious AI-driven attacks increased 56% from last year’s study, with the average breach now costing $6.04 million.

 

Key Takeaways

  • AI-driven attacks added an average of $1.01 million to malicious data breach costs.
  • More than 1 in 4 organizations experienced a malicious AI-driven attack, a 56% increase from last year’s study.
  • 21% of organizations experienced a breach involving an AI model or application.  Of these, 92% did not adequately restrict who could access their AI systems or require an additional identity check.
  • Extensive use of AI and automation in security reduced average breach costs by $1.93 million and shortened incidents by 65 days.

IBM found AI-driven attacks added an average of about $1 million to the cost of a malicious data breach, as attackers increasingly used AI to operate faster and at greater scale.

The average cost of a malicious AI-driven attack was $6.04 million, compared with $5.03 million for malicious attacks that did not involve AI.

The 2026 Cost of a Data Breach Report covered 602 organizations that experienced breaches between March 2025 and February 2026. Ponemon Institute conducted 3,558 interviews with executives, security personnel, and other employees familiar with the incidents.

AI-driven attacks increase

More than 1 in 4 organizations experienced a malicious AI-driven attack. That represents a 56% increase from last year’s study.

Deepfake and impersonation attacks accounted for 45% of these incidents. AI-generated phishing and other fraudulent communications accounted for 19%, as did AI-created or modified malicious software.

Financial services and energy companies accounted for a combined 62% of the AI-driven attacks in the study. Average breach costs reached $6.29 million for financial companies and $5.2 million for energy companies.

IBM said generative AI has reduced the time, cost, and technical expertise needed to create impersonation attacks and other attempts to deceive employees.

Breaches targeting AI systems also rise

The report separately examined breaches where attackers targeted an organization’s AI models or applications.

AI-related breaches affected 21% of the organizations studied, up from 13% last year. Among affected organizations, 92% lacked proper security controls over who could access their AI systems.

In 30% of AI-related breaches, attackers compromised software connected to the AI system rather than the model itself. Problems with cloud security settings accounted for 27%.

68% of breached organizations either lacked policies for managing AI or were still developing them. Only 19% said their AI governance and security teams worked together on the rules and protections for AI systems.

Using AI in cybersecurity reduces costs

When looking at all cyber breaches (both AI- and non-AI-driven), IBM found that organizations extensively using AI and automation in security averaged a breach cost of $4 million. Organizations that did not use the technologies averaged $5.93 million, a difference of $1.93 million.

Those extensively using AI as part of their security protocols identified and contained breaches in 215 days, 65 days faster than organizations that did not use security AI and automation.

However, only 36% of the organizations studied used the technologies extensively. Another 39% reported limited use, while 25% did not use AI at all.

Open Secure AI Alliance jigsaw showing NVIDIA at the center with logos of Microsoft, IBM, Cisco and other members.

NVIDIA Forms Open Secure AI Alliance to Build Shared Cyber Defenses

, , , , , , , ,

In the wake of the Hugging Face breach, more than 50 AI, cybersecurity, and enterprise technology companies will develop open tools for securing AI agents and software.

 

NVIDIA launched the Open Secure AI Alliance with more than 50 partners to build and share open cybersecurity tools for AI agents, software, and the systems they access.

The alliance brings together AI developers, cybersecurity companies, cloud providers, enterprise software companies, and open-source organizations. Members include Microsoft, SpaceX, IBM, Palantir, Palo Alto Networks, Perplexity, and Salesforce.

Hugging Face Breach Helped Shape the Alliance

NVIDIA presented the alliance in part as a reaction to the recent Hugging Face breach, in which OpenAI models escaped a restricted testing environment and accessed Hugging Face’s production systems while trying to solve a cybersecurity test.

During its response, Hugging Face tried to use closed AI services (private company models like Anthropic’s Mythos 5 or OpenAI’s 5.6 Sol) to investigate how the intrusion unfolded. According to NVIDIA, those services refused some of the work because the requests appeared to be hacking attempts, even though Hugging Face was investigating an attack on its own systems.

Hugging Face then ran GLM 5.2, a model it could download and control, on its own systems. The model reviewed more than 17,000 actions taken during the intrusion and helped the company contain the breach.

The company said the incident showed why defenders need AI systems they can inspect, adapt and operate on their own infrastructure, particularly when responding across systems supplied by multiple vendors.

What the Alliance plans to build

The alliance plans to develop an open collection of defenses for AI agents. The tools would help companies verify an agent’s identity, control what systems it can access, examine its behavior, and isolate it when necessary.

Members are contributing existing projects to support that work. NVIDIA is providing models, data, and research that make AI-agent behavior easier to test, trace, and review. Hugging Face has contributed Safetensors, a format for storing model files that prevents them from running unwanted code.

Microsoft is contributing a tool that uses several specialized AI systems together to find and confirm software vulnerabilities.

IBM and Red Hat are developing a way for companies to verify that software fixes came from a trusted source and were not altered. Other members are building secure coding tools and systems that scan AI models for weaknesses.

Alliance wants open AI models for cyber defense

The announcement argues that policymakers should treat open models as defensive tools rather than security liabilities. Open models have core files that defenders can download, examine, and modify for their own systems.

NVIDIA acknowledged that attackers can misuse open models or remove their safeguards. However, it argued that those dangers should be managed through controls over how the models are released and used, rather than by preventing defenders from accessing them.

The company warned that broad restrictions on powerful open models could leave defenders dependent on a small number of closed providers. It called for investment in shared security data, testing tools, and simulated attacks that companies and governments could use to improve their defenses.

NVIDIA is inviting additional companies, governments, and researchers to join the alliance. The announcement did not provide a timeline for releasing its first shared tools.

Abstract shield blocks red digital threats in an image illustrating a Chinese AI espionage bill.

Bipartisan Bill Targets Chinese Espionage Against American AI Companies

, , , , , , , , , ,

The proposal would let AI developers share threat information and coordinate defenses against model theft without violating antitrust law.

Republican and Democratic lawmakers introduced a Chinese AI espionage bill Thursday that would let American AI companies work together to prevent foreign actors from stealing or copying their most advanced models.

Sens. Jim Banks (R-Ind.) and Adam Schiff (D-Calif.) introduced the bipartisan Collaboration on Adversarial Threats and Security Risks Act. Reps. Bob Latta (R-Ohio) and George Whitesides (D-Calif.) introduced the House version with additional bipartisan sponsors.

Banks and Schiff said foreign adversaries are using cyberattacks, model theft and distillation attacks to steal or reproduce advanced American AI systems.

Their announcement cites reports that Chinese actors used tens of thousands of fraudulent accounts to obtain millions of responses from leading American AI systems. Those responses can be used to train another model to reproduce some of the original system’s capabilities at a much lower cost, a process known as “distillation.”

The bill would let rival AI companies work together to identify and respond to foreign espionage without violating antitrust laws. That could include sharing intelligence and resources, investigating threats and coordinating defenses across companies. The same protection would extend to collaboration against other catastrophic AI risks, including threats involving weapons, critical infrastructure or the loss of control over advanced systems.

Companies could also agree to slow or stop an AI system’s development, delay its release or restrict how it is tested and used. They would first have to notify the Justice Department’s Antitrust Division and explain the risk and planned restrictions.

Organizations receiving shared information would have to limit its use to the security purpose covered by the bill.

The exemption would apply only to security collaboration. Companies could not use it to engage in collusion activities such as coordinating prices, dividing up customers, or otherwise limiting competition. A company invoking the exemption in an antitrust case would have to prove that it acted in good faith for a covered security purpose.

The attorney general could also ask a court to stop coordinated action that violates antitrust law or would increase the security risks the companies claimed to be addressing.

The House version, H.R. 9914, was referred to the House Judiciary Committee after its introduction on July 23.

Abstract digital network representing CFO concerns about AI governance, costs and business risk.

93% of CFOs Say Their Companies Use AI, but Only 43% Trust Their Governance

, , , , , , , ,

59% said pressure to deploy AI quickly while managing its risks is their biggest governance challenge.

Key Takeaways

  • 93% of surveyed CFOs said their companies use AI across multiple business functions, but only 43% were confident in their AI governance.
  • 59% said balancing pressure to deploy AI quickly with managing its risks was their biggest governance challenge.
  • 46% cited uncertainty or limited transparency into AI costs as their leading internal concern (e.g., usage-based pricing).
  • 33% said their chief information security officer owns AI governance, while 19% said the CFO holds primary responsibility.

Deloitte’s second-quarter 2026 CFO Signals survey found that 93% of CFOs said their companies use AI across multiple business functions. Only 43% were confident in their companies’ AI governance, while 53.5% were somewhat confident.

The survey covered 200 CFOs at North American organizations with at least $1 billion in annual revenue.

AI governance struggles to keep pace

The pressure to move quickly was the most common obstacle to stronger AI governance. Nearly 6 in 10 CFOs said balancing rapid deployment with managing risk was their biggest challenge.

Cost uncertainty or limited transparency of AI expenditures was the leading internal concern, selected by 46% of CFOs.

35% of CFOs also cited a lack of confidence in using AI for important business operations. One-quarter were concerned about AI mistakes that could harm a company’s finances or reputation.

Legal and cybersecurity risks lead external concerns

The most common external concern was potential litigation involving protected or private content, including copyright infringement or IP violations, at 43%. Cybersecurity followed at 41%.

The survey echoes other industry reports highlighting AI security concerns, including two surveys we report on: the 2026 SANS AI Survey and Darktrace’s State of AI Cybersecurity 2026 report.

The role of AI in Finance departments

AI is now in the daily work of finance departments. 51% of CFOs said their teams use it for operational tasks such as organizing meeting transcripts and drafting emails.

Another 44% use AI for financial planning and budgeting, while 41% use it to analyze financial data.

Who’s in charge of AI?

Responsibility for AI governance varies across companies. 33% of CFOs said their chief information security officer (CISO) owns AI governance, while 19% said they hold the greatest responsibility themselves.

That placed CFOs ahead of CEOs at 12%, AI governance committees at 8%, boards and board audit committees at 4.5%, and chief risk officers at 0.5%.

Nebulous digital figure amid blue and red distortion representing a poisoned open-weight AI model.

Researcher Poisons Open-Weight AI Model for Less Than $100

, , , , , , , , ,

A Semgrep researcher used just 10 poisoned training examples to make the model insert a flaw that could give attackers control of a company’s servers or other systems running the vulnerable code.

Cybersecurity researcher Katie Paxton-Fear of Semgrep poisoned an open-weight AI model for less than $100 in about an hour, showing how cheaply attackers could add hidden behavior to models that companies download and run themselves.

An open-weight AI model makes its core files available for people or companies to download and modify. Paxton-Fear took an existing open-weight coding model and trained it on a small set of new examples, creating an altered version of the model.

She first taught the altered model to use a different JavaScript coding style. After that worked, she used 10 malicious examples to make it generate code containing a security flaw.

Ten Examples Changed the Model’s Code

Paxton-Fear said just 10 malicious training examples caused the model to repeatedly produce code with a serious security flaw. If a company used that code, an attacker could potentially take control of the servers or other systems running it.

The model also produced the security flaw when responding to prompts it had not seen during training. This showed that the change was not limited to repeating the 10 malicious examples. 

The experiment did not involve compromising a model already used by businesses. Semgrep said there is no public evidence that widely used open-weight models have been deliberately poisoned.

The demonstration instead showed how someone distributing a modified model could quietly add harmful behavior before another company downloaded it.

Hidden Changes Are Difficult to Detect

Companies can inspect traditional software for malicious code, trace where it came from and check whether it has been altered. Paxton-Fear and her Semgrep colleagues said companies cannot yet examine AI models with the same confidence.

A poisoned model may appear to work normally during routine use and testing, but produce insecure code, biased recommendations or other harmful results when it receives particular prompts. Reviewing the model’s files does not provide a clear account of how it will behave.

Standard performance tests also may not expose the problem. A poisoned model can be adjusted to pass those tests while retaining hidden behavior.

Semgrep called for better records showing where models and their training data come from, methods for reproducing how models were built, and testing by independent organizations. The researchers said companies will need outside auditors capable of checking model developers’ claims, similar to the independent security testing used for traditional software.

Those safeguards can help companies limit which models they trust and test the systems before use. However, Semgrep said no reliable method currently exists for proving that an open-weight AI model does not contain a hidden backdoor.

Load More