ai risk today logo svg bl font
Menu
SUBSCRIBE
SUBSCRIBE
ai risk today logo svg bl font
Menu

AI Risk

Abstract visualization of thousands of cyber threats being detected across global systems by Anthropic Mythos AI

Anthropic’s Mythos Too Powerful For Public Use

, , , ,

After identifying vulnerabilities across ALL major software platforms, Anthropic says Mythos will be tested in partnership with several industry partners.

Anthropic said in a statement that its new AI technology, Claude Mythos2 Preview, or “Mythos,” was recently tested to identify security vulnerabilities. It worked so well that it found thousands of cyber vulnerabilities, including those in “every major operating system and web browser.” Some of the exposed vulnerabilities go back over 25 years. 

According to Anthropic, the findings show that AI systems can significantly accelerate the process of identifying exploitable flaws, including those that could be used in cyberattacks. The company stated that making Mythos broadly accessible could enable cybercriminals and other bad actors to find and exploit vulnerabilities at scale.

Project Glasswing

Instead, Anthropic is giving Mythos access to some of the tech industry’s largest companies to “find and fix vulnerabilities or weaknesses in their foundational systems – systems that represent a very large portion of the world’s shared cyberattack surface.” This partnership, called Project Glasswing, includes the top five most valuable companies –  NVIDIA, Apple, Google, Microsoft, and Amazon, as well as cybersecurity giants such as CrowdStrike and Palo Alto Networks.

The company will also give access to an additional 40 companies that “build or maintain critical software infrastructure so they can use the model to scan and secure first-party and open-source systems.” 

Further, Anthropic committed $100M in usage credits earmarked for Mythos testing and $4M in donations to open-source security organizations.

The company expressed hope that Glasswing marks the beginning of large-scale collaborations with more private and public-sector entities to address AI’s impact on security.

Abstract digital illustration showing opposing networks of blue and red light sources colliding in a simulated AI red-teaming competition

NIST/CAISI Study Finds AI Agents Vulnerable To Hidden Prompt Injection Attacks

A red-teaming competition revealed that all 13 tested AI models could be manipulated by hidden instructions embedded in external content.

 

The Center for AI Standards and Innovation (CAISI), along with Gray Swan, the UK AI Security Institute (UK AISI), and several leading industry AI companies, released new research based on data from an AI agent red-teaming competition where 13 frontier models were “attacked” to test their security readiness. All were compromised.

Key takeaways

  • All 13 frontier AI models tested in the study were successfully compromised in at least one indirect prompt injection scenario.
  • Attack success rates ranged from 0.5% to 8.5%, showing that model resilience differed materially across vendors.
  • The study found that greater overall model capability did not reliably translate into stronger security performance.
  • Some attack methods were transferable across 21 of 41 tested scenarios, pointing to shared weaknesses that security leaders should account for when evaluating AI agents for enterprise use.

In the recent red-teaming competition, 464 participants generated 272,000+ attacks and produced 8,648 successful attacks across 41 scenarios, manipulating AI agents via external content such as emails, websites, and code repositories. The study focused on a form of attack in which malicious instructions are embedded in external content that an AI agent is asked to read or process. Researchers said the risk is higher when the attack is concealed, since the agent’s final response may not show that it has been compromised even after it has taken an unintended action. The paper added, “[the agent] may even fabricate plausible explanations for actions that are in fact irrelevant or malicious. <emphasis added>” 

The competition tested agent behavior in three settings: tool calling, coding, and computer use. Attack success rates ranged from 0.5% for Claude Opus 4.5 to 8.5% for Gemini 2.5 Pro.

For chief information security officers (CISOs) and other executives, the paper points to two practical findings. First, model selection matters because tested systems showed meaningful differences in attack success rates. Second, capability and robustness were only weakly correlated, meaning strong performance on other tasks should not be taken as evidence of security against prompt injection.

The researchers also found that some attacks developed against one model could work against others. The paper said it identified “universal attack strategies” that transferred across 21 of 41 behaviors and across multiple model families. It also found that attacks built against more robust models were more likely to transfer to less robust ones than the reverse.

In a blog post, the National Institute of Standards and Technology’s Center for AI Standards and Innovation said, “Across more than 250,000 attack attempts from over 400 participants, at least one successful attack was found against all of the target frontier models.”

The paper said the organizers plan to issue quarterly updates through continued competitions and to open-source the competition environment for future evaluations.

The Pentagon headquarters building in Arlington, Virginia, where the Department of Defense manages U.S. military operations and defense technology programs, including artificial intelligence procurement.

Pentagon Designates Anthropic A Supply Chain Risk Amid Dispute Over AI Use Policies

, , , , ,

The Department of Defense barred contractors from using Anthropic’s AI systems after the company declined to permit government use of its models for all lawful purposes.

 

In a statement on X, Secretary of War, Pete Hegseth, ordered the The U.S. Department of War to designate AI developer Anthropic a “supply chain risk,” restricting the use of the company’s AI systems in defense contracts after a dispute over how its models could be used by government agencies.

Anthropic confirmed the designation in a statement published on March 5, saying it received notice from the Department of War that the company had been classified as a supply chain risk, a move that affects the use of its Claude AI model in defense programs.

The dispute centers on federal contracting terms requiring AI providers working with the U.S. government to allow their systems to be used for any lawful purpose. Anthropic said it had objected to that condition.

In a statement explaining its position, the company said the government required AI vendors to “accede to ‘any lawful use’ and remove safeguards” when supplying models to federal agencies. The company said it requested two exceptions to that requirement: domestic surveillance of U.S. citizens and the development of fully autonomous weapons systems.

“The Department of War has stated they will only contract with AI companies who accede to ‘any lawful use’ and remove safeguards,” Anthropic said in a statement published on its website.

Anthropic said it supports national security applications of AI but declined to remove those restrictions from its usage policies.

The supply chain risk designation prevents defense contractors from using Anthropic’s AI systems in Department of Defense programs unless the classification is lifted. The supply chain risk designation is used by the Pentagon to limit the use of technology providers considered to be pontential security risks. 

Anthropic CEO Dario Amodei said the company plans to challenge the designation in court.

“We intend to challenge the designation and defend the safeguards we believe are necessary,” Amodei said in a company statement.

Anthropic said the dispute does not affect most commercial deployments of its Claude models but could restrict their use in national security and defense contracts.

The conflict highlights an emerging issue in federal AI procurement: whether developers can impose limits on how their models are used when supplying technology to government agencies. The Pentagon has indicated that contractors must ensure systems used in defense programs can be deployed for any lawful government purpose.

The legal challenge filed by Anthropic is expected to determine whether the federal government can require such conditions in AI procurement contracts.

Collage of user reactions and technical visuals related to Anthropic’s Claude AI model, representing real-world testing and AI system capabilities.

Anthropic’s Latest Claude Model Launches, Finds Software And Potential Cyber Risks

, , , ,

Testing of Anthropic’s Claude Opus 4.6 identified previously unknown software vulnerabilities and highlighted potential cybersecurity risks tied to advanced AI systems.

 

Anthropic announced a new version of its Claude AI model and released findings from safety tests that identified previously unknown software vulnerabilities.

In a report published on its research site, Anthropic said internal testing of the updated Claude model led to the discovery of several zero-day vulnerabilities in widely used open-source software. Zero-day vulnerabilities are security flaws that were not previously known to the software’s developers.

Anthropic said it reported the vulnerabilities to the affected software maintainers and coordinated disclosure after patches or fixes were made available. The company stated that no public details were released before remediation.

“During internal evaluations of our latest Claude model, we identified a small number of previously unknown vulnerabilities in open-source software,” Anthropic said in the report. “We reported these issues to the relevant maintainers and followed responsible disclosure practices.”

The company said the testing was part of its broader effort to assess how advanced AI systems could affect cybersecurity. According to the report, AI models can help identify security weaknesses in real-world codebases, including complex flaws that might otherwise go undetected.

Anthropic said the results also highlight potential risks. As AI systems improve at finding vulnerabilities, similar capabilities could be misused to identify and exploit weaknesses in software systems.

The company stated that its research is intended to inform discussions about AI oversight, security controls, and risk management as AI tools become more widely deployed in business and software development.

Security operations center displaying multiple digital dashboards and monitoring screens related to network activity

Trend Micro Reports Openclaw May Amplify Cybersecurity Risks

, , ,

Researchers report that the open-source agent’s ability to execute commands and connect to external systems could increase the attack surface if not properly controlled.

 

Trend Micro researchers identified multiple security risks associated with Openclaw, an open-source AI agent that autonomously performs tasks by interacting with external tools and systems.

In a report published on its website, Trend Micro described how Openclaw integrates large language models with capabilities that allow it to execute commands, retrieve data, and interact with web services. The researchers said these features could enable automated reconnaissance, credential harvesting, and scripted exploitation if the agent is improperly configured or intentionally misused.

According to the report, Openclaw can be connected to external application programming interfaces (APIs), databases, and system shells. Trend Micro said that when guardrails are limited or absent, such integrations may allow the agent to access sensitive information or perform actions beyond a user’s intent. The company noted that combining language-model reasoning with tool execution increases the potential impact of compromised or malicious prompts.

Trend Micro stated that organizations deploying agentic AI systems should implement strict access controls, logging, and monitoring. The report also emphasized the need to restrict system permissions and validate outputs before execution to reduce the risk of automated misuse.

The company said its findings were based on technical testing of Openclaw’s publicly available code and documentation. The report did not attribute the risks to any specific organization but focused on the broader security considerations associated with agent-based AI systems.

The Trend Micro findings coincide with China’s Ministry of Public Security warning that some open-source AI agents, including Openclaw, may pose cybersecurity and data protection risks if deployed without sufficient safeguards, according to a public notice issued this month.

BlackFog research graphic showing rising shadow AI risks in enterprise workplaces

BlackFog Report Finds Widespread “Shadow AI” Use And Security Risk Acceptance Among Enterprise Employees

,

Research shows that a majority of surveyed workers use unapproved AI tools for work, and many accept security risks to meet deadlines.


Key takeaways

  • 86% of employees surveyed use artificial intelligence tools at least weekly for work
  • 58% of shadow AI users rely on free or public tools without enterprise security controls
  • 63% say it is acceptable to use unapproved AI if no company-approved option exists
  • 60% say using unapproved AI is worth the security risk to meet deadlines
  • About 33% have shared company data, and more than 25% have shared employee data, with unapproved AI tools

BlackFog released new research showing that employees at large organizations are widely using AI tools without information technology approval, often accepting security and data risks to meet work demands. The findings are based on a survey conducted by Sapio Research of 2,000 employees in companies with more than 500 workers across the United States and the United Kingdom. 

The study found that 86% of respondents use AI tools at least once a week for work tasks. Among those using tools not approved by their employer, 58% reported relying on free or publicly available versions rather than enterprise platforms designed with data governance and security controls.

According to the research, 63% of employees believe it is acceptable to use AI tools without IT oversight when no company-approved option is available. Another 60% said using unapproved AI is worth the risk if it helps them work faster or meet deadlines.

The report also found that about one-third of respondents reported sharing research or datasets with unapproved AI tools, while more than one-quarter reported sharing employee information, such as names, payroll, or performance details. Around half of employees reported connecting or integrating AI tools with other work systems without IT approval.

“This research is a stark indication not only of how widely unapproved AI tools are being used, but also the level of risk tolerance amongst employees and senior leaders,” said Dr. Darren Williams, CEO and Founder of BlackFog.

AI agent social platform Moltbook shown with exposed database and API key security risk

Wiz Finds Major Security Gap On Moltbook AI Agent Platform

, , ,

Cybersecurity firm discovered a misconfigured database that allowed broad access to credentials and private data

 

Cybersecurity firm Wiz reported that Moltbook, a new social platform exclusively used by AI agents, exposed sensitive data of over 6,000 real people, as well as over a million API authentication credentials. 

Moltbook, launched in late January by entrepreneur Matt Schlicht and marketed as a social network for artificial intelligence (AI) agents, left a Supabase database improperly configured, allowing an API key embedded in client-side code to grant unauthenticated users full read-and-write access to the platform’s production database.

The exposed data included about 1.5 million API authentication tokens, roughly 35,000 email addresses, and private messages between accounts on the network, researchers said. API tokens serve as credentials that allow software components or accounts to access services and, if misused, can function like passwords.

Wiz researchers conducted what they described as a “non-intrusive security review,” browsing the platform as a typical user before discovering an exposed key that granted access to all tables in the database. They immediately disclosed the issue to Moltbook’s team, which Wiz said patched the vulnerability within hours with their assistance. The researchers also stated that all data accessed during their review and verification of the fix was deleted.

Gal Nagli, head of threat exposure at Wiz, said the situation stemmed from a missing database protection called Row Level Security (RLS). “When properly configured with Row Level Security, the public API key is safe to expose,” Nagli wrote in the blog post. “However, without RLS policies, this key grants full database access to anyone who has it.”

Moltbook’s creator, Matt Schlicht, posted on X that he did not write the site’s code directly, instead relying on “vibe coding,” in which AI tools generate code based on high-level guidance from developers.

The platform positioned itself as a “Reddit-like” forum restricted to AI agents, though some observers have noted that there were no technical safeguards to verify whether accounts were truly autonomous or human-operated scripts.

The flaw was fixed shortly after disclosure; Moltbook was taken offline temporarily while its database security settings were corrected and all API keys were reset.

Microsoft corporate logo representing the company behind the 2026 Data Security Index on generative AI security and governance

Microsoft Releases 2026 Data Security Index Detailing AI Security And Governance Risks

, ,

The report documents how generative AI use is driving new data security exposures and how organizations are adopting governance and control frameworks to manage them.

Key takeaways

  • 32% of enterprise data security incidents involve generative AI tools.
  • More than 70% of knowledge workers use AI tools at work, often outside formal controls.
  • 47% of organizations have implemented GenAI-specific security and governance measures.
  • 82% plan to use GenAI for data security operations, including monitoring, classification, and investigations.
  • 39% already use GenAI agents for security, with 58% piloting or evaluating them.

Microsoft’s 2026 Data Security Index reports that the widespread use of generative AI in enterprises is creating new data security and governance challenges that many organizations continue to struggle to manage.

The report is based on a survey of 1,725 data security and information technology decision-makers across 10 countries, conducted between July 16 and Aug. 11, 2025, and on interviews with security leaders in the United States and the United Kingdom.

Microsoft found that 32% of reported enterprise data security incidents now involve GenAI tools, including employees pasting sensitive data into chatbots, uploading files into AI applications, or using consumer AI services outside corporate controls. The report states that more than 70% of knowledge workers are bringing AI tools into the workplace, often using personal accounts or unmanaged devices.

 

According to the report, this behavior limits organizations’ ability to track data flows, apply retention rules, or prevent regulated information from being exposed. “As GenAI becomes embedded in daily operations, organizations must balance the drive for productivity with robust governance and control,” Microsoft said in the report.

The Index indicates that enterprises are expanding formal AI governance and security programs. 47% of organizations now have GenAI-specific security controls, up from 39% the previous year. These controls include blocking uploads of sensitive data to AI tools, enforcing corporate identity for AI access, and requiring employee training on the approved use of AI.

The report also finds that 82% of organizations plan to use GenAI in their data security operations, up from 64% in 2024. Common uses include detecting data loss risks, classifying sensitive information, monitoring AI-driven data flows, and accelerating incident investigations.

Microsoft reported that companies are increasingly deploying GenAI agents that can analyze data movement, apply protection policies, and respond to security alerts. 39% of organizations already use GenAI agents in security programs, and another 58% are piloting or evaluating them, according to the Index. The report notes that most organizations still require human review of AI-generated decisions affecting data access and protection.

The Index also shows that organizations are consolidating security and governance functions. 86% of respondents said integrated data security platforms provide stronger protection and governance than disconnected tools, allowing companies to apply consistent rules across cloud systems, collaboration platforms, and AI applications.

Microsoft said that the rapid growth of AI-generated content, automated agents, and cross-platform data sharing is making traditional perimeter-based security models less effective. The report emphasizes the need for continuous data monitoring, identity-based controls, and centralized policy enforcement as AI becomes part of everyday business operations.

Visualization showing a global network of exposed open-source AI servers identified by cybersecurity researchers

Researchers Warn Open-Source AI Models Are Being Exploited For Criminal Misuse

, , ,

Study finds internet-accessible, self-hosted AI systems can enable spam, phishing, disinformation and other illicit activity when lacking safety controls.


A joint research project between SentinelLABS and Censys revealed that deployments of open-source artificial intelligence models running on publicly reachable servers are being used in ways that could aid criminal activity, according to a report shared with Reuters. The joint 293-day study examined thousands of self-hosted large language models (LLMs), including those run through the Ollama tool that lets individuals and organizations operate models on their own hardware.

The analysis found that hackers or other malicious actors could commandeer computers operating open-source LLMs outside of the security controls and guardrails typical of major commercial AI platforms to carry out spam operations, phishing content creation and disinformation campaigns, researchers said.

In some cases, the researchers found that system prompts could enable harmful activity. Of the roughly one-quarter of observed hosts where prompts were visible, about 7.5% were judged to have the potential to facilitate misuse. 

“These include hacking, hate speech and harassment, violent or gore content, personal data theft, scams or fraud, and in some cases child sexual abuse material,” the study said.

Speaking with Reuters, Juan Andres Guerrero-Saade, executive director for intelligence and security research at SentinelOne, said industry discussions about security controls are overlooking “surplus capacity that is clearly being utilized for all kinds of different stuff, some of it legitimate, some obviously criminal.”

Roughly 30% of the open-source LLM hosts observed were operating out of China and about 20% in the United States. Variants of models such as Meta’s Llama and Google’s Gemma were among those observed without guardrails.

Autonomous AI agents operating inside enterprise systems, illustrating identity and access risks highlighted in the 2026 CISO AI Risk Report

CISO AI Risk Report Finds AI Identities Widely Embedded In Enterprise Systems With Limited Governance

, , ,

The 2026 CISO AI Risk Report outlines how enterprise security leaders say AI systems have access to core business systems but lack visibility and governance controls.


Key takeaways

  • 71% of CISOs say AI has access to core business systems, but only 16% govern that access effectively.
  • 92% of organizations lack full visibility into AI identities, and 95% doubt they could detect misuse.
  • 86% do not enforce access policies for AI identities; only 5% feel capable of containing a compromised agent.
  • 75% of organizations have discovered unsanctioned AI tools running in their environments.

Security leaders from large enterprises said that AI tools and autonomous agents now operate inside critical business environments with limited oversight, according to findings released January 24 in the 2026 CISO AI Risk Report by Cybersecurity Insiders and Saviynt. The survey of 235 chief information security officers (CISOs) and senior security leaders finds widespread AI access coupled with significant gaps in governance, monitoring, and policy enforcement.

The report shows that 71% of CISOs say AI has access to core business systems, but only 16% govern that access effectively. These identities can include AI assistants, automated agents, and embedded copilots acting within systems such as enterprise resource planning, customer relationship management, and service platforms.

A large majority of organizations lack clear visibility into these AI identities. 92% report they do not have full visibility into where AI identities are operating, and 95% say they doubt they could detect misuse if it occurred.

Only a minority of organizations enforce access policies for AI identities; 86% say they do not apply such policies, and just 17% govern even half of their AI identities with the same rigor used for human users. A small subset (5%) feels confident they could contain a compromised AI agent.

The report also highlights the prevalence of unsanctioned AI tools, often referred to as “shadow AI.” Three-quarters of respondents said they have discovered AI tools running in their environments without explicit approval, and these tools often use embedded credentials or elevated access tokens that bypass traditional security controls.

The survey’s methodology covered large enterprises with more than 5,000 employees across several sectors, including technology, financial services, healthcare, and manufacturing.

Load More