Menu
SUBSCRIBE
SUBSCRIBE
Menu

AI Risk Management

Cover art of an Illumio survey on how companies let AI make decisions, showing a person, documents, and a gear.

70% Of Companies Surveyed Let AI Make Decisions That Once Needed Human Approval, But Only Half Say They Can Track Every Action

, , , , , , , , , ,

In an Illumio survey, 49% of 1,100 IT and security managers said they can see and track every action their AI agents take, and 26% said an AI tool connected to their systems without approval.

Network security company Illumio surveyed 1,100 IT and security managers. 70% said their companies now let AI make decisions that required human approval a year ago. 49% of the managers said they can see and track every action their AI agents take.

Key takeaways

  • 70% said their companies let AI make decisions that needed human approval a year ago.
  • 84% said they can clearly see what data and systems their AI uses, but 49% said they can track every action it takes.
  • 92% said that if an attacker took over one of their AI tools, or it went beyond its job, they could stop it before it reached critical systems or data.
  • 62% said they started using AI tools before they had the safeguards they needed.
  • Asked to pick their two main ways of controlling AI, respondents most often chose guardrails built into the AI (44%), limits on what the AI can access or affect (42%), and human oversight and approval (41%).

Up to a third of respondents are comfortable letting AI act on security and IT systems without human review

32% of respondents said they are comfortable letting AI run security investigations with no one reviewing its work. 29% would let it release new code without review, and 28% would let it change the live systems their business runs on.

26% are comfortable letting AI grant or revoke employees’ access to systems without human oversight. 26% would also let it reset passwords for the administrator accounts staff use to run critical systems.

56% said their company accepts more AI-related risk than it did a year ago. 62% said they started using AI tools before they had the safeguards they needed.

Most say they can see and stop their AI, but only half say they can track every action

84% said they can clearly see what data and systems their AI uses. 92% said that if an attacker took over one of their AI tools, or it went beyond its job, they could stop it before it reached critical systems or data.

When Illumio asked what their security teams can observe, 49% said they can see and track every action an AI agent takes. Another 32% said they see most actions, 12% see only what the AI produced and not the steps it took, and 6% see little or nothing.

Over three in four had an AI-related security problem in the past year

76% of respondents said they had at least one AI-related security problem in the past year. 35% reported their AI recommended something that would have created a security risk if they’d followed it.

31% said outside attackers used AI to write phishing emails or impersonate someone to get into their systems. 28% reported an AI-related security incident or near-miss.

Respondents also reported their own AI going beyond what it was allowed:

  • 26% said an AI tool connected to their systems without security approval.
  • 24% said AI reached data it wasn’t supposed to.
  • 24% said AI took an action nobody authorized.

The report doesn’t say whether any of the problems led to financial losses or data breaches.

Companies most often control AI with built-in guardrails, access limits, and human oversight

Illumio asked respondents to name the two main ways they control what their AI can do. The most common answers were rules set inside the AI (44%), limits on what the AI can access or affect (42%), and having a person oversee and approve its work (41%). 35% selected monitoring the AI’s activity. Least common, at 25%, was giving the AI a fixed list of approved actions and blocking everything else.

IBM and G2 surveys also found companies letting AI agents act without approval and with limited oversight

In an IBM survey of 3,221 executives, 75% expect AI agents to act without a person approving each step by 2027, but 37% say their governance is built for that. In a G2 survey of 1,019 business software buyers, 14% said their company watches its AI agents in real time and flags unusual behavior.

Methodology

Illumio surveyed 1,100 IT managers, security managers, and executives at companies with at least 500 employees in 10 countries and regions, including the U.S., the UK, Japan, and Brazil.

Colorful 3D blocks and pathways illustrating NIST’s TEVV-Athlon Framework for evaluating AI systems.

NIST Proposes Framework to Help Companies Test AI in Real-World Use

, , , , , , , , , , , ,

The agency’s proposed four-stage process would let organizations build tests around the risks, goals, and setting of a particular AI system.

 

The National Institute of Standards and Technology (NIST) asked companies and other stakeholders to help shape the proposed TEVV-Athlon Framework before it is finalized.

NIST released the initial public draft on August 7 and will accept comments through October 6. The TEVV-Athlon Framework is guidance, not a new requirement.

The TEVV-Athlon Framework would give AI developers, testing organizations, and companies deciding whether to buy or deploy AI a four-stage process for testing whether a specific system can meet its intended purpose in the real-world setting.

NIST calls the process Test, Evaluation, Verification, and Validation, or TEVV. In plain terms, it is a way to test whether an AI system meets an organization’s goals while limiting negative effects.

Draft process starts with business goals

The process would first ask organizations to define what they want to test and why. They would then decide what to measure, run the tests, and review the results.

That structure could let companies tailor their testing to a system’s use, rather than relying on one standard test for every type of AI.

NIST said the approach is designed for AI systems that analyze text, images, or other data, as well as systems that can perform tasks with limited human direction.

NIST wants business input before finalizing the draft

NIST specifically invited comments from organizations that conduct AI evaluations and from organizations that use evaluation reports to make decisions about AI systems, including business leaders, procurement specialists, researchers, and technical staff.

Comments are due October 6. NIST said comments may be made public and should not include proprietary information.

Luminous digital network undergoing analysis for an article about Aon’s AI Risk Diagnostic and governance risk.

Aon Launches AI Risk Diagnostic to Assess AI Governance Risk

, , , , , , , , ,

Aon says more than 90% of potential AI coverage remains unclear, leaving companies uncertain which policies would respond when AI causes harm.

 

Aon launched AI Risk Diagnostic, a company-wide assessment tool that measures weaknesses in how organizations govern AI and connects those gaps to potential losses and insurance coverage. 

The assessment looks at how a company approves, uses, and monitors AI, then identifies where weak controls could lead to losses or insurance gaps. 

Companies complete a secure questionnaire covering how they approve, use, and monitor AI. Aon then evaluates the responses against standards and rules that include guidance from the National Institute of Standards and Technology (NIST), the European Union’s AI Act, and the International Organization for Standardization (ISO).

The assessment gives companies a company-wide view of where weak AI controls create the greatest financial and insurance exposure, helping leaders decide which risks need attention first.

 

Two views of AI risk

The diagnostic assesses a company from two directions.

The first measures how developed its AI governance, approval processes, and controls are. The second identifies where AI could create new risks or make existing risks worse.

That approach can show, for example, whether a company has rules for approving AI systems but lacks adequate monitoring after they are put into use. It can also identify which business operations face the greatest exposure if an AI system fails, produces inaccurate information, or uses sensitive data improperly.

 

Aon said the assessment can help companies explain their AI risk position to regulators, boards, and other internal stakeholders. It can also identify areas that may need immediate attention. The diagnostic does not determine whether an insurance claim will be covered.

The diagnostic is one step in Aon’s broader process for assessing and managing AI risk. Aon says it helps companies estimate the likelihood and potential cost of AI-related losses before they decide which risks to reduce, retain and/or insure.

Abstract digital network representing CFO concerns about AI governance, costs and business risk.

93% of CFOs Say Their Companies Use AI, but Only 43% Trust Their Governance

, , , , , , , ,

59% said pressure to deploy AI quickly while managing its risks is their biggest governance challenge.

Key Takeaways

  • 93% of surveyed CFOs said their companies use AI across multiple business functions, but only 43% were confident in their AI governance.
  • 59% said balancing pressure to deploy AI quickly with managing its risks was their biggest governance challenge.
  • 46% cited uncertainty or limited transparency into AI costs as their leading internal concern (e.g., usage-based pricing).
  • 33% said their chief information security officer owns AI governance, while 19% said the CFO holds primary responsibility.

Deloitte’s second-quarter 2026 CFO Signals survey found that 93% of CFOs said their companies use AI across multiple business functions. Only 43% were confident in their companies’ AI governance, while 53.5% were somewhat confident.

The survey covered 200 CFOs at North American organizations with at least $1 billion in annual revenue.

AI governance struggles to keep pace

The pressure to move quickly was the most common obstacle to stronger AI governance. Nearly 6 in 10 CFOs said balancing rapid deployment with managing risk was their biggest challenge.

Cost uncertainty or limited transparency of AI expenditures was the leading internal concern, selected by 46% of CFOs.

35% of CFOs also cited a lack of confidence in using AI for important business operations. One-quarter were concerned about AI mistakes that could harm a company’s finances or reputation.

Legal and cybersecurity risks lead external concerns

The most common external concern was potential litigation involving protected or private content, including copyright infringement or IP violations, at 43%. Cybersecurity followed at 41%.

The survey echoes other industry reports highlighting AI security concerns, including two surveys we report on: the 2026 SANS AI Survey and Darktrace’s State of AI Cybersecurity 2026 report.

The role of AI in Finance departments

AI is now in the daily work of finance departments. 51% of CFOs said their teams use it for operational tasks such as organizing meeting transcripts and drafting emails.

Another 44% use AI for financial planning and budgeting, while 41% use it to analyze financial data.

Who’s in charge of AI?

Responsibility for AI governance varies across companies. 33% of CFOs said their chief information security officer (CISO) owns AI governance, while 19% said they hold the greatest responsibility themselves.

That placed CFOs ahead of CEOs at 12%, AI governance committees at 8%, boards and board audit committees at 4.5%, and chief risk officers at 0.5%.

Abstract blue and white visualization accompanying Google's AI Control Roadmap and AI agent governance announcement.

Google DeepMind Creates Blueprint To Police Its Own AI Agents

, , , , , , , , , , , ,

The company is making its AI Control Roadmap public to help other frontier AI labs corral rogue AI agents before they become more capable and autonomous.

Google DeepMind published a new framework designed to help AI companies control and monitor their own increasingly capable AI agents, releasing what it calls an “AI Control Roadmap” as the industry races toward more autonomous systems.

The roadmap focuses on ensuring advanced AI agents continue to act as intended, even as they gain access to more tools, data and decision-making authority.

Rather than concentrating on external cyber threats, the framework examines how AI developers can prevent their own systems from behaving in unexpected or undesirable ways. Google DeepMind describes the effort as part of a broader field known as AI control, which seeks to develop safeguards capable of detecting, constraining and correcting problematic AI behavior.

The company argues that traditional oversight methods will become less effective as AI systems become more autonomous. Developers may need additional mechanisms to ensure agents remain aligned with their intended objectives.

The roadmap outlines several areas of research, including methods for monitoring AI behavior, identifying deceptive actions, limiting an agent’s ability to cause harm, and creating systems that can safely intervene when problems are detected. The company said these approaches could eventually provide a foundation for managing risks associated with highly capable AI agents.

Google is sharing its work publicly. The company said the roadmap is intended to help other frontier AI developers pursue similar safeguards and encourage greater collaboration across the industry.

Google DeepMind noted that significant research is still needed before effective AI control mechanisms can be deployed at scale. However, the company argues that work on those safeguards should begin well before the most capable AI agents become widely available.