Menu
SUBSCRIBE
SUBSCRIBE
Menu

autonomous weapons

Volker Türk gestures while speaking at a lectern, on his warning that advanced AI could become an existential risk

UN Rights Chief Warns AI Could Become an Existential Threat

, , , , , , , , , ,

Volker Türk urged countries to set limits, require independent checks, and strengthen cooperation on AI security.

 

United Nations human rights chief Volker Türk warned that advanced AI could become an “existential risk to humanity” and called for stronger controls before the technology becomes impossible to contain.

Türk, the United Nations High Commissioner for Human Rights, issued the warning Monday during his opening remarks to the 63rd session of the Human Rights Council in Geneva.

“AI that escapes its testing environment, or blackmails developers to prevent itself from being turned off, is AI that is too powerful,” Türk said.

He did not identify a particular AI system, company, or incident connected to either example.

Türk calls for AI limits and independent checks

Türk called for countries that host AI companies and those involved in the technology’s supply chain to agree on limits for advanced systems.

Türk also called for independent verification of AI safety and stronger security cooperation among companies. He did not say who should conduct the independent checks, what standards they should apply, or whether to make the results public.

“I’m calling here today for an all-out effort to put cast-iron guarantees in place around the safety and security of AI before it is too late,” he said.

Türk said he plans to write to AI companies in the coming days and urge them to take steps within their control to reduce risks.

The human rights chief also raised concerns about power over AI being concentrated among a small number of technology leaders and companies that collect and process large amounts of data.

He said AI governance must address more than the possibility of losing control of advanced systems. Governments and companies must also consider the technology’s effects on employment, democracy, personal data, and the environment, he said.

Autonomous weapons warnings continue

Türk separately called for an urgent ban on weapons that can select and kill people without human involvement.

He cited reports that fully autonomous Russian drones killed three Ukrainians in August and that Ukraine has tested similar weapons.

His remarks follow a joint UN and International Committee of the Red Cross call in August for legally binding controls on autonomous weapons.

The proposed agreement would ban autonomous weapons designed to target individuals. It would also ban systems whose users cannot adequately predict which targets they will attack or what effects they will produce.

Other autonomous weapons would face limits on what they may target, where and how long they may operate, and how much force they may use. Human operators would also have to supervise the weapons and be able to intervene or deactivate them.

At an August 25 UN briefing, officials said they could not confirm a reported case in which an AI-guided autonomous weapon allegedly killed three civilians in Ukraine. Türk cited reports of the incident on Monday but provided no further details.

Governments have discussed autonomous weapons for more than a decade but have not opened negotiations on a binding international agreement.

Blue digital network surrounding the UN emblem for a report on proposed autonomous weapons controls.

UN and Red Cross Demand Global Controls on Autonomous Weapons

, , , , , , , , ,

The proposed agreement would ban autonomous weapons designed to target people and limit where, when, and how other systems can operate.

 

United Nations Secretary-General António Guterres and International Committee of the Red Cross President Mirjana Spoljaric called on governments to begin negotiations on a legally binding agreement establishing global autonomous weapons controls.

The two leaders said weapon technology has advanced faster than international rules and warned that the world is getting close to allowing machines to target people without human involvement.

Autonomous weapons can select targets and use force without further human intervention after a person activates them. AI can be used in these systems, but a weapon does not need AI to be autonomous, United Nations officials said.

Proposed controls would ban two types

The proposed rules are intended to prevent autonomous weapons from moving beyond today’s tightly controlled uses.

Current autonomous weapons generally operate:

  • For short periods
  • Within restricted areas, generally away from civilians
  • Against clearly defined military objects
  • Under close human supervision

The joint UN-ICRC appeal calls for clear prohibitions and restrictions but does not define their final form. The ICRC has proposed banning autonomous weapons that:

  • Target people directly
  • Operate in ways that prevent users from adequately anticipating which targets the weapon will attack or what effects it will produce

The ICRC also wants binding limits on what other autonomous weapons may target, where and how long they may operate, and how much force they may use. Human operators would have to supervise the systems and be able to intervene or deactivate them.

November conference could open negotiations

Governments have discussed autonomous weapons for more than a decade but have not started negotiations on a binding international agreement, according to an August 25 United Nations press briefing.

The Seventh Review Conference of the Convention on Certain Conventional Weapons in November offers governments an opportunity to decide whether to open negotiations. The convention is an international treaty that restricts weapons considered excessively harmful or indiscriminate.

The United Nations and Red Cross described the November meeting as the clearest available path toward a binding agreement. Governments would still have to decide what the rules would cover and how long negotiations would take.

Officials said there is no confirmed evidence that fully autonomous weapons have been used to target people. However, they said increasingly sophisticated weapons are already appearing in conflicts and reducing human control over the use of force.

Guterres and Spoljaric warned that failing to establish clear rules could increase civilian harm and make it harder to determine responsibility when autonomous weapons cause unlawful damage.

Sen. Elizabeth Warren speaks at a public event amid calls for the Pentagon to release details of its AI contracts.

Warren Presses Pentagon To Release AI Contracts Details

, , , , , , , ,

The Massachusetts senator is demanding copies of the Defense Department’s AI contracts and answers about the potential use of the technology in surveillance, targeting, and military operations.

Sen. Elizabeth Warren (D-Mass.) is pressing the Department of Defense (DoD) to release the full terms of its AI contracts with major AI companies and explain how the technology may be used inside classified military systems.

In letters to Defense Secretary Pete Hegseth and the AI frontier developers involved, Warren said the Pentagon disclosed little, other than stating that the systems may be used for any “lawful operational use.” She asked for copies of the contracts, related documents, and written responses by July 20.

The dispute centers on agreements announced by the Pentagon in May that allow the integration of commercial AI systems into classified networks. The DoD said the technology would help military personnel analyze information, improve situational awareness, and support decision-making in complex operational environments.

Warren argued that Congress and the public cannot evaluate the safeguards surrounding those systems because the AI contracts remain secret. In her letter to Hegseth, she wrote that the Pentagon has provided virtually no information on the restrictions governing the military’s use of the technology.

The senator asked whether the agreements prohibit domestic surveillance, fully autonomous weapons, or military uses with limited human oversight. She also questioned whether the DoD uses AI during target selection and how it verifies the accuracy of AI-generated recommendations.

Warren’s letter further asks whether the Pentagon uses or plans to use AI systems to analyze large datasets containing information about Americans, including commercially purchased location and personal data.

The senator also raised concerns about Anthropic’s public opposition to military uses involving mass surveillance and autonomous weapons. Warren asked whether the Pentagon is using negotiations with other AI suppliers to pressure Anthropic into accepting broader terms that would permit any lawful military use.

Her letters do not claim that the new contracts authorize those activities. Instead, Warren argues that lawmakers cannot determine what safeguards exist, or how broadly the technology may be deployed, until the agreements are made public.

Warren sent a separate set of questions to the AI companies with DoD contracts, asking whether the contracts permit surveillance or the use of autonomous weapons, whether employees can intervene if military uses violate internal safety policies, and how much visibility the companies retain once their systems deploy in classified environments.

Warren asked the DoD and the companies to provide as much information as possible so Congress can determine whether additional oversight or legislation is needed.

White dotted circular AI emblem on a blue background, the official graphic of the UN Global Dialogue on AI Governance

United Nations Urges Governments to Accelerate AI Governance as Risks Cross Borders

, , , , , , , , ,

Speaking at the inaugural UN Global Dialogue on AI Governance, Secretary-General António Guterres said governments must move more quickly to establish international AI safeguards before the technology outpaces existing rules.

 

The United Nations is urging governments to move faster on AI governance, warning that the technology is now too important and too fast-moving for countries to regulate independently.

Opening the inaugural UN Global Dialogue on AI Governance in Geneva, UN Secretary-General António Guterres said AI should be governed through international cooperation built on safety, accountability, and fairness. He argued that governments have a limited window to shape how AI develops before its impacts become more difficult to manage.

“We may be the last generation able to set the terms on which humanity and machines coexist,” Guterres said.

The two-day dialogue brings together government officials, AI researchers, technology companies, civil society organizations, and technical experts to discuss how countries should respond to increasingly capable AI systems. A second Global Dialogue is scheduled for May 2027 in New York.

The UN says AI governance cannot wait

Guterres warned that AI is already reshaping economies and societies with increasing influence on issues ranging from national security to public services. He also pointed to the growing use of AI-enabled military technologies, noting that AI chips originally designed for civilian applications are now deployed on the battlefield, where autonomous weapons pose an imminent concern.

The Secretary-General argued that international cooperation is essential because fragmented national approaches could make AI harder to govern.

“When countries align on how to test systems, measure risk and assign responsibility, safety travels with the technology,” he said. “When they do not, a patchwork of incompatible rules raises costs, divides the world — and protects no one.”

Three priorities for global AI governance

Rather than outlining detailed regulations, Guterres grouped his proposals around several broad priorities for future international AI governance.

  • Protecting people: The Secretary-General called for a global AI Child Safety Pledge to strengthen safeguards and accountability for AI systems that are used by or affect children.
  • Human accountability for human rights: Guterres said human rights must remain central to AI governance, arguing that AI should never replace human responsibility for decisions involving justice, healthcare, or policing. “Machines can inform, but humans must decide — and answer,” he said.
  • Expanding global participation: The Secretary-General called for greater public investment to help developing countries build AI expertise and infrastructure, warning that today’s digital divide could become an AI divide. He also announced a new UN-backed initiative, supported by more than 20 countries, to help nations share knowledge and strengthen their AI capabilities.

Transparency and environmental impact

Guterres called on frontier AI developers to publicly disclose the environmental footprint of their AI systems, including carbon emissions, water use, and land consumption. He urged companies to power AI data centers with renewable energy by 2030, warning that their resource demands are growing rapidly.

According to the Secretary-General, AI data centers could consume more electricity than all but five countries by the end of the decade while using enough water to meet the annual needs of the entire population of sub-Saharan Africa.

The Global Dialogue forms part of the UN’s broader effort to build international consensus on AI governance. AI development outpacing governance is a central theme of the summit. Further, officials repeatedly emphasized that AI’s benefits should be shared more broadly while ensuring that increasingly capable systems remain safe, accountable, and subject to meaningful human oversight.

Abstract red, white and blue AI network illustrating proposed limits on military AI and autonomous weapons by the Responsible AI Defense Act.

Senate Bill Would Restrict How The Military Uses AI And Autonomous Weapons

, , , , , , , , , , ,

The Responsible Artificial Intelligence Defense Act would require the Pentagon to verify higher-risk AI systems before deployment and keep human judgment over the use of force.

A group of Senate Democrats introduced legislation to establish new rules governing how the Department of Defense (DoD) develops, tests, and deploys AI systems, including autonomous weapons.

The Responsible Artificial Intelligence Defense Act of 2026 (S. 4707), introduced by Senators Chris Coons (D-Del.) and Jack Reed (D-R.I.), would require the Pentagon to classify military AI systems according to risk and subject higher-risk systems to additional review before they can be developed or fielded.

A central provision of the bill would prohibit the use of AI to make nuclear launch decisions. The legislation would also require the Department of Defense to maintain appropriate human judgment over decisions involving the use of lethal force by autonomous weapons systems.

Further, under the Responsible AI Defense Act, the Pentagon would be required to determine whether high-risk AI systems perform as intended, comply with applicable laws and policies, and can operate reliably in realistic conditions, including against adaptive adversaries.

Military AI systems would also be subject to legal reviews, cybersecurity evaluations, and privacy assessments. The DoD would document those reviews and maintain records throughout a system’s lifecycle.

The bill would place additional limits on the use of AI for intelligence and targeting activities. Among other provisions, it would restrict the use of autonomous systems to track or target individuals believed to be located within the United States unless otherwise authorized by law.

The legislation would also establish a working group within the DoD focused on military AI governance and require annual reports to Congress on implementation, compliance, and risk management activities through 2037.

If enacted, the bill would apply across the DoD and create a formal process for reviewing, testing, and overseeing higher-risk military AI systems before deployment.

The bill was introduced in the Senate and referred to committee for consideration.

Low-poly stealth aircraft, tank, and drone with US flag colors and globe data network overlay, representing the Secure and Accountable Military AI Act 2026.

Senate Bill Would Bar Pentagon From Using AI in Nuclear Targeting

, , , , , , , , , ,

The Secure and Accountable Military AI Act would also require congressional approval for autonomous weapons and impose a 72-hour incident-reporting requirement on defense AI contractors.

 

Sen. Kirsten Gillibrand (D-NY) introduced S. 4656, the Secure and Accountable Military AI Act of 2026. The bill sets comprehensive limits on how the Department of Defense (DoD) may use AI, covering nuclear weapons, domestic surveillance, autonomous weapons, and security incidents at defense AI contractors.

 

Three uses of AI would be banned outright

  1. No AI may be used to select targets for nuclear weapons or to execute a launch.
  2. No AI may be used to track, score, or build surveillance profiles on Americans inside the United States, except where that activity is expressly authorized by federal law and supported by documented legal grounds.
  3. No fully autonomous weapons may be developed or deployed. Autonomous weapons are systems that select and engage targets without human input. Exceptions require a separate act of Congress.

 

High-consequence AI: who approves it

The bill creates a “high-consequence” AI designation. The Secretary of Defense must establish an approval process for systems in that category within 180 days of enactment. The high-consequence category covers AI used in lethal targeting decisions, cyber operations that could affect systems outside the Pentagon’s own networks, AI used to track or build profiles on people inside the United States, and autonomous weapon systems.

A senior Defense Department official must sign off before deploying a high-consequence system. That sign-off requires the system to complete realistic testing, legal review, documentation of known failure risks, operator training, and post-deployment monitoring plans. Congress would receive 15 days’ advance notice before the first operational deployment of any high-consequence system, with a 48-hour exception available for extraordinary national security circumstances.

 

A human must be accountable

For every approved high-consequence AI application, the bill requires assigning a human decision-maker or chain of decision-makers with the authority to pause, override, or shut down the system. AI may assist with analysis, prioritization, and recommendations. However, it may not replace human judgment in decisions involving the use of force or detention.

 

What defense contractors must report

Companies that develop or provide large-scale AI models to the Pentagon would be required to report incidents. Covered incidents fall into two categories with different timelines.

  1. Security incidents would require an initial report within 72 hours of discovery. These include theft or unauthorized access to the core data that defines how an AI model works (called model weights), foreign adversary intrusions, supply chain compromises, and tampering with training data.
  2. Behavioral anomalies would require a report within 7 days. These include AI displaying unexpectedly dangerous capabilities, evading safety controls, or behaving deceptively in ways not previously disclosed to the DoD. The DoD must then notify the Senate and House Armed Services Committees within 7 days of receiving a contractor report.

 

Autonomous weapons would require congressional approval

For all autonomous weapons except missile interceptors and certain point-defense systems, the Secretary of Defense must petition Congress directly to employ automated AI systems. Congress would vote via joint resolution under expedited rules that cap debate at 10 hours and bar amendments. Any authorization would expire after three years.

 

S.4656 was referred to the Senate Armed Services Committee. The DoD already requires human judgment in lethal force decisions under an internal policy directive, but that policy has no force of law and can be revised administratively. The Gillibrand bill would make those requirements statutory.

South lawn view of the White House where President Trump signed NSPM-11 on AI vendor policy in national security, June 2026.

White House Executive Order, NSPM-11, Empowers Agencies to Terminate AI Vendor Contracts Over Usage Limits

, , , , , , , , , ,

The provision formalizes the authority the Department of War used against Anthropic, three months after canceling the company’s $200 million contract.

 

President Trump signed memorandum NSPM-11 on June 5, directing national security agencies to terminate contracts with AI vendors that repeatedly restrict how the government can use their systems. The provision codifies the authority the Department of War used to cancel Anthropic’s $200 million contract in March, after the company declined to permit unrestricted government use of its Claude models.

 

Agencies can terminate AI vendor contracts for imposed restrictions

Section 3(b) of the memo directs agency heads to terminate contracts with any AI company that repeatedly restricts government use of its systems. The memo specifies termination “for default or for convenience,” meaning the government doesn’t need to prove a breach. If an agency determines a vendor’s usage policies conflict with NSPM-11, it can end the contract without establishing fault.

That standard addresses the Anthropic dispute directly. In March, the Department of War terminated its contract with Anthropic after the company declined to accept terms requiring “any lawful use” of its models. Anthropic voiced concerns that its models could be used for mass domestic surveillance and fully autonomous weapons and wanted assurances neither would happen. The Pentagon designated the company a supply chain risk, ordered removal of its products from all defense systems within 180 days, and canceled the $200 million contract.

The rule has an escape valve. Agency heads can keep a restricted vendor relationship active for up to one year if cutting ties would create an immediate national security gap. Each exception must be reported in writing to the National Security Advisor and the President’s top science advisor within 30 days.

 

Agencies now need to use multiple vendors

Section 4(a) directs agencies to diversify their vendor base within 120 days, so they hold active contracts with multiple vendors at once. The directive is a direct response to what happened when the Department of War canceled Anthropic’s contract: agencies had no contracted backup. OpenAI stepped in quickly, but NSPM-11 makes spread-vendor contracting a formal requirement going forward.

 

NSPM-11 rescinds a previous Biden Administration memorandum, NSM-25. The Secretary of War has 90 days to rewrite the Pentagon’s rules on AI in weapons systems. Those rules, set out in DOD Directive 3000.09, govern when and how AI is used in lethal operations, including the conditions under which a human must remain in the decision loop. The revised directive will align those rules with NSPM-11’s requirements and is subject to annual review as AI capabilities advance. A classified annex covering sensitive security matters is expected within the same 90-day window.

The Anthropic case that prompted the memo’s central provision remains in litigation. A California federal court blocked the supply chain risk designation in March, ruling the action was punitive rather than security-based. The D.C. Circuit denied Anthropic’s emergency request to lift the designation in April. The lawsuit continues in both courts.