ai risk today logo svg bl font
Menu
SUBSCRIBE
SUBSCRIBE
ai risk today logo svg bl font
Menu

Online Safety

United States Capitol building in Washington, accompanying coverage of Senate legislation affecting AI chatbots used by minors, the SAFE Kids Act.

Bipartisan Senators Propose New Rules For Children’s AI Chatbot Use

, , , , , , , , , ,

The SAFE KIDS Act would require providers to identify minor users and add safeguards intended to reduce online harms.

 

Senators John Curtis (R-Utah) and Adam Schiff (D-Calif.) introduced the Safeguarding AI Features to Ensure Kids’ Informed Digital Safety Act, or SAFE KIDS Act. If enacted, the legislation could significantly change how children and teenagers use AI chatbots while creating new compliance requirements for providers.

Under the proposal, companies would generally have to treat users as minors unless they can determine the user is at least 18 years old. Providers could use age-estimation technology or approved age signals from operating systems and app stores.

The bill would require AI chatbot providers to:

  • Assess child-safety risks before releasing a chatbot, before releasing major updates, and at least annually thereafter
  • Offer parental controls that allow parents to manage chatbot memory settings, limit usage, restrict features, and control whether a child’s information can be used to train AI systems
  • Establish safeguards to reduce risks involving self-harm, suicide, eating disorders, unlawful activity, and other harmful interactions involving minors
  • Create crisis-response procedures for situations involving potential self-harm, suicide, or threats of violence
  • Prohibit advertising to children through AI chatbots and restrict the sale or sharing of children’s personal information without parental consent
  • Undergo annual independent child-safety audits and submit reports to the Federal Trade Commission (FTC)

The FTC would be responsible for enforcing the law and could seek civil penalties against companies that violate its requirements.

The SAFE KIDS Act has been introduced in the Senate and must pass both chambers of Congress before it can be sent to the president for signature.

Canada viewed from space with an AI overlay for coverage of Bill C-34 and AI chatbot providers.

Canada Proposes New Child-Safety Law For AI Chatbot Providers

, , , , , , , , , ,

New legislation would bring AI chatbot services into Canada’s online harms framework, creating compliance obligations.

 

The Government of Canada introduced legislation that would extend new online safety requirements to AI chatbot providers, bringing AI services into a broader effort to reduce online harms affecting children.

Bill C-34, the Online Harms Act, would require social media platforms and AI chatbot services to meet new online safety requirements. While public attention has largely focused on proposed restrictions affecting youth access to social media, the legislation also establishes new expectations for organizations operating AI chatbots in Canada.

The legislation would create a new Digital Safety Commission to oversee regulated services and enforce compliance. A Digital Safety Ombudsperson would support users and help navigate complaints, while a Digital Safety Office would provide administrative support.

Under the proposal, regulated services would be required to take measures to reduce users’ exposure to harmful content and activity. The government said the legislation is intended to address harms including content that sexually victimizes children, promotes self-harm, incites violence, or facilitates the exploitation of children online.

AI chatbot services are specifically included within the scope of the government’s child-safety initiative. According to the Department of Canadian Heritage, the legislation is designed to ensure that both social media services and AI chatbots implement safeguards that better protect young users.

The legislation does not yet specify all obligations that would apply to AI chatbot providers. While the bill would bring AI chatbot services within the scope of the online harms framework and under the oversight of the proposed Digital Safety Commission, the precise compliance requirements and their impact on developers would depend on the final text of the legislation and any accompanying regulations.

Canada is the latest country to propose legislation around AI and child safety. The European Union recently finalized the EU AI Act, banning, among other things, AI-generated child sexual abuse material (CSAM). In the United States, individual states such as California and Illinois passed laws that protect minors from AI chatbots.

If adopted, the legislation would establish one of Canada’s most significant regulatory frameworks governing AI chatbot safety and child-protection obligations.