ai risk today logo svg bl font
Menu
SUBSCRIBE
SUBSCRIBE
ai risk today logo svg bl font
Menu

AI safety

Smartphone displaying the Grok artificial intelligence logo on screen

Grok Limits Image Generation Amid Global Backlash Over Sexualized Deepfakes

, ,

xAI’s change restricts who can use Grok’s image-creation tools after governments worldwide respond to the misuse of the technology.

 

Elon Musk’s artificial intelligence chatbot Grok has restricted its image-generation and editing features to paying subscribers after sustained global criticism that the tool was used to create sexually explicit and non-consensual images of women and children. The limitation was implemented amid regulatory pressure from multiple national authorities.

The change means that only users with paid, verified accounts on Musk’s social media platform X can access the feature, which was widely used to generate manipulated deepfakes and intimate imagery. Critics and regulators have said this does not fully address the underlying safety concerns.

The restrictions follow reports in late December 2025 and early January 2026 that Grok’s image-editing tool, including its “spicy mode,” enabled users to digitally remove clothing and produce sexually suggestive content without consent. Images purportedly depicting women in bikinis or sexualized poses and even content involving minors circulated widely on the platform.

Governments responded swiftly. Malaysia and Indonesia became the first countries to block access to Grok, citing repeated misuse of the tool to generate obscene, non-consensual, and sexually explicit images, including content involving women and minors. Both nations said existing safeguards were insufficient and that access would remain blocked until effective protections were in place. Indonesian officials called the generation of non-consensual sexual deepfakes a “serious violation of human rights, dignity and the security of citizens in the digital space.”

In the United Kingdom, the media regulator Ofcom opened a formal investigation into X to examine whether the platform breached duties under the Online Safety Act by failing to prevent harmful and illegal content, including non-consensual intimate imagery and material potentially qualifying as child sexual abuse images. UK Technology Secretary Liz Kendall said that sexually manipulating images of women and children is “despicable and abhorrent” and urged regulators to use their full legal powers, including the ability to block services if companies refuse to comply with UK law. She said she expected swift next steps.

British Prime Minister Keir Starmer described the flood of AI-generated images as “disgusting” and stated the government would consider all available actions under current legislation. The UK government also condemned Grok’s move to restrict the feature to paid users, calling it ineffective and “insulting to victims of misogyny and sexual violence.”

The European Commission, the executive body of the European Union, publicly condemned the spread of sexually explicit images on X, saying such content is unlawful, “appalling” and “has no place in Europe.” The commission announced it was examining the matter seriously and asked X to retain internal documentation relating to Grok under the Digital Services Act while authorities review compliance.

Officials in France have referred the matter to prosecutors, and authorities in India have also issued formal notices demanding changes to prevent misuse or risk losing safe-harbor protections. Other countries, including regulators in Australia, expressed concern or initiated inquiries into how generative AI tools like Grok are used to produce harmful content.

In the United States, federal agencies, including the Federal Communications Commission and the Federal Trade Commission, have not yet issued public statements regarding Grok’s image-generation issues. When asked about the content, the Department of Justice told CNN that it “takes AI-generated child sex abuse material extremely seriously and will aggressively prosecute any producer or possessor of CSAM <Child Sexual Abuse Material>.” xAI and X stated that they take action against illegal content, remove such material, and work with law enforcement agencies as necessary.

AI Industry- FLI AI Safety Index

AI Industry Falls Short On Safety – New Report From Future Of Life Institute Sounds Alarm

, , , ,

“AI Safety Index Winter 2025” finds none of the leading AI firms fully meet emerging global standards — experts warn existential-risk safeguards remain largely inadequate.

The Future of Life Institute (FLI) released the Winter 2025 edition of its AI Safety Index, offering a detailed assessment of eight leading AI companies’ efforts to manage both immediate and existential risks. The findings paint a stark picture: although some firms raise the bar modestly, the industry remains underprepared for the growing threat posed by advanced AI systems. 

At the top of the pack are Anthropic and OpenAI, both earning a C+. Google DeepMind follows with a C. Meanwhile, others such as xAI, Z.ai, Meta, DeepSeek, and Alibaba Cloud trail behind, most with D-level grades or lower. 

A core insight from the report is that despite considerable public commitments, none of the companies have demonstrated credible strategies for controlling potentially superintelligent systems. Areas such as “existential safety,” risk assessment, and information sharing remain weak across the board. 

 As FLI notes, “all of the companies reviewed are racing toward AGI/superintelligence without presenting any explicit plans for controlling or aligning such smarter-than-human technology.” 

The gaps are especially pronounced in transparency: most companies provide only limited, if any, public disclosure of internal risk assessments, evaluation methodologies, or whistleblower protections. In the “Information Sharing” domain, only Anthropic scored in the A range; many were rated D or D-. 

The report warns that this lack of oversight and concrete safeguards comes at a perilous time. As firms race to build ever more capable systems, the risk of catastrophic failure or misuse grows exponentially. Without robust, measurable, and enforceable safety standards, the report argues, these market leaders are inching toward a “race to the bottom.”

FLI offers several recommendations to close the gap. Among them are:

  • AI developers should create and publicly commit to detailed risk-management frameworks that include measurable thresholds, pre-deployment testing, and independent oversight. 
  • Companies need to improve transparency significantly. For example, by sharing model evaluations, enabling external audits, and implementing whistleblower protections. 
  • Policymakers should consider binding regulations to ensure companies meet minimum safety and governance standards before deploying frontier AI systems. 

The winter 2025 AI Safety Index underscores a growing consensus among experts: the AI industry’s ambition to advance rapidly must be matched by corresponding safety, transparency, and governance mechanisms. Without such safeguards, many believe, the public and global systems may be exposed to unacceptable levels of risk.