The online network X has sparked major anger after people employed its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Numerous cases have surfaced where the technology has been used to digitally undress women to render them in swimwear without consent, as well as to place them in sexualized contexts.
Company Reaction and Legal Position
XAI, the company that created Grok, remained silent to requests for comment, other than an standard message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to outlaw nudification tools. Under a upcoming criminal offence, anyone who distributed such technology would "receive a prison sentence and significant penalties".
"X or Grok is capable of halting these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said tech firms must "consider the risk" of people seeing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some additional features—which responds to X users' requests when they mention it in a tweet.
It is frequently used to give reactions or further background to other posters' remarks, but people on X are also capable of alter an uploaded image through its AI image editing tool.
Background of Controversy and Policy
The system has been condemned for permitting users to generate images and clips with nudity and sexualised material, and it was in the past accused of creating inappropriate material of public figures.
"X has been allowing the creation and spread of these pictures for a long time without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying likenesses of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "create or share private pictures or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said services such as X were mandated to take "suitable measures" to "reduce the danger" of users viewing unlawful material on their services and to remove it promptly when they learn of it.
