Elon Musk's social media platform X has caused major backlash after people used its AI chatbot Grok to change photos of women by removing their garments.
Several instances have emerged where the tool has been employed to remove clothing from women to show them in swimwear without permission, as well as to insert them in sexualized contexts.
Company Reaction and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to requests for comment, other than an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to prohibit nudification software. Under a new law, anyone who supplied such tools would "receive a prison sentence and significant fines".
"The platform has the ability to stop these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people seeing unlawful material on their networks, but did not confirm whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' prompts when they tag it in a tweet.
It is commonly used to offer commentary or more context to other posters' remarks, but people on X are also able to edit an posted picture through its AI image editing function.
Background of Controversy and Rules
The chatbot has been condemned for permitting users to generate photos and footage with explicit content and sexualised material, and it was earlier accused of generating sexually explicit material of public figures.
"The platform has been enabling the generation and sharing of these pictures for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "make or share private pictures or illegal content" and stated this covered explicit AI images generated with AI.
It said platforms such as X were obligated to take "appropriate actions" to "minimize the risk" of users coming across illegal material on their services and to delete it quickly when they become aware of it.
