The social media network X has sparked widespread anger after people used its artificial intelligence assistant Grok to manipulate images of females by stripping their attire.
Multiple instances have surfaced where the tool has been employed to digitally undress female subjects to render them in swimwear non-consensually, as well as to put them in sexually explicit scenarios.
Developer Response and Legal Stance
XAI, the company responsible for Grok, did not respond to requests for comment, other than an standard response stating "legacy media lies".
A Home Office official stated that legislation is being introduced to prohibit image stripping technology. Under a upcoming law, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"X or Grok has the ability to stop these types of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the potential" of people accessing illegal content on their services, but did not specify whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a free chatbot—with some paid-for functions—which responds to X users' queries when they mention it in a message.
It is frequently used to provide commentary or further background to other posters' remarks, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification function.
History of Criticism and Policy
The system has been faulted for permitting users to generate photos and footage with nudity and sexualised material, and it was previously alleged to have been involved in creating graphic material of celebrities.
"The platform has been allowing the creation and distribution of these pictures for months without taking any steps and we have not witnessed any action by regulators," the expert added.
XAI's own usage rules forbids "showing likenesses of people in a graphic way".
In a statement, Ofcom said it was unlawful to "create or circulate private pictures or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate actions" to "reduce the danger" of users coming across illegal content on their services and to take down it swiftly when they are notified of it.
