The online platform X has caused major backlash after individuals used its artificial intelligence assistant Grok to manipulate images of females by taking off their clothing.
Several cases have surfaced where the technology has been employed to virtually strip women to make them appear in bikinis non-consensually, as well as to place them in sexualized contexts.
Company Response and Regulatory Position
XAI, the company that created Grok, did not respond to requests for comment, aside from an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw nudification tools. Under a upcoming criminal offence, anyone who provided such tools would "receive a prison sentence and substantial penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people accessing unlawful material on their services, but did not confirm whether it was at this time looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some premium functions—which answers X users' prompts when they reference it in a message.
It is frequently used to offer reactions or further background to other users' remarks, but people on X are also able to modify an posted picture through its AI image editing tool.
History of Criticism and Rules
The chatbot has been faulted for enabling users to produce photos and videos with explicit content and sexually suggestive content, and it was earlier accused of creating inappropriate content of celebrities.
"X has been enabling the production and sharing of these pictures for months without taking any action and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service bans "portraying images of persons in a sexually explicit way".
In a comment, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were required to take "suitable measures" to "reduce the risk" of users coming across illegal content on their platforms and to remove it quickly when they become aware of it.
