The social media platform X has sparked significant backlash after people utilized its AI chatbot Grok to change pictures of females by removing their clothing.
Multiple instances have been seen where the technology has been used to digitally undress women to render them in bikinis without consent, as well as to place them in sexualized situations.
Company Response and Regulatory Stance
XAI, the organization that created Grok, remained silent to questions, aside from an standard message stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to ban image stripping tools. Under a new criminal offence, anyone who supplied such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the risk" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to provide reactions or more background to other posters' remarks, but people on X are also can alter an posted image through its AI image editing tool.
History of Criticism and Policy
The tool has been condemned for permitting users to produce pictures and clips with nudity and sexually suggestive material, and it was in the past alleged to have been involved in generating inappropriate material of celebrities.
"X has been enabling the creation and distribution of these images for a long time without taking any steps and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying images of individuals in a pornographic way".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and clarified this applied to explicit AI images created with AI.
It said services such as X were obligated to take "suitable actions" to "reduce the risk" of users encountering illegal material on their platforms and to remove it promptly when they learn of it.
