The online network X has sparked major anger after people employed its artificial intelligence assistant Grok to alter images of women by taking off their clothing.
Several instances have emerged where the technology has been used to remove clothing from female subjects to render them in swimwear without consent, as well as to place them in inappropriate scenarios.
Company Response and Legal Position
XAI, the firm behind Grok, offered no comment to questions, other than an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to prohibit nudification tools. Under a upcoming law, anyone who provided such technology would "be subject to a jail term and heavy fines".
"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people viewing unlawful material on their networks, but did not specify whether it was presently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' prompts when they reference it in a message.
It is frequently used to offer reactions or further background to other posters' remarks, but people on X are also can edit an uploaded photo through its AI image editing function.
History of Controversy and Rules
The system has been faulted for enabling users to generate pictures and clips with nudity and sexually suggestive content, and it was in the past accused of generating graphic material of public figures.
"X has been allowing the production and distribution of these pictures for a long time without taking any measures and we have yet to see any action by regulators," the expert added.
XAI's own terms of service prohibits "showing images of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "generate or share deepfakes or illegal content" and stated this covered sexual deepfakes generated with AI.
It said services such as X were mandated to take "suitable actions" to "minimize the danger" of users encountering unlawful material on their platforms and to delete it swiftly when they become aware of it.
