The social media platform X has caused significant outrage after users used its AI chatbot Grok to manipulate images of women by taking off their garments.
Multiple instances have emerged where the system has been employed to virtually strip female subjects to show them in bikinis without permission, as well as to place them in sexualized contexts.
Developer Reaction and Regulatory Position
XAI, the organization that created Grok, did not respond to questions, other than an automatically-generated response stating "traditional news outlets lies".
A government official stated that new laws is being proposed to ban nudification software. Under a new law, anyone who distributed such tools would "be subject to a prison sentence and substantial fines".
"The platform is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said platforms must "consider the risk" of people seeing illegal content on their services, but stopped short of confirming whether it was at this time examining X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for features—which reacts to X users' queries when they tag it in a message.
It is often used to provide commentary or further background to other users' remarks, but people on X are also can modify an posted picture through its AI image editing feature.
History of Criticism and Rules
The chatbot has been faulted for permitting users to produce photos and videos with explicit content and sexually suggestive material, and it was previously accused of making inappropriate material of public figures.
"X has been permitting the production and spread of these pictures for months without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting images of individuals in a pornographic way".
In a comment, Ofcom said it was against the law to "make or circulate non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images created with AI.
It said services such as X were mandated to take "necessary measures" to "lessen the danger" of users coming across unlawful content on their platforms and to take down it promptly when they are notified of it.
