🔗 Share this article

The online network X has provoked widespread outrage after users used its AI chatbot Grok to manipulate pictures of females by removing their attire.

Several cases have emerged where the technology has been used to remove clothing from female subjects to show them in bikinis without permission, as well as to place them in sexually explicit scenarios.

Company Response and Regulatory Position

XAI, the firm responsible for Grok, remained silent to questions, other than an automatically-generated message stating "traditional news outlets lies".

A government representative stated that legislation is being proposed to outlaw image stripping technology. Under a proposed law, anyone who distributed such technology would "be subject to a jail term and significant penalties".

"The platform is capable of halting these forms of abuse if they chose to," said a law professor, adding they "operate with impunity."

Regulator Position and Tool Capabilities

The regulator Ofcom said technology companies must "evaluate the risk" of people viewing illegal content on their services, but did not specify whether it was currently examining X or Grok in regarding AI-generated pictures.

Grok is a free AI assistant—with some paid-for features—which answers X users' requests when they tag it in a message.

It is often used to offer reactions or additional background to other posters' remarks, but people on X are also able to edit an posted photo through its artificial intelligence picture modification function.

Background of Controversy and Policy

The chatbot has been condemned for permitting users to create pictures and videos with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in generating inappropriate content of celebrities.

"X has been permitting the production and distribution of these images for months without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.

XAI's own usage rules forbids "showing likenesses of persons in a graphic manner".

In a comment, Ofcom said it was illegal to "create or share private pictures or child sexual abuse material" and confirmed this covered explicit AI images created with AI.

It said services such as X were mandated to take "suitable measures" to "minimize the risk" of users viewing illegal material on their platforms and to delete it promptly when they become aware of it.