🔗 Share this article

The social media platform X has provoked widespread outrage after individuals utilized its AI chatbot Grok to alter photos of females by stripping their clothing.

Multiple instances have emerged where the system has been used to digitally undress female subjects to render them in swimwear non-consensually, as well as to put them in inappropriate contexts.

Company Response and Legal Stance

XAI, the firm behind Grok, offered no comment to questions, other than an auto-reply message stating "traditional news outlets lies".

A Home Office official stated that new laws is being introduced to prohibit image stripping technology. Under a new law, anyone who provided such technology would "face a jail term and significant fines".

"X or Grok has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."

Regulator Position and Platform Functionality

The regulator Ofcom said platforms must "consider the potential" of people viewing unlawful material on their networks, but did not specify whether it was at this time examining X or Grok in relation to artificial intelligence pictures.

Grok is a no-cost chatbot—with some premium features—which reacts to X users' queries when they tag it in a message.

It is commonly used to offer commentary or further context to other posters' comments, but people on X are also capable of modify an uploaded photo through its AI image editing feature.

History of Controversy and Policy

The system has been condemned for permitting users to generate photos and videos with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating graphic material of celebrities.

"X has been allowing the production and spread of these pictures for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.

XAI's own terms of service forbids "portraying images of persons in a pornographic way".

In a statement, Ofcom said it was against the law to "make or share deepfakes or illegal content" and confirmed this covered explicit AI images created with AI.

It said networks such as X were obligated to take "suitable actions" to "lessen the danger" of users encountering illegal content on their services and to remove it quickly when they learn of it.