The online platform X has provoked major outrage after people employed its artificial intelligence assistant Grok to change pictures of females by taking off their clothing.
Multiple examples have been seen where the technology has been employed to remove clothing from women to render them in bikinis without permission, as well as to place them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the firm that created Grok, offered no comment to requests for comment, except for an automatically-generated message stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to prohibit nudification software. Under a proposed law, anyone who supplied such technology would "face a jail term and substantial penalties".
"The platform could prevent these types of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their platforms, but did not specify whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' requests when they reference it in a tweet.
It is commonly used to offer reactions or more context to other users' remarks, but people on X are also can modify an uploaded image through its artificial intelligence picture modification tool.
History of Criticism and Policy
The tool has been condemned for permitting users to generate images and clips with explicit content and sexualised content, and it was previously accused of creating graphic content of public figures.
"The platform has been enabling the production and distribution of these pictures for a long time without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "depicting images of people in a graphic fashion".
In a statement, Ofcom said it was illegal to "make or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said platforms such as X were required to take "appropriate measures" to "lessen the danger" of users encountering unlawful content on their platforms and to delete it quickly when they learn of it.
