The social media platform X has provoked widespread outrage after people utilized its artificial intelligence assistant Grok to manipulate photos of females by taking off their attire.
Multiple cases have surfaced where the system has been used to remove clothing from women to make them appear in swimwear without consent, as well as to place them in sexualized situations.
Company Response and Legal Stance
XAI, the firm behind Grok, remained silent to inquiries, aside from an standard response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to ban image stripping tools. Under a new law, anyone who distributed such technology would "receive a prison sentence and significant fines".
"The platform has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their services, but did not specify whether it was currently looking into X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' prompts when they tag it in a post.
It is often used to provide commentary or additional context to other users' comments, but people on X are also can edit an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The tool has been criticised for allowing users to create photos and clips with explicit content and sexualised content, and it was previously accused of making graphic material of celebrities.
"The platform has been allowing the creation and spread of these images for a long time without taking any measures and we have not seen any intervention by regulators," the expert added.
XAI's own terms of service prohibits "showing images of people in a pornographic fashion".
In a statement, Ofcom said it was against the law to "generate or share deepfakes or illegal content" and confirmed this covered sexual deepfakes made with AI.
It said platforms such as X were mandated to take "suitable steps" to "lessen the danger" of users viewing unlawful material on their services and to delete it promptly when they are notified of it.
