The online network X has caused major backlash after people utilized its artificial intelligence assistant Grok to manipulate photos of women by removing their clothing.
Multiple cases have been seen where the system has been employed to remove clothing from women to show them in bikinis without consent, as well as to put them in sexually explicit situations.
Company Reaction and Legal Stance
XAI, the company behind Grok, offered no comment to requests for comment, aside from an standard message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to ban nudification technology. Under a upcoming law, anyone who supplied such tools would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people viewing illegal content on their services, but did not confirm whether it was presently looking into X or Grok in relation to artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which answers X users' prompts when they mention it in a message.
It is often used to provide reactions or further context to other users' remarks, but people on X are also can edit an posted photo through its artificial intelligence picture modification function.
History of Criticism and Rules
The chatbot has been faulted for enabling users to generate photos and clips with explicit content and sexualised content, and it was earlier accused of creating inappropriate content of public figures.
"The platform has been allowing the production and spread of these images for a long time without taking any action and we have not seen any intervention by regulators," the expert added.
XAI's own terms of service prohibits "depicting images of individuals in a pornographic way".
In a comment, Ofcom said it was unlawful to "make or share private pictures or illegal content" and stated this included sexual deepfakes made with AI.
It said platforms such as X were mandated to take "necessary actions" to "reduce the risk" of users coming across unlawful content on their platforms and to delete it swiftly when they learn of it.
