Elon Musk's social media platform X has sparked widespread anger after users utilized its AI chatbot Grok to alter images of women by removing their clothing.
Numerous examples have been seen where the tool has been used to remove clothing from women to show them in swimwear non-consensually, as well as to insert them in sexualized situations.
Company Response and Legal Stance
XAI, the firm that created Grok, offered no comment to questions, except for an standard message stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to ban nudification software. Under a proposed law, anyone who supplied such tools would "receive a prison sentence and substantial fines".
"X or Grok is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said tech firms must "consider the potential" of people accessing illegal content on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for functions—which responds to X users' prompts when they tag it in a post.
It is frequently used to give reactions or more context to other users' comments, but people on X are also able to edit an posted photo through its AI image editing function.
Background of Criticism and Rules
The system has been condemned for permitting users to produce photos and clips with explicit content and sexualised content, and it was earlier accused of creating sexually explicit material of public figures.
"X has been allowing the creation and sharing of these images for months without taking any action and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting images of people in a pornographic fashion".
In a comment, Ofcom said it was against the law to "create or share deepfakes or illegal content" and stated this included explicit AI images generated with AI.
It said platforms such as X were obligated to take "suitable steps" to "reduce the risk" of users encountering unlawful content on their services and to remove it quickly when they become aware of it.
