KUALA LUMPUR, Malaysia (AP) — Malaysia and Indonesia have become the first countries to block Grok, an artificial intelligence chatbot developed by xAI, citing concerns that the tool has been used to generate sexually explicit and nonconsensual images, including content involving women and minors.
The decisions underscore growing global scrutiny of generative AI technologies capable of producing realistic images, audio and text, amid fears that safeguards are failing to prevent abuse. Grok, which is accessible through X, has drawn criticism for generating manipulated images, including depictions of women in sexualized poses and altered images of children.
Last week, Grok restricted image generation and editing features to paying users following international backlash over sexually explicit deepfakes. Regulators and critics, however, said the changes did not go far enough to address the risks.
The Associated Press sent an emailed request for comment to xAI but received an automated response stating, “Legacy Media Lies,” the same reply the outlet said it received during previous inquiries related to the backlash.
Indonesian authorities temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. Regulators in both countries said existing controls were inadequate to prevent the creation and spread of fake pornographic material.
“The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space,” said Meutya Hafid, Indonesia’s Minister of Communication and Digital Affairs.
Her ministry said the move was intended to protect women, children and the broader public from AI-generated fake pornography. Alexander Sabar, Indonesia’s director-general of digital space supervision, said preliminary findings showed Grok lacked effective safeguards to stop users from creating and sharing pornographic content based on real photos of Indonesian residents, raising concerns over privacy violations and psychological and reputational harm.
In Malaysia, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok after what it described as “repeated misuse” of the tool to generate obscene and nonconsensual manipulated images, including content involving minors.
The regulator said notices issued earlier this month to X Corp. and xAI calling for stronger protections resulted mainly in responses relying on user reporting systems. The restriction, it said, is a preventive measure and will remain in place until effective safeguards are implemented.
Broader international scrutiny
Launched in 2023, Grok is free to use on X and allows users to ask questions and generate content directly on the platform. An image-generation feature introduced last summer included a so-called “spicy mode” that enabled the creation of adult content.
The Southeast Asian bans come as Grok faces increasing scrutiny in other regions, including the European Union, the United Kingdom, India and France.
On Monday, the U.K.’s media regulator Ofcom said it had launched a formal investigation into whether Grok complies with its obligations to protect users from illegal content.
Ofcom said Grok-generated images depicting the sexualization of children or the removal of clothing from individuals could constitute pornography or child sexual abuse material under U.K. law.