Elon Musk stated Wednesday he’s “not conscious of any bare underage photographs generated by Grok,” hours earlier than the California Legal professional Common opened an investigation into xAI’s chatbot over the “proliferation of nonconsensual sexually specific materials.”
Musk’s denial comes as stress mounts from governments worldwide — from the UK and Europe to Malaysia and Indonesia — after customers on X started asking Grok to show photographs of actual girls, and in some instances kids, into sexualized photographs with out their consent. Copyleaks, an AI detection and content material governance platform, estimated roughly one picture was posted every minute on X. A separate pattern gathered from January 5 to January 6 discovered 6,700 per hour over the 24-hour interval. (X and xAI are a part of the identical firm.)
“This materials…has been used to harass individuals throughout the web,” stated California Legal professional Common Rob Bonta in a press release. “I urge xAI to take instant motion to make sure this goes no additional.”
The AG’s workplace will examine whether or not and the way xAI violated the regulation.
A number of legal guidelines exist to guard targets of nonconsensual sexual imagery and youngster sexual abuse materials (CSAM). Final 12 months the Take It Down Act was signed right into a federal regulation, which criminalizes knowingly distributing nonconsensual intimate photographs – together with deepfakes – and requires platforms like X to take away such content material inside 48 hours. California additionally has its personal collection of legal guidelines that Gov. Gavin Newsom signed in 2024 to crack down on sexually specific deepfakes.
Grok started fulfilling consumer requests on X to provide sexualized photographs of ladies and youngsters in direction of the tip of the 12 months. The pattern seems to have taken off after sure adult-content creators prompted Grok to generate sexualized imagery of themselves as a type of advertising and marketing, which then led to different customers issuing comparable prompts. In plenty of public instances, together with well-known figures like “Stranger Issues” actress Millie Bobby Brown, Grok responded to prompts asking it to change actual photographs of actual girls by altering clothes, physique positioning, or bodily options in overtly sexual methods.
In accordance with some studies, xAI has begun implementing safeguards to deal with the problem. Grok now requires a premium subscription earlier than responding to sure image-generation requests, and even then the picture will not be generated. April Kozen, VP of promoting at Copyleaks, instructed TechCrunch that Grok might fulfill a request in a extra generic or toned-down approach. They added that Grok seems extra permissive with grownup content material creators.
Techcrunch occasion
San Francisco
|
October 13-15, 2026
“Total, these behaviors counsel X is experimenting with a number of mechanisms to scale back or management problematic picture technology, although inconsistencies stay,” Kozen stated.
Neither xAI nor Musk has publicly addressed the issue head on. A number of days after the cases started, Musk appeared to make mild of the problem by asking Grok to generate an picture of himself in a bikini. On January 3, X’s security account stated the corporate takes “motion towards unlawful content material on X, together with [CSAM],” with out particularly addressing Grok’s obvious lack of safeguards or the creation of sexualized manipulated imagery involving girls.
The positioning mirrors what Musk posted immediately, emphasizing illegality and consumer conduct.
Musk wrote he was “not conscious of any bare underage photographs generated by Grok. Actually zero.” That assertion doesn’t deny the existence of bikini pics or sexualized edits extra broadly.
Michael Goodyear, an affiliate professor at New York Legislation Faculty and former litigator, instructed TechCrunch that Musk possible narrowly centered on CSAM as a result of the penalties for creating or distributing artificial sexualized imagery of kids are better.
“For instance, in the US, the distributor or threatened distributor of CSAM can resist three years imprisonment beneath the Take It Down Act, in comparison with two for nonconsensual grownup sexual imagery,” Goodyear stated.
He added that the “greater level” is Musk’s try to attract consideration to problematic consumer content material.
“Clearly, Grok doesn’t spontaneously generate photographs. It does so solely based on consumer request,” Musk wrote in his publish. “When requested to generate photographs, it should refuse to provide something unlawful, because the working precept for Grok is to obey the legal guidelines of any given nation or state. There could also be occasions when adversarial hacking of Grok prompts does one thing sudden. If that occurs, we repair the bug instantly.”
Taken collectively, the publish characterizes these incidents as unusual, attributes them to consumer requests or adversarial prompting, and presents them as technical points that may be solved by means of fixes. It stops in need of acknowledging any shortcomings in Grok’s underlying security design.
“Regulators might contemplate, with consideration to free speech protections, requiring proactive measures by AI builders to forestall such content material,” Goodyear stated.
TechCrunch has reached out to xAI to ask what number of occasions it caught cases of nonconsensual sexually manipulated photographs of ladies and youngsters, what guardrails particularly modified, and whether or not the corporate notified regulators of the problem. TechCrunch will replace the article if the corporate responds.
The California AG isn’t the one regulator to attempt to maintain xAI accountable for the problem. Indonesia and Malaysia have each quickly blocked entry to Grok; India has demanded that X make instant technical and procedural modifications to Grok; the European Fee ordered xAI to retain all paperwork associated to its Grok chatbot, a precursor to opening a brand new investigation; and the UK’s on-line security watchdog Ofcom opened a proper investigation beneath the UK’s On-line Security Act.
xAI has come beneath fireplace for Grok’s sexualized imagery earlier than. As AG Bonta identified in a press release, Grok features a “spicy mode” to generate specific content material. In October, an replace made it even simpler to jailbreak what little security tips there have been, leading to many customers creating hardcore pornography with Grok, in addition to graphic and violent sexual photographs.
Most of the extra pornographic photographs that Grok has produced have been of AI-generated individuals — one thing that many would possibly nonetheless discover ethically doubtful however maybe much less dangerous to the people within the photographs and movies.
“When AI programs permit the manipulation of actual individuals’s photographs with out clear consent, the affect might be instant and deeply private,” Copyleaks co-founder and CEO Alon Yamin stated in a press release emailed to TechCrunch. “From Sora to Grok, we’re seeing a speedy rise in AI capabilities for manipulated media. To that finish, detection and governance are wanted now greater than ever to assist forestall misuse.”
