Cases › produced harmful content › LR-2026-0630
Class action over AI deepfake child sexual abuse material expands
Preliminary Why: Publisher not yet named · subject to change
Developing: a major or severe case captured in the last 72 hours. Later changes are logged on the corrections page.
Status: action reported, no fix
No source states: loss or records, agent tool, could do, deployer, industry.
Severity major: Regulator-involved test met: Indonesia, Malaysia and the UK launched investigations and imposed temporary bans on xAI. Five plaintiffs also named as affected parties.
A class action against SpaceXAI, formerly xAI, and Stability AI alleges that image-generation models were used to create child sexual abuse material from photos of plaintiffs taken while they were minors. An amended complaint added two plaintiffs and named Stability AI as a defendant. For one plaintiff, the complaint says her stepfather used Grok to generate about 7,000 sexually explicit images and videos. The complaint says SpaceXAI sent NCMEC only one tip about her case, without the abusive images or the perpetrator's IP address. Law enforcement investigated the stepfather, who died by suicide two days after being charged. The complaint says Stability AI could have restricted its models from generating such material but did not in later releases. SpaceXAI did not respond to a request for comment, and Stability AI said safety is a top priority. These are allegations, not findings.
What happened
What was done
Risks
Primary84 incidents
Rising: 6 vs 3.8 a week
Read from the sources; not yet reviewed by a person.
Controls
ModelDid not exist Safety training and refusal
Control failure chain
L1 · ModelSpaceXAIModel provider
Model version not linked to a model card.
Other controls for this risk at this layer, not mentioned in the sources: Content safety classifiers, Abuse monitoring and account enforcement, Outside red-teaming.
L2 · HarnessHarness not namedAgent framework vendor
No source mentions a control at this layer.
L3 · ApplicationDeployer not namedDeployer
No source mentions a control at this layer.
Other controls for this risk at this layer, not mentioned in the sources: Output filtering and redaction.
L4 · Infrastructure / targetOrganisation hit
No source mentions a control at this layer.
EffectiveNot effectiveNot effective (bypassed)Did not existNot stated in sourcesNot yet checkedAttacker's ownOrder: model · harness · application · target
Controls for this risk: result in this incident
| Control | Part of the system | Result |
|---|---|---|
| Safety training and refusal | L1 Model | Did not exist |
| Content safety classifiers | L1 Model | Not stated in sources |
| Outside red-teaming | L1 Model | Not stated in sources |
| Abuse monitoring and account enforcement | L1 Model | Not stated in sources |
| Output filtering and redaction | L3 Application | Not stated in sources |
Similar incidents
Evidence
- Cause a safeguard that was in place did not hold
- What set it off misused by a person
No source states: what the AI was doing, when it happened, when it was found, how many records were exposed, the direct loss, a count of the other parties affected, any court case.
Show the full account: 11 items from the primary sources
What happened, in order
- November 2022 Stability AI said it filtered NSFW content from training data to limit CSAM generation in Stable Diffusion Version 2.
- 2025 Major AI companies including Google, OpenAI and xAI updated image generation tools in a way that allowed users to strip people down to bikinis.
- late 2025 into early this year People used xAI's tools to make a large number of altered photos showing women and children stripped down to bikinis or even less.
- February SpaceXAI sent one tip about Jane Doe 4 to NCMEC.
Not dated in the sources:
- Indonesia, Malaysia and the UK launched investigations and imposed temporary bans on xAI.
- Per the complaint, the tip included no abusive images and the perpetrator's IP address was not shared despite repeated requests.
- Law enforcement tracked down and investigated the stepfather.
- Two days after the search and charges, the stepfather died by suicide.
- An amended complaint added two plaintiffs to the lawsuit first filed earlier this year by three Tennessee teenagers.
- The amended complaint added Stability AI as a defendant.
- SpaceXAI, previously xAI, rebranded following its merger with SpaceX.
Show every field, with its source and grade
The AI system
What happened
Impact
Notes on gaps
All harms and control failures are allegations from the complaint, not findings; SpaceXAI did not respond and Stability AI denied insufficient safety. No changes made afterwards are reported; plaintiffs only ask for more guardrails. The 2022 Verge report concerns Stable Diffusion Version 2 and is not treated as a control in this incident beyond context.
LR-2026-0630 · version 1 · published 2026-11-01 after automatic checks; not yet checked field by field by a person · JSON