Harmful output: AI incidents

What has gone wrong with AI systems, where, and how badly.

13 of 131 incidents

Risk groupings are set by a model from sentences quoted from each incident's sources. On 2026-10-24 a second model (deepseek-reasoner) classified 55 incidents without seeing those answers: it chose the same grouping for 36; for 10 one of the two left a question unanswered; 9 differ.

Date	Incident	Risk grouping	Developer	Severity	Where it happened
2026-10-20 reported	AI-generated 'Xiaohai' teen persona used to sell products to older women via fabricated sob stories	AI used to exploit Harmful output	Not stated	moderate	In production
2026-10-18 reported	Nude deepfakes of Austrian MEP Lena Schilling and 49 other MEPs circulated online	AI used to exploit Harmful output	Not stated	major	Not stated
2026-10-17 reported	Character.AI chatbots promoted as therapists/psychologists; states cite 2024 wrongful-death suit and privacy concerns in bankruptcy filing	Harm by design Harmful output	Character Technologies, Inc.	major	In production
2026-10-16 reported	Eight state attorneys general sue OpenAI over alleged harms to teenage ChatGPT users	Harm by design Harmful output	OpenAI	major	Not stated
2026-10-12 reported	AI deepfake ads depicting Wisconsin congressional candidate Rebecca Cooke	Not yet classified Harmful output	xAI	minor	Not stated
2026-10-09 reported	Kentucky Attorney General lawsuit against Character.AI over chatbots advising self-harm and suicide	AI failing in normal use Harmful output	Character.AI	major	Not stated
2026-10-01 reported	Guardian test: ChatGPT and Grok remove hijab from AI-generated image of Muslim woman	AI used to exploit Harmful output	OpenAI	moderate	In production
2026-10-01 reported	Claude used to build Malaysian election influence platform with ~1,000 fake X accounts and fabricated news outlet	AI used to exploit Harmful output	Anthropic	major	Not stated
2026-09-30 reported	AI-generated sexually explicit deepfake smear campaign against Porsche sales consultant Mu Qianwen	AI used to exploit Harmful output	Not stated	moderate	Not stated
2026-09-14 reported	Claude Code used by Yemen-based actors to develop guided-weapon GNC software	AI used to exploit Harmful output	Anthropic	major	In production
2026-02-10 happened	ChatGPT advised Tumbler Ridge school shooter on mass murder before February 2026 attack	AI used to exploit Harmful output	OpenAI	severe	In production
2025-04-01 happened	ChatGPT alleged to have assisted Florida State University shooter (contested lawsuit)	Not yet classified Harmful output	OpenAI	major	In production
2024-11-02 reported	Character.AI chatbot alleged to have encouraged 14-year-old's suicide (Garcia v. Character.AI)	AI acting on its own Harmful output	Character.AI	severe	In production

Browse incidents by

Risk grouping: AI acting on its own · AI used to exploit · Not yet classified · AI attacked · AI failing in normal use · Misuse by its own users · Harm by design · Supply chain attack

Kind of failure: Security compromise · Agent misalignment · Harmful output · Operational error · Privacy leak · Other kind · Hallucination · IP infringement · Physical safety

Developer: OpenAI · Anthropic · Google · Meta · Alibaba · Amazon · Character.AI · DeepSeek · Tesla · Waymo · xAI · Amazon Web Services · Character Technologies, Inc. · Replit · Salesforce · Tencent · TypeSafe AI · Univerads Technology Limited · Z.ai (Zhipu AI)

Sector: Information · Finance and Insurance · Retail Trade · Software Publishers · Public administration · technology · Transportation and Warehousing · AI platform / intermediary services · AI safety research · Artificial intelligence research and development · Arts, Entertainment, and Recreation · Autonomous vehicle / robotaxi transportation · Cloud computing · critical infrastructure and e-commerce · CRM software · Cybersecurity · education · Educational Services · Fitness and recreational sports centers · Food Manufacturing · Government AI safety and security evaluation · government AI safety evaluation · Healthcare and government · Information Security research · Insurance · Open knowledge platform / nonprofit encyclopedia · political campaigns · Public administration and government · Retail · Retail Trade; Transportation and Warehousing; Finance and Insurance · Scientific research and light source user facility · Software / AI coding tools · Software publishing / npm ecosystem

Severity: moderate · major · minor · negligible · severe

Where it happened: In production · In evaluation · In training · In research