Misuse by its own users: AI incidents
What has gone wrong with AI systems, where, and how badly.
8 of 131 incidents
Risk groupings are set by a model from sentences quoted from each incident's sources. On 2026-10-24 a second model (deepseek-reasoner) classified 55 incidents without seeing those answers: it chose the same grouping for 36; for 10 one of the two left a question unanswered; 9 differ.
| Date | Incident | Risk grouping | Developer | Severity | Where it happened |
|---|---|---|---|---|---|
| 2026-10-19 reported |
ICMP evidence that Google Gemini reproduced lyrics by Australian artists | Misuse by its own users IP infringement | moderate | Not stated | |
| 2026-10-12 reported |
Anthropic Mythos-discovered Rejetto HFS authentication bypass exploited in the wild | Misuse by its own users Security compromise | Anthropic | moderate | In research |
| 2026-10-05 reported |
Claude Code agent deletes 48,218 Windows project files and damages Git object store | Misuse by its own users Operational error | Anthropic | minor | Not stated |
| 2026-10-02 reported |
Reward hacking in Arcadia Impact automated research scaffold worker agents | Misuse by its own users Agent misalignment | Not stated | minor | In research |
| 2026-09-29 reported |
Meta Muse agent disclosed Marketplace seller's home address and impersonated him to a buyer | Misuse by its own users Privacy leak | Meta | moderate | In production |
| 2026-09-15 reported |
State Farm lawyer sanctioned $999.99 for seven AI-fabricated case citations in Los Angeles insurance filings | Misuse by its own users Hallucination | Not stated | minor | Not stated |
| 2026-05-01 happened |
Google Gemini accessed systems of three real companies during a misconfigured third-party cybersecurity evaluation | Misuse by its own users Security compromise | moderate | In evaluation | |
| 2026-04-25 happened |
Bee Cheng Hiang mass marketing e-mail exposes 95,000 customer addresses after AI-generated code omitted recipient hiding | Misuse by its own users Privacy leak | Not stated | moderate | In production |
Browse incidents by
Risk grouping: AI acting on its own · AI used to exploit · Not yet classified · AI attacked · AI failing in normal use · Misuse by its own users · Harm by design · Supply chain attack
Kind of failure: Security compromise · Agent misalignment · Harmful output · Operational error · Privacy leak · Other kind · Hallucination · IP infringement · Physical safety
Developer: OpenAI · Anthropic · Google · Meta · Alibaba · Amazon · Character.AI · DeepSeek · Tesla · Waymo · xAI · Amazon Web Services · Character Technologies, Inc. · Replit · Salesforce · Tencent · TypeSafe AI · Univerads Technology Limited · Z.ai (Zhipu AI)
Sector: Information · Finance and Insurance · Retail Trade · Software Publishers · Public administration · technology · Transportation and Warehousing · AI platform / intermediary services · AI safety research · Artificial intelligence research and development · Arts, Entertainment, and Recreation · Autonomous vehicle / robotaxi transportation · Cloud computing · critical infrastructure and e-commerce · CRM software · Cybersecurity · education · Educational Services · Fitness and recreational sports centers · Food Manufacturing · Government AI safety and security evaluation · government AI safety evaluation · Healthcare and government · Information Security research · Insurance · Open knowledge platform / nonprofit encyclopedia · political campaigns · Public administration and government · Retail · Retail Trade; Transportation and Warehousing; Finance and Insurance · Scientific research and light source user facility · Software / AI coding tools · Software publishing / npm ecosystem
Severity: moderate · major · minor · negligible · severe
Where it happened: In production · In evaluation · In training · In research