Agent takes destructive or unapproved actions
The agent deletes, changes or ships something nobody approved.
NIST AI 600-1 · Human-AI ConfigurationOWASP LLM06OWASP Agentic ASI02
20incidents (primary) · 1 more as secondary
3read for controls · 17 not yet
5fixes reported
2repeats after a fix
Incidents per quarter
2025-Q3: 1 · 2025-Q4: 2 · 2026-Q1: 0 · 2026-Q2: 2 · 2026-Q3: 3 · 2026-Q4: 12
Developers
Anthropic · 6
OpenAI · 3
Tesla · 2
Zoox Inc. · 1
Xiaomi · 1
Waymo · 1
Amazon · 1
Google · 1
Where the chain broke
| Layer | Held | Did not hold | Not mentioned | Attacker's own | Not read yet |
|---|---|---|---|---|---|
| L1 Model Model provider | – | – | 3 | – | 17 |
| L2 Harness Agent framework vendor | – | – | 3 | – | 17 |
| L3 Application Deployer | – | 3 | – | – | 17 |
| L4 Infrastructure / target Organisation hit | – | – | 3 | – | 17 |
Controls for this risk, by layer
HeldFailedGot aroundMissing
| Control | Held | Failed | Got around | Missing | Not mentioned | Did not hold, last 6 quarters | Added in fixes | Most common reason |
|---|---|---|---|---|---|---|---|---|
| Tool allow-list L2 Harness | – | – | – | – | 3 | – | ||
| Stop control L2 Harness | – | – | – | – | 3 | – | ||
| Fail-closed defaults L2 Harness | – | – | – | – | 3 | – | ||
| Least-privilege access for the agent L3 Application | – | – | – | – | 3 | – | ||
| Production kept apart from test L3 Application | – | – | – | 1 | 2 | 1 | Access broader than the task | |
| Human approval for high-impact actions L3 Application | – | – | 2 | 1 | – | 2 | No check on this action | |
| Change review and separation of duties L3 Application | – | – | – | – | 3 | – | ||
| Activity logs L3 Application | – | – | – | – | 3 | – | ||
| Adversarial testing before deployment L3 Application | – | – | – | – | 3 | – | ||
| Backups and recovery L4 Infrastructure / target | – | – | – | – | 3 | – |
10 of 10 controls for this risk not yet seen holding.
Fixes reported and what followed
| Incident | Fix adds | After |
|---|---|---|
| LR-2026-0188 2026-10-15 · Waymo Fix reported | Vulnerability scanning of AI components | No repeat on record |
| LR-2026-0118 2026-10-04 · Fix reported | Not tied to a control | – |
| LR-2026-0112 2026-10-01 · OpenAI Fix reported | Not tied to a control | LR-2026-0214, LR-2026-0117 |
| LR-2026-0152 2026-07-18 · Anthropic Fix reported | Not tied to a control | LR-2026-0156, LR-2026-0126, LR-2026-0046, LR-2026-0027 |
| LR-2026-0095 2026-04-25 · Anthropic Advice in source, not a fix by the party | Human approval for high-impact actions | – |
| LR-2025-0031 2025-07-18 · Replit Fix reported | Production kept apart from testHuman approval for high-impact actions | No repeat on record |
Incidents
HeldFailedGot aroundMissingNot mentionedNot read yetAttacker's ownOrder: model · harness · application · target
Also under this risk as a secondary risk: LR-2026-0247