Over-reliance and loss of human oversight
People accept AI output or approve agent actions without real checking, or cannot keep up with it.
NIST AI 600-1 · Human-AI ConfigurationEU AI Act Art. 14OWASP Agentic ASI09
5incidents (primary) · 9 more as secondary
0read for controls · 5 not yet
0fixes reported
0repeats after a fix
Incidents per week
No rise: 0 in the week of 26 Oct 2026, against an average of 0 over the 4 weeks before.
Developers
Developer not named in the sources.
Where the chain broke
| Part of the system | Incidents where the control was not effective or did not exist | Incidents where the control was effective | Incidents where sources don't mention the control | Incidents not yet checked for controls |
|---|---|---|---|---|
| L1 Model Model provider | 5 | |||
| L2 Harness Agent framework vendor | 5 | |||
| L3 Application Deployer | 5 | |||
| L4 Infrastructure / target Organisation hit | 5 |
Controls for this risk, by layer
| Control | Incidents where the control was not effective or did not exist | Incidents where the control was effective | Incidents where sources don't mention the control | Most common reason it was not effective |
|---|---|---|---|---|
| Human approval for high-impact actions L3 Application | ||||
| Change review and separation of duties L3 Application | ||||
| Checks that approvals are real reviews L3 Application | ||||
| Live behaviour monitoring L3 Application | ||||
| User policy and training L3 Application | ||||
| Grounding and fact checks L3 Application |
6 of 6 controls for this risk not yet recorded as effective.
Fixes reported and what followed
No remediation reported for incidents of this risk.
Incidents
EffectiveNot effectiveNot effective (bypassed)Did not existNot stated in sourcesNot yet checkedAttacker's ownOrder: model · harness · application · target
| Date | Incident | Risk | Where it failed | Controls mentioned in sources |
|---|---|---|---|---|
| 2026-08-17 | Sainsbury's facial recognition wrongly accuses customers of shoplifting; scanners suspended | Over-reliance and loss of human oversight | Not yet checked | |
| 2026-07-08 | Flock camera misread leads to armed police stop of driver of a Range Rover in Plymouth, Minnesota | Over-reliance and loss of human oversight | Not yet checked | |
| 2026-06-15 | Judge throws out case after lawyers admit AI briefs with fake citations | Over-reliance and loss of human oversight | Not yet checked | |
| 2026-03-15 | AI facial recognition misidentification leads to wrongful jailing of Tennessee grandmother | Over-reliance and loss of human oversight | Not yet checked | |
| 2023-09-01 | Lawsuit alleges Reno police made thousands of unlawful facial ID arrests | Over-reliance and loss of human oversight | Not yet checked |
Also under this risk as a secondary risk: LR-2026-0651, LR-2026-0628, LR-2026-0535, LR-2026-0538, LR-2026-0537, LR-2026-0455, LR-2026-0311, LR-2026-0272, LR-2026-0437