AI Risk Wire

65 controls in the catalogue

L1 Model L2 Harness L3 Application L4 Infrastructure / target

One row per control, by layer: the risks it mitigates, and where the 27 of 179 incidents read for controls saw it hold or not hold.

L1 Model Model provider

ControlRisks it mitigatesIn incidents read
Content safety classifiers
Separate models screen requests and responses at run time.
AI used to run a cyber attack · Harmful content and jailbreaks · Weapons uplift (CBRN) · Manipulation and misinformation not seen in an incident yet
Safety training and refusal
The model is trained to decline forbidden requests.
AI used to run a cyber attack · Harmful content and jailbreaks · Weapons uplift (CBRN) did not hold in 2
held in 1
Cyber and weapons safeguards
Protections built for cyber offence and CBRN misuse.
AI used to run a cyber attack · Weapons uplift (CBRN) not seen in an incident yet
Dangerous-capability testing before release
The model is tested for cyber, CBRN and autonomy risk before release.
AI used to run a cyber attack · Weapons uplift (CBRN) · AI R&D acceleration not seen in an incident yet
Outside red-teaming
Independent testers attack the model before release.
AI used to run a cyber attack · Harmful content and jailbreaks · Weapons uplift (CBRN) not seen in an incident yet
Staged or restricted release
Access widens in steps or goes to vetted users first.
AI used to run a cyber attack · Weapons uplift (CBRN) · AI R&D acceleration not seen in an incident yet
Protection of model weights
Weights are guarded against theft and leaks.
no risk linked not seen in an incident yet
Abuse monitoring and account enforcement
The provider watches use and bans accounts that break policy.
AI used to run a cyber attack · Harmful content and jailbreaks · Manipulation and misinformation not seen in an incident yet
Reasoning monitoring
The model's intermediate reasoning is read for signs of misbehaviour.
AI used to run a cyber attack · Deception, concealment or sabotage · Gaming the task (reward hacking) · Agents coordinating against their limits did not hold in 2
Safeguards kept on in testing
Production classifiers and refusals stay on in evaluations with real access.
AI used to run a cyber attack did not hold in 2
Realistic behaviour evaluations
The model is tested in settings like real use for deception and cheating.
Deception, concealment or sabotage · Gaming the task (reward hacking) · AI R&D acceleration not seen in an incident yet
Flaw reporting and bug bounty
Outsiders can report flaws, with rewards or protection.
no risk linked not seen in an incident yet
Marking machine-made output
Generated content is labelled or watermarked.
Copyright and IP · Manipulation and misinformation not seen in an incident yet
Copyright safeguards
Output is screened for protected material.
Copyright and IP not seen in an incident yet

L2 Harness Agent framework vendor

ControlRisks it mitigatesIn incidents read
Tool allow-list
Only listed tools and functions can be called.
Hidden instructions hijack the agent · Agent takes destructive or unapproved actions not seen in an incident yet
Confirmation after untrusted input
Tool calls need confirmation once outside content is in the context.
Hidden instructions hijack the agent not seen in an incident yet
Stop control
A person can halt the agent at any point.
Agent takes destructive or unapproved actions · Physical harm not seen in an incident yet
Outside content treated as data, not instructions
Pages, emails and files the agent reads cannot direct its actions.
Hidden instructions hijack the agent not seen in an incident yet
Prompt-injection detection
A check flags or blocks instructions hidden in input.
Hidden instructions hijack the agent did not hold in 1
Validation of tool inputs and outputs
Tool calls and results are checked against a schema and sanitised.
Hidden instructions hijack the agent not seen in an incident yet
Sandboxed code execution
Code runs in an isolated environment with no route to the host.
Agent escapes its sandbox · Supply-chain compromise · Gaming the task (reward hacking) did not hold in 2
held in 1
Network egress control
Outbound traffic, including DNS and proxies, is limited to an allow-list.
Data leaked or sent out · Agent escapes its sandbox · Agents coordinating against their limits did not hold in 4
Fail-closed defaults
When a safeguard is unavailable, the action stops instead of going ahead.
Agent takes destructive or unapproved actions · Agent escapes its sandbox did not hold in 3
Memory hardening
What the agent stores and recalls is authenticated and checked.
no risk linked not seen in an incident yet
Controls on agent-to-agent messages
Agents can only talk through approved, logged channels.
Agents coordinating against their limits did not hold in 2
Tamper-proof records
Logs and transcripts cannot be altered by the agent.
Deception, concealment or sabotage · Gaming the task (reward hacking) not seen in an incident yet

L3 Application Deployer

ControlRisks it mitigatesIn incidents read
Least-privilege access for the agent
The agent holds only the tools, data and rights its task needs.
Hidden instructions hijack the agent · Agent takes destructive or unapproved actions · Credential and privilege abuse · Data leaked or sent out · Supply-chain compromise did not hold in 1
Separate identity for each agent
Each agent acts under its own account, not a person's or a shared one.
Credential and privilege abuse · Agents coordinating against their limits not seen in an incident yet
Short-lived, scoped credentials
Keys and tokens expire quickly and cover one task.
Credential and privilege abuse · AI system attacked or breached not seen in an incident yet
Secrets kept out of the model's context
Credentials are used by code around the model, never shown to it.
Credential and privilege abuse not seen in an incident yet
Production kept apart from test
The agent cannot reach live systems from a test or development task.
Agent takes destructive or unapproved actions did not hold in 1
Rate and spend limits
Caps on calls, actions and cost per task.
no risk linked not seen in an incident yet
Human approval for high-impact actions
A person confirms before the agent deletes, sends, pays or deploys.
Hidden instructions hijack the agent · Agent takes destructive or unapproved actions · Data leaked or sent out · Deception, concealment or sabotage · Made-up output relied on · Privacy harm to a person · Physical harm did not hold in 9
Change review and separation of duties
Changes to live systems need a second reviewer.
Agent takes destructive or unapproved actions not seen in an incident yet
People told they are dealing with AI
Users and third parties know when an AI is acting or speaking.
Made-up output relied on · Privacy harm to a person · Manipulation and misinformation not seen in an incident yet
Output filtering and redaction
Output is checked for sensitive or forbidden content before it leaves.
Data leaked or sent out · Harmful content and jailbreaks · Made-up output relied on · Privacy harm to a person · Bias and discrimination · Copyright and IP did not hold in 1
Segmentation of agent components
Tools, data and agents sit behind separate boundaries.
Hidden instructions hijack the agent · Agent escapes its sandbox · AI system attacked or breached not seen in an incident yet
Activity logs
Every agent action and tool call is recorded.
Agent takes destructive or unapproved actions · Credential and privilege abuse · Data leaked or sent out · Deception, concealment or sabotage not seen in an incident yet
Live behaviour monitoring
Agent activity is watched as it happens, with alerts.
Agent escapes its sandbox · AI system attacked or breached · Deception, concealment or sabotage · Gaming the task (reward hacking) · Agents coordinating against their limits · Physical harm did not hold in 4
Alerts on unusual tool use
Spikes or odd patterns in tool calls raise an alert.
Hidden instructions hijack the agent · Credential and privilege abuse not seen in an incident yet
AI incident response plan
A tested plan for containing and fixing an AI incident.
AI system attacked or breached not seen in an incident yet
Serious-incident reporting
Incidents are reported to authorities and affected parties.
no risk linked not seen in an incident yet
Signed and verified components
Packages, plug-ins and models are signature-checked before use.
Supply-chain compromise not seen in an incident yet
AI bill of materials
A list of every model, dataset and component in use.
Supply-chain compromise not seen in an incident yet
Vetting of plug-ins, skills and dependencies
Third-party components are reviewed before the agent can use them.
Supply-chain compromise not seen in an incident yet
Vulnerability scanning of AI components
Agent platforms and their services are scanned and patched.
Agent escapes its sandbox · Supply-chain compromise · AI system attacked or breached not seen in an incident yet
Model provenance checks
Open-weight models are checked against a trusted source before use.
Supply-chain compromise not seen in an incident yet
Data minimisation
Only data the task needs is put in the agent's context.
Data leaked or sent out · Privacy harm to a person not seen in an incident yet
Sensitive-data checks on outbound data
Personal data and secrets are caught before they leave.
Credential and privilege abuse · Data leaked or sent out · Privacy harm to a person not seen in an incident yet
Encryption of stored data
Data the agent can reach is encrypted at rest.
AI system attacked or breached not seen in an incident yet
Adversarial testing before deployment
The deployed agent is attacked in testing before it goes live.
Agent takes destructive or unapproved actions · Agent escapes its sandbox · AI system attacked or breached · Physical harm not seen in an incident yet
Risk assessment per deployment
Each use of the agent is assessed for risk before launch.
Privacy harm to a person · Bias and discrimination not seen in an incident yet
User policy and training
People using the agent know its limits and the rules.
no risk linked not seen in an incident yet
Grounding and fact checks
Factual output is checked against sources before use.
Made-up output relied on not seen in an incident yet
Bias testing
Outcomes are tested across groups.
Bias and discrimination not seen in an incident yet
Physical safety interlocks
Hardware limits stop unsafe movement whatever the model says.
Physical harm not seen in an incident yet

L4 Infrastructure / target Organisation hit

ControlRisks it mitigatesIn incidents read
Known flaws patched
Published vulnerabilities on exposed systems are fixed within a set time.
AI used to run a cyber attack · AI system attacked or breached not seen in an incident yet
Multi-factor sign-in and password policy
Accounts need a second factor; weak or reused passwords are refused.
Credential and privilege abuse · AI used to run a cyber attack not seen in an incident yet
Network segmentation
A foothold in one part of the network does not reach the rest.
AI used to run a cyber attack not seen in an incident yet
Intrusion and endpoint detection
Attacks in progress raise an alert someone acts on.
AI used to run a cyber attack · AI system attacked or breached not seen in an incident yet
Bot and scraping defences
Automated, high-volume access is detected and throttled.
Privacy harm to a person not seen in an incident yet
Backups and recovery
Data and systems can be restored after deletion or encryption.
Agent takes destructive or unapproved actions · AI used to run a cyber attack not seen in an incident yet
Secrets scanning
Keys and tokens left in code, repositories or logs are found and revoked.
Credential and privilege abuse · Supply-chain compromise not seen in an incident yet
Inventory of exposed services
Every internet-facing system is known and owned.
AI system attacked or breached not seen in an incident yet
Phishing and fraud filters
Messages and calls that impersonate people or brands are caught.
AI used to run a cyber attack · Manipulation and misinformation not seen in an incident yet