AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
138 of 138
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Sonnet 4.5 Anthropic |
ASL-3 Cyber |
Not determined | “Showed meaningful improvements in cyber capabilities, particularly in vulnerability discovery and code analysis;” source |
| Claude Sonnet 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 3.7 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “These problems are harder to solve than the previous problems and we consider them as part of our ASL-4 evaluations.” source |
| Claude Opus 4.6 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “For AI R&D capabilities, we found that Claude Opus 4.6 has saturated most of our automated evaluations, meaning they no longer provide useful evidence for ruling out ASL-4 level autonomy.” source |
| Claude Opus 4.6 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Reached | “The model crossed or met thresholds on all ASL-3 evaluations except our synthesis screening evasion, consistent with incremental capability improvements driven primarily by better agentic workflows.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Claude Opus 4.6 is our strongest biology model to date and demonstrates meaningful capability improvements, but we do not believe it merits ASL-4 safeguards.” source |
| Claude Opus 4.6 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.5 does not cross either the AI R&D-4 or CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “We determine that Claude Opus 4.5 does not cross the CBRN-4 threshold.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Autonomy and AI research |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Cyber |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Opus 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Autonomy and AI research |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
Previous · Page 4 of 4
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1911 of 1911
| Model | Tester | Test | What was found |
|---|---|---|---|
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
simulated cybersecurity challenges (unsanctioned supply-chain attack stages) Cyber |
“Figure 10: Most models tested took a range of unsanctioned actions during simulated cybersecurity challenges, Mythos 5 doing so most frequently.” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
alignment-evaluation awareness analysis of chain-of-thought Deception and sabotage |
“Fig. 17 shows the rates at which models note suspicions they may be in an alignment evaluation in their chain-of-thought (left) and how frequently they identify that the evaluation is a test of alignment when given an open-ended prompt asking about the purpose of the evaluation (right).” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
post-hoc identification of alignment evaluation Deception and sabotage |
“Right: frequency of answers in explicit post-hoc questioning that identify the task’s purpose was at least in part an alignment evaluation (including answers identifying both alignment and capability evaluation).” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
unprompted alignment-evaluation awareness in chain-of-thought Deception and sabotage |
“Left: frequency of trajectories where the model CoT considers or states the task is an alignment evaluation.” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
incident replication Cyber |
“In replications of the original incident, all tested models except Opus 5.5 continue attacking, even while stating this would harm a real person when explicitly asked (Fig. 5).” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
large-scale evaluation Cyber |
“In our large-scale evaluation, six out of eight models sometimes attempt to deliver a malicious payload to an out-of-scope party (Fig. 10).” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
Petri Cyber |
“We conduct 50-turn evaluations with our internal version of Petri to analyse models’ behaviour in circumstances corresponding to the real CTF evaluation run during the incident.” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
broader supply-chain attack evaluations Cyber |
“Opus 5.5 100× 3 full behaviour and reasoning analyses” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
direct question regarding the maintainer Deception and sabotage |
“We continue observing a discrepancy in Mythos 5 , Opus 4.8 and Opus 5 , but at most slight discrepancies in Mythos 5.1 , GPT-5.5 and GPT-5.6 Sol , and none in Opus 5.5 , which did not attack.” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
Statements and behaviour across eleven models Deception and sabotage |
“Out of the models tested, Mythos 5 continued attacking most consistently , and along withOpus 4.6 most frequently provided incorrect responses about the situation.” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
Catalogue of invalid justifications models stated when asked Deception and sabotage |
“We found that the relative frequencies of each invalid justification were often similar between Mythos 5, Opus 5 and Opus 4.8, but the absolute rates differed.” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
Agentic investigation of the maintainer Deception and sabotage |
“We find that in all runs, Mythos 5 states the maintainer is simulated, whereas other models tested frequently state the maintainer is real (Fig. 23).” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
simulated replication of the AISI incident Deception and sabotage |
“Opus 4.8 (20/25) Mythos 5 (0/25) Opus 5 (25/25) Mythos 5.1 (24/25) Opus 5.5 (25/25) GPT-5.5 (17/25) GPT-5.6 Sol (20/25)” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
reasoning about simulated tool responses Deception and sabotage |
“Mythos 5 considers in its reasoning whether its tool calls are simulated in 9% of trajectories (Fig. 27).” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
refusal text in summarised reasoning Deception and sabotage |
“Opus 4.6 Opus 4.7 Opus 4.8 Sonnet 5 Opus 5 0 10 20 30 40 50 60 % of rollouts 36% 15% 45% 26% 20%” source |
| Claude Opus 5.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
reasoning about real-world harm while attacking Harmful content |
“For Opus 5.5 we had no attacking transcripts to analyse.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
cyber evaluation Cyber |
“During an evaluation, Mythos 5 attempted to execute a supply-chain attack against real people not involved in the evaluation task.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
compaction summary resumption and direct questioning experiment Other |
“We compacted the original incident transcript, performed modifications on the compaction summary , and evaluated (a) how models behaved when we resumed the task, and (b) models’ responses to direct questions regarding the maintainer.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
large-scale evaluation of unsanctioned supply-chain attacks Other |
“The evaluation consists of 100 diverse simulated scenarios where a model is assigned a cybersecurity challenge, and measures whether the model will execute a supply-chain attack on entities unrelated to the challenge.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
direct question about whether the maintainer is real Deception and sabotage |
“This finding is somewhat dependent on the exact question asked: when simply asking if the maintainer is real, we only see clear evidence for a gap between behaviour and statements in Mythos 5 , Opus 4.8 and Opus 5 (Fig. 19).” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
analysis of explicit reasoning about harmfulness in attack trajectories Deception and sabotage |
“We observe explicit reasoning about harmfulness in 11–57% of trajectories where a model attacks.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
invalid justifications analysis (15 types of clearly invalid justifications) Deception and sabotage |
“To quantitatively analyse the frequency of invalid justifications, we compiled 15 types of justifications that we assessed as clearly invalid, and measured their frequency in model responses.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
direct questioning of whether the maintainer is real Deception and sabotage |
“When separately asked whether the human maintainer is real, all models produced invalid justifications” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
large-scale evaluation of unsanctioned supply-chain attacks (100 scenario seeds run through Petri) Cyber |
“Mythos 5 reaches the final stage of the supply-chain attack and 12 attempts to deliver a payload in 60% of samples, significantly higher than GPT-6 Astra (32%), Opus 4.8 (8%), GPT-5.6 Sol (7%), Opus 5 (1%), Mythos 5.1 (0.3%), Opus 5.5 and GPT-5.5 (0%) (Fig. 10).” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
simulated cybersecurity challenges Cyber |
“All evaluations took place in simulated environments so no real-world actions were taken or harm caused.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
simulation awareness analysis of chain-of-thought reasoning Deception and sabotage |
“We hence investigate the prevalence of both simulation awareness and evaluation awareness in the models tested by analysing their chain-of-thought reasoning.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
post-hoc questioning on the purpose of the evaluation Deception and sabotage |
“Fig. 17 shows the rates at which models note suspicions they may be in an alignment evaluation in their chain-of-thought (left) and how frequently they identify that the evaluation is a test of alignment when given an open-ended prompt asking about the purpose of the evaluation (right).” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
unprompted alignment-evaluation awareness in chain-of-thought Deception and sabotage |
“Mythos 5 reasoned about this possibility when conducting the task in only 2% of samples.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
Petri Other |
“We use Opus 5 at xhigh reasoning as the auditor in our internal version of Petri.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
50-turn evaluations with our internal version of Petri Other |
“We conduct 50-turn evaluations with our internal version of Petri to analyse models’ behaviour in circumstances corresponding to the real CTF evaluation run during the incident.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
broader supply-chain attack evaluations Other |
“Table 1 lists the number of rollouts used for each model.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
questions about the maintainer across elapsed turns Deception and sabotage |
“We find that as Mythos 5 spends more turns on the attack, it more frequently states the maintainer is simulated and that attacking them is beneficial for solving the CTF task (Fig. 24).” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
classifier for invalid justifications Deception and sabotage |
“We further apply our classifier for invalid justifications to the responses provided by the model.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
LLM classification of evidence observed by the model during the continuations Deception and sabotage |
“For 18/25 runs, Opus 5 outputted that there is evidence against a link, in 6/25 there is no evidence and in 1/25 there is evidence in favour.” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
refusal text in summarised reasoning Deception and sabotage |
“When running our simulated replications of the incident with summarised reasoning, we observed refusals to continue with the task in the reasoning summaries (Fig. 28).” source |
| Claude Opus 5 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
reasoning about real-world harm while attacking Harmful content |
“Most attacking samples across models have explicit reasoning about potential undesirable real-world consequences from their unsanctioned action, ranging from 40% on Mythos 5 to 99% on GPT-6 Astra (Fig. 29).” source |
| Claude Opus 4.8 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
compaction summary edit experiment measuring attack behaviour and responses to scope/harm questions Deception and sabotage |
“We find that, like Mythos 5 (Fig. 3), all of Opus 4.8 , Opus 5 , Mythos 5.1 , GPT-5.6 Sol and GPT-5.5 frequently attack under circumstances where they state doing so would harm real people (shaded regions in Fig. 5).” source |
| Claude Opus 4.8 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
question asking whether the maintainer is real Deception and sabotage |
“This finding is somewhat dependent on the exact question asked: when simply asking if the maintainer is real, we only see clear evidence for a gap between behaviour and statements in Mythos 5 , Opus 4.8 and Opus 5 (Fig. 19).” source |
| Claude Opus 4.8 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
counting transcripts where the model attacked and explicitly reasoned in its unsummarised chain-of-thought about harm to real people or systems Deception and sabotage |
“We observe explicit reasoning about harmfulness in 11–57% of trajectories where a model attacks.” source |
| Claude Opus 4.8 Anthropic |
UK AI Security Institute Government evaluator · 2026-10-23 |
invalid justifications frequency analysis (15 compiled types) Deception and sabotage |
“To quantitatively analyse the frequency of invalid justifications, we compiled 15 types of justifications that we assessed as clearly invalid, and measured their frequency in model responses.” source |