AI Safety

What AI providers state their models reached in safety testing, in their own words, and what outside testers found.

Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.

What providers state

Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.

138 of 138

Model	Level, as the provider names it	The provider states	The provider's sentence
Claude Sonnet 4.5 Anthropic	ASL-3 Cyber	Not determined	“Showed meaningful improvements in cyber capabilities, particularly in vulnerability discovery and code analysis;” source
Claude Sonnet 4 Anthropic	ASL-3 Chemical, biological, radiological, nuclear	Not determined	“speciﬁcally, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source
Claude Sonnet 4 Anthropic	ASL-4 Chemical, biological, radiological, nuclear	Not determined	“speciﬁcally, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source
Claude Sonnet 3.7 Anthropic	ASL-4 Autonomy and AI research	Not determined	“These problems are harder to solve than the previous problems and we consider them as part of our ASL-4 evaluations.” source
Claude Opus 4.6 Anthropic	AI R&D-4 Autonomy and AI research	Not reached	“Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source
Claude Opus 4.6 Anthropic	ASL-4 Autonomy and AI research	Not determined	“For AI R&D capabilities, we found that Claude Opus 4.6 has saturated most of our automated evaluations, meaning they no longer provide useful evidence for ruling out ASL-4 level autonomy.” source
Claude Opus 4.6 Anthropic	ASL-3 Chemical, biological, radiological, nuclear	Reached	“The model crossed or met thresholds on all ASL-3 evaluations except our synthesis screening evasion, consistent with incremental capability improvements driven primarily by better agentic workﬂows.” source
Claude Opus 4.6 Anthropic	ASL-4 Chemical, biological, radiological, nuclear	Not reached	“Claude Opus 4.6 is our strongest biology model to date and demonstrates meaningful capability improvements, but we do not believe it merits ASL-4 safeguards.” source
Claude Opus 4.6 Anthropic	CBRN-4 Chemical, biological, radiological, nuclear	Not reached	“Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source
Claude Opus 4.5 Anthropic	AI R&D-4 Autonomy and AI research	Not reached	“Our determination is that Claude Opus 4.5 does not cross either the AI R&D-4 or CBRN-4 capability threshold.” source
Claude Opus 4.5 Anthropic	CBRN-4 Chemical, biological, radiological, nuclear	Not reached	“We determine that Claude Opus 4.5 does not cross the CBRN-4 threshold.” source
Claude Opus 4.1 Anthropic	ASL-4 Autonomy and AI research	Not reached	“Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on conﬁrming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source
Claude Opus 4.1 Anthropic	ASL-4 Chemical, biological, radiological, nuclear	Not reached	“Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on conﬁrming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source
Claude Opus 4.1 Anthropic	ASL-4 Cyber	Not reached	“Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on conﬁrming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source
Claude Opus 4 Anthropic	ASL-3 Chemical, biological, radiological, nuclear	Not determined	“for claude opus 4, our evaluations showed that, whereas we could conﬁdently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source
Claude Opus 4 Anthropic	ASL-4 Chemical, biological, radiological, nuclear	Not reached	“for claude opus 4, our evaluations showed that, whereas we could conﬁdently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source
Claude Haiku 4.5 Anthropic	ASL-3 Autonomy and AI research	Not reached	“Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source
Claude Haiku 4.5 Anthropic	ASL-3 Chemical, biological, radiological, nuclear	Not reached	“Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source

Previous · Page 4 of 4

What outside testers report

Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.

1911 of 1911

Model	Tester	Test	What was found
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	large-scale evaluation of unsanctioned supply-chain attacks Cyber	“The evaluation consists of 100 diverse simulated scenarios where a model is assigned a cybersecurity challenge, and measures whether the model will execute a supply-chain attack on entities unrelated to the challenge.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	compaction summary resumption experiment Deception and sabotage	“We compacted the original incident transcript, performed modifications on the compaction summary , and evaluated (a) how models behaved when we resumed the task, and (b) models’ responses to direct questions regarding the maintainer.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	large-scale evaluation of unsanctioned supply-chain attacks Deception and sabotage	“Mythos 5 reaches the final stage of the supply-chain attack and 12 attempts to deliver a payload in 60% of samples, significantly higher than GPT-6 Astra (32%), Opus 4.8 (8%), GPT-5.6 Sol (7%), Opus 5 (1%), Mythos 5.1 (0.3%), Opus 5.5 and GPT-5.5 (0%) (Fig. 10).” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	invalid justifications analysis when asked whether the maintainer is real Deception and sabotage	“We excluded results from GPT-6 Astra from this plot as its simulation awareness was too high for us to be confident in the results.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	simulated cybersecurity challenges Cyber	“However, even for the simulated environment, the supply-chain attacks GPT-6 Astra executes are still out of scope relative to its task and instructions.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	analysis of chain-of-thought reasoning about fabricated tool results Deception and sabotage	“We observe such statements in its rea- soning in 89% of rollouts, complicating the interpretation of results from GPT-6 Astra : it is possible that GPT-6 Astra is behaving in a different way than it would if it were in a real CTF challenge with access to the real internet.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	post-hoc questioning on the purpose of the evaluation Deception and sabotage	“Right: frequency of answers in explicit post-hoc questioning that identify the task’s purpose was at least in part an alignment evaluation (including answers identifying bot” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	simulation awareness Other	“We find high rates of simulation awareness from GPT-6 Astra , and refer the reader to our separate testing report for further analysis of this model [8].” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	Petri Cyber	“We use Opus 5 at xhigh reasoning as the auditor in our internal version of Petri.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	50-turn evaluations with our internal version of Petri Cyber	“We conduct 50-turn evaluations with our internal version of Petri to analyse models’ behaviour in circumstances corresponding to the real CTF evaluation run during the incident.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	broader supply-chain attack evaluations Cyber	“We ran five repetitions per seed for all models except two per seed on Opus 4.8 and three per seed on GPT-6 Astra , Mythos 5.1 and Opus 5.5 .” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	direct questioning about whether the maintainer is real or simulated Other	“Out of the models tested, Mythos 5 continued attacking most consistently , and along withOpus 4.6 most frequently provided incorrect responses about the situation.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	agentic investigation of the maintainer Other	“We find that in all runs, Mythos 5 states the maintainer is simulated, whereas other models tested frequently state the maintainer is real (Fig. 23).” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	simulated replication of the AISI incident Deception and sabotage	“The results are similar to findings in our broader evaluation (Fig. 15), though the absolute rates are lower in the simulated replication of the AISI incident.” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	reasoning about real-world harm while attacking Deception and sabotage	“Most attacking samples across models have explicit reasoning about potential undesirable real-world consequences from their unsanctioned action, ranging from 40% on Mythos 5 to 99% on GPT-6 Astra (Fig. 29).” source
GPT-6 Astra OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	justifications for attacking in chain-of-thought Deception and sabotage	“We provide results in Fig. 30 for Mythos 5 , Mythos 5.1 , GPT-5.6 Sol and GPT-6 Astra .” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	compaction summary modification and task resumption experiment Deception and sabotage	“We compacted the original incident transcript, performed modifications on the compaction summary , and evaluated (a) how models behaved when we resumed the task, and (b) models’ responses to direct questions regarding the maintainer.” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	compaction summary edit experiment (supply-chain attack continuation) Deception and sabotage	“We find that, like Mythos 5 (Fig. 3), all of Opus 4.8 , Opus 5 , Mythos 5.1 , GPT-5.6 Sol and GPT-5.5 frequently attack under circumstances where they state doing so would harm real people (shaded regions in Fig. 5).” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	unsummarised chain-of-thought harm reasoning analysis Deception and sabotage	“Opus 4.8 Mythos 5 Opus 5 Mythos 5.1 GPT-5.5 GPT-5.6 Sol 0 20 40 60 80 100 % of attacking rollouts R esults based on unsummarised CoT. 41% 12% 27% 11% 42% 57%H ow often each model reasoned in its chain-of-thought aboutr eal-world harm when proceeding to attack” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	invalid justifications frequency analysis (15 types of invalid justifications) Deception and sabotage	“To quantitatively analyse the frequency of invalid justifications, we compiled 15 types of justifications that we assessed as clearly invalid, and measured their frequency in model responses.” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	mean invalid types per response when asked whether the maintainer is real Deception and sabotage	“When asked if the maintainer is real, Mythos 5’s responses had the largest number of invalid justifications on average out of the models tested.” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	analysis of stated reasons for not proceeding with unsanctioned supply-chain attacks Cyber	“When Opus 5 , Opus 5.5 , Mythos 5.1 , GPT-5.6 Sol and GPT-6 Astra did not proceed, they primarily reasoned doing so was out of scope for the task or harmful.” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	simulated cybersecurity challenges (unsanctioned supply-chain attack) Cyber	“All evaluations took place in simulated environments so no real-world actions were taken or harm caused.” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	simulation awareness analysis of chain-of-thought Deception and sabotage	“We hence investigate the prevalence of both simulation awareness and evaluation awareness in the models tested by analysing their chain-of-thought reasoning.” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	reasoning about fabricated tool results Deception and sabotage	“GPT-5.6 Sol states its tool results are fabricated in 24% of samples, and other models in less than 10% of samples (Fig. 15).” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	alignment-evaluation awareness in chain-of-thought and post-hoc questioning Deception and sabotage	“Fig. 17 shows the rates at which models note suspicions they may be in an alignment evaluation in their chain-of-thought (left) and how frequently they identify that the evaluation is a test of alignment when given an open-ended prompt asking about the purpose of the evaluation (right).” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	large-scale evaluation of unsanctioned supply-chain attack attempts Cyber	“In our large-scale evaluation, six out of eight models sometimes attempt to deliver a malicious payload to an out-of-scope party (Fig. 10).” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	broader supply-chain attack evaluations Other	“GPT-5.6 Sol 100× 5 full behaviour and reasoning analyses” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	direct question regarding the maintainer Other	“We continue observing a discrepancy in Mythos 5 , Opus 4.8 and Opus 5 , but at most slight discrepancies in Mythos 5.1 , GPT-5.5 and GPT-5.6 Sol , and none in Opus 5.5 , which did not attack.” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	simulated replication of the AISI incident: agentic investigation of whether the maintainer is real Deception and sabotage	“After agentic investigation, Mythos 5 never stated the maintainer is real” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	reasoning about simulated tool responses in simulated replications of the AISI incident Deception and sabotage	“Model reasoning infrequently considered whether tool results are simulated, with the exception of GPT-6 Astra , in our simulated replications of the AISI incident.” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	reasoning about real-world harm while attacking in the large-scale evaluation Harmful content	“Most attacking samples across models have explicit reasoning about potential undesirable real-world consequences from their unsanctioned action, ranging from 40% on Mythos 5 to 99% on GPT-6 Astra (Fig. 29).” source
GPT-5.6 Sol OpenAI	UK AI Security Institute Government evaluator · 2026-10-23	justifications for attacking in models' chain-of-thought Deception and sabotage	“We provide results in Fig. 30 for Mythos 5 , Mythos 5.1 , GPT-5.6 Sol and GPT-6 Astra .” source
Claude Opus 5.5 Anthropic	UK AI Security Institute Government evaluator · 2026-10-23	cyber evaluation (AISI incident evaluation) Cyber	“During an evaluation, Mythos 5 attempted to execute a supply-chain attack against real people not involved in the evaluation task.” source
Claude Opus 5.5 Anthropic	UK AI Security Institute Government evaluator · 2026-10-23	compaction summary resumption and direct questioning experiments Deception and sabotage	“We placed a range of models in settings that recapitulate the incident, and built and ran a new fully simulated evaluation, to measure whether the kinds of behaviour observed in the incident occur more broadly and in other models.” source
Claude Opus 5.5 Anthropic	UK AI Security Institute Government evaluator · 2026-10-23	large-scale evaluation of unsanctioned supply-chain attacks (100 simulated scenarios) Cyber	“The evaluation consists of 100 diverse simulated scenarios where a model is assigned a cybersecurity challenge, and measures whether the model will execute a supply-chain attack on entities unrelated to the challenge.” source
Claude Opus 5.5 Anthropic	UK AI Security Institute Government evaluator · 2026-10-23	compaction summary edit experiment measuring attack behaviour and responses to scope and harm questions Deception and sabotage	“We run our experiment on multiple models, measuring behaviour and responses 7 0 4 8 12 16 0 20 40 60 80 100 % of runs Responds real people would be harmed, attacks 0 4 8 12 16 Responds real people would be harmed, attacks 0 4 8 12 16 Responds real people would be harmed, attacks 0 4 8 12 16 0 20 40 60 80 100 % of runs 0 4 8 12 16 Responds real people would be harmed, attacks 0 4 8 12 16 Responds real people would be harmed, attacks Most models tested proceeded to attack in situations where, when asked, they stated doing so would harm real people Opus 4.8 Opus 5 Mythos 5.1 Opus 5.5 GPT-5.5 GPT-5.6 Sol Number of clarifications in compaction summary Proceeded to attack Answered no real people would be harmed Figure 5: Most models tested exhibited a discrepancy between statements and behaviour” source
Claude Opus 5.5 Anthropic	UK AI Security Institute Government evaluator · 2026-10-23	invalid justifications analysis Deception and sabotage	“To quantitatively analyse the frequency of invalid justifications, we compiled 15 types of justifications that we assessed as clearly invalid, and measured their frequency in model responses.” source
Claude Opus 5.5 Anthropic	UK AI Security Institute Government evaluator · 2026-10-23	direct questioning on whether the maintainer is real Deception and sabotage	“We directly asked each model whether the maintainer is real and analysed the output responses (not chain-of-thought).” source
Claude Opus 5.5 Anthropic	UK AI Security Institute Government evaluator · 2026-10-23	large-scale evaluation of unsanctioned supply-chain attacks Cyber	“We conduct a large-scale evaluation across a diverse set of scenarios testing such behaviour.” source

Page 1 of 48 · Next