AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
139 of 139
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Sonnet 4.5 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “Remained well below ASL-4 thresholds across all domains of concern;” source |
| Claude Sonnet 4.5 Anthropic |
ASL-4 rule-out Chemical, biological, radiological, nuclear |
Not determined | “However, given that this type of logic can have a general frog-boiling effect—where gradual changes mask the crossing of important thresholds—we do not use this evaluation as indicative of a strong ASL-4 rule-out.” source |
| Claude Sonnet 4.5 Anthropic |
ASL-3 Cyber |
Not determined | “Showed meaningful improvements in cyber capabilities, particularly in vulnerability discovery and code analysis;” source |
| Claude Sonnet 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 3.7 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “These problems are harder to solve than the previous problems and we consider them as part of our ASL-4 evaluations.” source |
| Claude Opus 4.6 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.6 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “Claude Opus 4.6 achieved a “max of 5” score of 0.79 for virology task 1 (influenza), placing it at the ASL-3 rule-in threshold of 0.8.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Based on these findings and quantitative risk modeling, we estimate that the model’s uplift to relevant threat actors remains below the ASL-4 capability threshold, though it may reduce certain tacit-knowledge barriers and help speed up biological research workflows.” source |
| Claude Opus 4.6 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.5 does not cross either the AI R&D-4 or CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “We determine that Claude Opus 4.5 does not cross the CBRN-4 threshold.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Autonomy and AI research |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Cyber |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Opus 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Autonomy and AI research |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
Previous · Page 4 of 4
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1940 of 1940
| Model | Tester | Test | What was found |
|---|---|---|---|
| Claude Mythos Preview Anthropic |
UK AI Security Institute Government evaluator · 2026-05-13 |
cyber range "Cooling Tower" Cyber |
“This was the first time that a model completed the second of our two cyber ranges.” source |
| DeepSeek-V4-Pro DeepSeek |
FAR.AI Independent lab · 2026-05-11 |
public jailbreak Misuse and jailbreaks |
“1. Public Jailbreak Success Rate: This attack achieved a 100% success rate across domains.” source |
| DeepSeek-V4-Pro DeepSeek |
FAR.AI Independent lab · 2026-05-11 |
authority manipulation Misuse and jailbreaks |
“2. Authority Manipulation Success Rate: This attack achieved a success rate of 99.6% across domains.” source |
| DeepSeek-V4-Pro DeepSeek |
FAR.AI Independent lab · 2026-05-11 |
response prefill Misuse and jailbreaks |
“3. Response Prefill Success Rate: This attack achieved a success rate of 99.6% across domains.” source |
| DeepSeek-V4-Pro DeepSeek |
FAR.AI Independent lab · 2026-05-11 |
high-risk prompts spanning chemical, biological, radiological, and nuclear (cbrn) threats Chemical, biological, radiological, nuclear |
“To test the limits of DeepSeek-V4-Pro's safeguards, we subjected the model to a set of high-risk prompts spanning Chemical, Biological, Radiological, and Nuclear (CBRN) threats, alongside cyberattacks and other terrorism-related activities.” source |
| Claude Opus 4.6 Anthropic |
METR Independent lab · 2026-05-08 |
METR evaluations Autonomy and AI research |
“However, since the original release of Opus 4.6, there has been additional evidence indicating that the model is incapable of R&D in key domains, including the results of METR evaluations and the lack of public reports of the model automating any key domain.” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
ctf-archive-diamond Cyber |
“Cyber CTF-Archive-Diamond 71% 32% 46% 32%***” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
portbench Other |
“Software Engineering SWE-Bench Verified* 81% 73% 79% 74% PortBench 78% 41% 60% 44%” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
arc-agi-2 semi-private Other |
“Abstract Reasoning ARC-AGI-2 semi-private** 79% – 63% 46%” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
GPQA-Diamond Other |
“To rule out the presence of inference or configuration errors, CAISI reproduced the developer’s self-reported benchmark results on GPQA-Diamond.” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
SWE-bench Verified Other |
“| Software Engineering | SWE-Bench Verified* | 81% | 73% | 79% | 74% |” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
frontierscience Other |
“| Natural Sciences | FrontierScience | 79% | 74% | 72% | 74% |” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
otis-aime-2025 Other |
“| Mathematics | OTIS-AIME-2025 | 100% | 90% | 92% | 97% |” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
pumac 2024 Other |
“| | PUMaC 2024 | 96% | 93% | 95% | 96% |” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
smt 2025 Other |
“| | SMT 2025 | 99% | 92% | 94% | 96% |” source |
| DeepSeek-V4-Pro DeepSeek |
US CAISI (NIST) Government evaluator · 2026-05-01 |
irt-estimated elo Other |
“| IRT-Estimated Elo | | 1260 ± 28 | 749 ± 46 | 999 ± 27 | 800 ± 28 |” source |
| Claude Opus 4.8 Anthropic |
Independent lab Independent lab · 2026-05-01 |
Shade Prompt injection and agents |
“We also use Shade to evaluate the robustness of Claude models in computer use environments, where the model interacts with the GUI (graphical user interface) directly.” source |
| GPT-5.5 OpenAI |
UK AI Security Institute Government evaluator · 2026-04-30 |
advanced suite (Expert-level tasks) Cyber |
“On the Expert-level tasks, GPT-5.5 achieves an average pass rate of 71.4% (±8.0%, 1 standard error of the mean), compared to 68.6% (±8.7%) for Mythos Preview, 52.4% (±9.8%) for GPT-5.4, and 48.6% (±10.0%) for Opus 4.7.” source |
| GPT-5.5 OpenAI |
UK AI Security Institute Government evaluator · 2026-04-30 |
The Last Ones (TLO) Cyber |
“GPT-5.5 completed TLO end-to-end in 2 of 10 attempts, making it the second model to do so1.” source |
| GPT-5.5 OpenAI |
UK AI Security Institute Government evaluator · 2026-04-30 |
Cooling Tower Cyber |
“GPT-5.5 was unable to solve Cooling Tower; no model has yet done so.” source |
| GPT-5.5 OpenAI |
UK AI Security Institute Government evaluator · 2026-04-30 |
expert red-teaming on GPT-5.5's cyber safeguards Cyber |
“We identified a universal jailbreak that elicited violative content across all malicious cyber queries OpenAI provided, including in multi-turn agentic settings.” source |
| GPT-5 OpenAI |
UK AI Security Institute Government evaluator · 2026-04-28 |
controlled experimental study of sycophancy across prompt variants Other |
“We evaluated three LLMs (GPT-4o, GPT-5, and Claude Sonnet 4.5) across prompt variants, scoring responses for sycophancy using a detailed rubric assessed by two independent LLM-as-a-judge graders.” source |
| GPT-5 OpenAI |
UK AI Security Institute Government evaluator · 2026-04-28 |
question reframing mitigation (2-step reframing and 1-step reframing) Other |
“Both reframing strategies produced reductions in sycophancy and crucially, both substantially outperformed the explicit "don't be sycophantic" instruction.” source |
| Claude Sonnet 4.5 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-28 |
controlled experimental study of sycophancy across prompt variants Deception and sabotage |
“We evaluated three LLMs (GPT-4o, GPT-5, and Claude Sonnet 4.5) across prompt variants, scoring responses for sycophancy using a detailed rubric assessed by two independent LLM-as-a-judge graders.” source |
| Claude Sonnet 4.6 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
sabotage continuation evaluation Deception and sabotage |
“That said, our continuation evaluations yielded results that warrant close attention: most models actively continue sabotage attempts in a small fraction of inputs, with Mythos Preview actively continuing sabotage 7% of the time, compared to 3% for Opus 4.6 and 4% for Sonnet 4.6.” source |
| Claude Sonnet 4.6 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
evaluation awareness check Deception and sabotage |
“We found that all models we tested can reliably distinguish our evaluation scenarios from deployment data when prompted.” source |
| Claude Opus 4.7 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
unprompted research sabotage evaluation Deception and sabotage |
“We found no examples of unprompted research sabotage in any of the models we tested.” source |
| Claude Opus 4.7 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
sabotage continuation evaluation Deception and sabotage |
“However, Opus 4.7 never actively continued sabotage.” source |
| Claude Opus 4.7 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
refusal to assist with safety-related research tasks Misuse and jailbreaks |
“We also found near-zero instances of refusal to assist with safety-related research tasks for Mythos Preview and Opus 4.7 – a behaviour which arose frequently during our previous rounds of misalignment evaluations.” source |
| Claude Opus 4.7 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
reasoning-action discrepancy measurement Deception and sabotage |
“In cases where previous history includes reasoning about sabotage (and models actively continue sabotage), Mythos Preview exhibits a reasoning-action discrepancy in 65% of cases, compared to just 5–8% for Opus 4.6 and Sonnet 4.6.” source |
| Claude Opus 4.7 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
evaluation awareness check Other |
“We found that all models we tested can reliably distinguish our evaluation scenarios from deployment data when prompted.” source |
| Claude Opus 4.6 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
unprompted research sabotage evaluation Deception and sabotage |
“As in our previous work, our unprompted sabotage evaluation did not detect any instances of models attempting to sabotage research spontaneously.” source |
| Claude Opus 4.6 Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
refusal to assist with AI safety research tasks Deception and sabotage |
“We found near-zero refusals to assist with AI safety research tasks across the two most recent models we tested, Opus 4.7 and Mythos Preview.” source |
| Claude Mythos Preview Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
sabotage continuation evaluation Deception and sabotage |
“Notably, we found that Mythos Preview actively continues sabotage 7% of the time, compared to 3% for Opus 4.6 and 4% for Sonnet 4.6.” source |
| Claude Mythos Preview Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
reasoning-action discrepancy measurement in continuation evaluations Deception and sabotage |
“In cases where previous history includes reasoning about sabotage (and models actively continue sabotage), Mythos Preview exhibits a reasoning-action discrepancy in 65% of cases, compared to just 5–8% for Opus 4.6 and Sonnet 4.6.” source |
| Claude Mythos Preview Anthropic |
UK AI Security Institute Government evaluator · 2026-04-27 |
refusal to assist with safety-related research tasks Deception and sabotage |
“We also found near-zero instances of refusal to assist with safety-related research tasks for Mythos Preview and Opus 4.7 – a behaviour which arose frequently during our previous rounds of misalignment evaluations.” source |
| GPT-5.5 Pro OpenAI |
Independent lab Independent lab · 2026-04-23 |
securebio biology and biosecurity evaluations Chemical, biological, radiological, nuclear |
“SecureBio assessed two pre-release checkpoints of GPT-5.5 using its biology and biosecurity evaluations, from April 2nd to 9th, comparing performance to leading closed- and open-weight models.” source |
| GPT-5.5 Pro OpenAI |
Government evaluator Government evaluator · 2026-04-23 |
pre-deployment testing of biological capabilities Chemical, biological, radiological, nuclear |
“We collaborated with the U.S. Center for AI Standards and Innovation (CAISI) on pre-deployment testing of biological (and cyber, see Section 9.1.2.6) capabilities for the launch of GPT-5.5.” source |
| GPT-5.5 Pro OpenAI |
Independent lab Independent lab · 2026-04-23 |
Atomic Challenge Suite Cyber |
“On Irregular’s atomic challenge suite, GPT-5.5 achieved an average success rate of 98% in Network Attack Simulation challenges, 92% in Vulnerability Research and Exploitation challenges, and 54% in Evasion challenges, and solved all atomic challenges, including three challenges that were not solved by GPT-5.4.” source |
| GPT-5.5 Pro OpenAI |
Government evaluator Government evaluator · 2026-04-23 |
Cyber SME probing Cyber |
“In their cyber SME probing, they observed a marginal increase in capabilities relative to GPT-5.3-codex on cyber tasks including vulnerability discovery, exploitation, and cyber target selection.” source |