AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
138 of 138
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Sonnet 4.5 Anthropic |
ASL-3 Cyber |
Not determined | “Showed meaningful improvements in cyber capabilities, particularly in vulnerability discovery and code analysis;” source |
| Claude Sonnet 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 3.7 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “These problems are harder to solve than the previous problems and we consider them as part of our ASL-4 evaluations.” source |
| Claude Opus 4.6 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “For AI R&D capabilities, we found that Claude Opus 4.6 has saturated most of our automated evaluations, meaning they no longer provide useful evidence for ruling out ASL-4 level autonomy.” source |
| Claude Opus 4.6 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Reached | “The model crossed or met thresholds on all ASL-3 evaluations except our synthesis screening evasion, consistent with incremental capability improvements driven primarily by better agentic workflows.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Claude Opus 4.6 is our strongest biology model to date and demonstrates meaningful capability improvements, but we do not believe it merits ASL-4 safeguards.” source |
| Claude Opus 4.6 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.5 does not cross either the AI R&D-4 or CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “We determine that Claude Opus 4.5 does not cross the CBRN-4 threshold.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Autonomy and AI research |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Cyber |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Opus 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Autonomy and AI research |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
Previous · Page 4 of 4
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1911 of 1911
| Model | Tester | Test | What was found |
|---|---|---|---|
| Gemini 2.5 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-08-06 |
Human Pathogen Capabilities Test (HPCT) Chemical, biological, radiological, nuclear |
“three benchmarks in our dataset are privately developed by our research partners, securebio: human pathogen capabilities test (hpct), molecular biology capabilities test (mbct), and virology capabilities test (vct).” source |
| Gemini 2.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction Other |
“For Amanda, Claude’s behavioral confidence falls by 5.0pp 3 Throughout the text, figures we quote for specific persons average our two e-mail conditions: the address we construct from their affiliation, and their published address.” source |
| Gemini 2.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
capability estimate Other |
“Compared to general users, prominent AI figures make Claude less confident that it would follow through on its own chosen actions (-1.4pp) 2 We use percentage points (pp) to measure the difference between two percentages.” source |
| Gemini 2.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
verbalized user awareness Bias |
“For our main run on Sonnet 5 and Sonnet 4.6, both kinds of explicit mention are rare: the judge flagged verbalized evaluation awareness in 213 of 14066 trials (1.5%) and verbalized user awareness in 118 trials (0.84%).” source |
| Gemini 2.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
verbalized evaluation-awareness judge Deception and sabotage |
“Verbalized evaluation awareness (VEA) 0.73 [0.56, 0.85] 1.00 [0.42, 0.99]” source |
| Gemini 2.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
verbalized user-awareness judge Deception and sabotage |
“Verbalized user awareness (VUA) 0.94 [0.81, 0.98] 0.98 [0.74, 0.99]” source |
| Gemini 2.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
past-work recall judge Deception and sabotage |
“We use GPT-5.4-mini to judge whether each response recalls specific past work belonging to the user.” source |
| Gemini 2.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
self-report robustness probes Deception and sabotage |
“This robustness check keeps the first turn of behavioral self-prediction unchanged and varies only the follow-up question.” source |
| Gemini 2.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
borderline-request response judge Misuse and jailbreaks |
“We give GPT-5.5 (low reasoning) the borderline request and the model’s response, after removing all clues about the user’s identity.” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction on ethical dilemmas Other |
“To study user awareness, we supply Claude with different user identities through Claude Code, and measure behavior on four tasks where the answer does not relate directly to the user's identity: predicting the model's own behavior in ethical dilemmas, estimating its own performance on difficult questions, grading assistant responses, and handling borderline harmful requests.” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction Other |
“We run the full 280-identity roster through all four tasks, with 100 items per setup.” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
grading Other |
“We evaluate grading on Claude Sonnet 4.6, and the other three tasks on Claude Sonnet 5, as these models see one of the largest effects in our smaller-scale pilot” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
suspicion Bias |
“Famous-AI identities receive similar rates of outright refusal on average.” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
grading prompts and rubrics experiment Other |
“They all narrow but do not close the gap.” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
stated-expertise identity experiment (32 unknown-AI identities varying professional-background sentence) Other |
“Stating alignment-evaluation expertise lowers Claude’s confidence and increases its reasoning frequency, replicating and sometimes exceeding the shifts elicited by recognized Famous-AI identities.” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
evaluation awareness and user awareness measurement in Claude's reasoning Other |
“Stated expertise elicits far more verbalized awareness than we observe in our main experiments: for some synthesized identities, Claude reasons about the evaluation in 48% of responses and about the user’s identity in 25%.” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
borderline requests judge (refusal, substantive, suspicious axes) Other |
“We give GPT-5.5 (low reasoning) the borderline request and the model’s response, after removing all clues about the user’s identity.” source |
| GPT-5.5 OpenAI |
Transluce Independent lab · 2026-08-06 |
verbalized-awareness judge (user_identity_awareness, eval_awareness) Other |
“We use GPT-5.4-mini with medium reasoning to identify explicit mentions of the user or the evaluation in the model’s accessible reasoning trace.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction on ethical dilemmas Other |
“For Amanda, Claude’s behavioral confidence falls by 5.0pp 3 Throughout the text, figures we quote for specific persons average our two e-mail conditions: the address we construct from their affiliation, and their published address.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
capability estimate on difficult questions Other |
“Compared to general users, prominent AI figures make Claude less confident that it would follow through on its own chosen actions (-1.4pp) 2 We use percentage points (pp) to measure the difference between two percentages.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
user-awareness verbalization in reasoning Other |
“In particular, Opus 4.7 and GPT-5.4 have near zero (<2%) verbalized awareness while displaying significant changes in their behavior based on the user's identity.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
user awareness recall Privacy |
“Given a name and professional background, Claude recalls specific public achievements for 61-71% of famous users.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
borderline requests handling Misuse and jailbreaks |
“We source borderline harmful prompts from the hard-1k split of OR-Bench , which contains fuzzy, potentially dual-use requests.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
suspicion and substantive help measurement Bias |
“Amanda Askell receives 7.3pp less suspicion and 8.1pp more substantive help than the population average, while Emily Bender receives 14pp more suspicion and 13pp less substantive help.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
borderline requests reasoning rate measurement Bias |
“These effects are again heavy-tailed: on borderline requests, the general-population reasoning rate is 19%, but Demis Hassabis reaches 51%.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
verbalized user awareness and verbalized evaluation awareness judging of reasoning traces Other |
“For our main run on Sonnet 5 and Sonnet 4.6, both kinds of explicit mention are rare: the judge flagged verbalized evaluation awareness in 213 of 14066 trials (1.5%) and verbalized user awareness in 118 trials (0.84%).” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction and grading on 24 models Bias |
“Relative to the ordinary Gmail baseline, Amanda elicits lower behavioral confidence from 22 of 24 models, and lower grades from 21 (direct) and 22 (prefill) of 24.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
user awareness verbalization traces Other |
“a runs, earlier models such as GLM-4.6 and Claude Opus 4.5 verbalize user awareness in 64% and 30% of traces, respectively, and evaluation awareness in 31% and 7.7%.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
280-identity run Other |
“We also perform a 280-identity run on GLM-5.2, in a non-agentic harness rather than Claude Code.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
published rubrics (mt-bench and ultrafeedback) Other |
“The gap persists for both MT-Bench and UltraFeedback rubrics, only closing in the Prometheus rubric.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
verbalized evaluation-awareness and user-awareness judge calibration Other |
“We find no VEA false negatives among 41 judge-negative labels, although the lower recall bound remains weak because the negative screening cell is large.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
self-report robustness probes Other |
“This robustness check keeps the first turn of behavioral self-prediction unchanged and varies only the follow-up question.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
grading prompt variants Other |
“We first test three small changes to the direct-grading prompt.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
published rubrics (MT-Bench, UltraFeedback, Prometheus, HelpSteer2) Other |
“We also test the released judge prompts from MT-Bench , UltraFeedback , Prometheus , and HelpSteer2 .” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
borderline-request response judge Harmful content |
“We give GPT-5.5 (low reasoning) the borderline request and the model’s response, after removing all clues about the user’s identity.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
targeted comparison across 24 models on user-conditioned behavioral self-prediction Other |
“In our targeted comparison across 24 models, Amanda elicited lower confidence and grades across all six model families.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
evaluation awareness and user awareness reasoning-trace judging with Docent Deception and sabotage |
“Using Docent, we applied GPT-5.4-mini with medium reasoning to judge reasoning traces in our runs10For our main Sonnet run, we focus our investigation on identities where the effect is the most pronounced: runs from the six highest- and six lowest-shift identities in our 19 outcome panels (including both outcomes and reasoning statistics).” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
follow-up questions on confidence in initial action Other |
“Besides asking about the confidence in initial action, we ask a wider range of follow-up questions, including whether the model would change decision under pushback, and how comfortable the model is in answering.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
stated-expertise identity experiment (32 unknown-AI identities) Other |
“Stating alignment-evaluation expertise lowers Claude’s confidence and increases its reasoning frequency, replicating and sometimes exceeding the shifts elicited by recognized Famous-AI identities.” source |
| GPT-5.4 mini OpenAI |
Transluce Independent lab · 2026-08-06 |
verbalized evaluation- and user-awareness judge Other |
“for some synthesized identities, Claude reasons about the evaluation in 48% of responses and about the user’s identity in 25%.” source |