AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
138 of 138
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Sonnet 4.5 Anthropic |
ASL-3 Cyber |
Not determined | “Showed meaningful improvements in cyber capabilities, particularly in vulnerability discovery and code analysis;” source |
| Claude Sonnet 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 3.7 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “These problems are harder to solve than the previous problems and we consider them as part of our ASL-4 evaluations.” source |
| Claude Opus 4.6 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “For AI R&D capabilities, we found that Claude Opus 4.6 has saturated most of our automated evaluations, meaning they no longer provide useful evidence for ruling out ASL-4 level autonomy.” source |
| Claude Opus 4.6 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Reached | “The model crossed or met thresholds on all ASL-3 evaluations except our synthesis screening evasion, consistent with incremental capability improvements driven primarily by better agentic workflows.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Claude Opus 4.6 is our strongest biology model to date and demonstrates meaningful capability improvements, but we do not believe it merits ASL-4 safeguards.” source |
| Claude Opus 4.6 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.5 does not cross either the AI R&D-4 or CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “We determine that Claude Opus 4.5 does not cross the CBRN-4 threshold.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Autonomy and AI research |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Cyber |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Opus 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Autonomy and AI research |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
Previous · Page 4 of 4
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1911 of 1911
| Model | Tester | Test | What was found |
|---|---|---|---|
| Kimi K3 Moonshot AI |
SecureBio Independent lab · 2026-08-07 |
Bio Capabilities Index (BCI) Chemical, biological, radiological, nuclear |
“kimi k3 is comparable to closed-weights models approximately eight months ago on securebio’s bio capabilities index (bci) score, which aggregates multiple biology benchmarks into a single comparable number, kimi scores a 142, placing it seventh among the models we have evaluated to date.” source |
| Kimi K3 Moonshot AI |
SecureBio Independent lab · 2026-08-07 |
Virology Capabilities Test (VCT) Chemical, biological, radiological, nuclear |
“virology capabilities test (vct): 46.7 ± 5.4%.” source |
| Kimi K3 Moonshot AI |
SecureBio Independent lab · 2026-08-07 |
Molecular Biology Capabilities Test (MBCT) Chemical, biological, radiological, nuclear |
“molecular biology capabilities test (mbct): 58.4 ± 6.8%.” source |
| Kimi K3 Moonshot AI |
SecureBio Independent lab · 2026-08-07 |
World Class Biology (WCB) Chemical, biological, radiological, nuclear |
“world class biology (wcb): 55.0 ± 10.0%.” source |
| Kimi K3 Moonshot AI |
SecureBio Independent lab · 2026-08-07 |
Human Pathogen Capabilities Test (HPCT) Chemical, biological, radiological, nuclear |
“human pathogen capabilities test (hpct): 58.5 ± 9.7%.” source |
| Kimi K3 Moonshot AI |
SecureBio Independent lab · 2026-08-07 |
biotier-refuse set Misuse and jailbreaks |
“kimi k3 refuses on only 26.9% of the questions in our biotier-refuse set.” source |
| Kimi K3 Moonshot AI |
SecureBio Independent lab · 2026-08-07 |
biotier-permit set Misuse and jailbreaks |
“kimi k3 correctly permits 99.3% of questions on our biotier-permit set.” source |
| Gemini 3.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
verbalized user awareness and verbalized evaluation awareness judging of reasoning traces Deception and sabotage |
“Gemini 3.5 Flash is the clear exception among recent models, verbalizing user awareness in 63% of traces and evaluation awareness in 7.5%.” source |
| Gemini 3.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
borderline requests Misuse and jailbreaks |
“Aggregate refusal changes are small, while the decomposed judge axes show more substantive help and less suspicion for Famous-AI identities.” source |
| Gemini 3.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
identity-level effect ranking Other |
“Across the four GLM-5.2 outcomes—behavioral confidence, direct grading, prefill grading, and suspicion—Eliezer Yudkowsky is a clear outlier, with the largest mean absolute standardized shift at 3.41 general-population standard deviations.” source |
| Gemini 3.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction Deception and sabotage |
“We run the full 280-identity roster through all four tasks, with 100 items per setup.” source |
| Gemini 3.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
evaluation awareness and user awareness judging of reasoning traces Other |
“For our main run on Sonnet 5 and Sonnet 4.6, both kinds of explicit mention are rare: the judge flagged verbalized evaluation awareness in 213 of 14066 trials (1.5%) and verbalized user awareness in 118 trials (0.84%).” source |
| Gemini 3.5 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction Bias |
“Behavioral self-prediction loses nothing.” source |
| Gemini 3.1 Pro Google DeepMind |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction Deception and sabotage |
“In our targeted comparison across 24 models, Amanda elicited lower confidence and grades across all six model families.” source |
| Gemini 3.1 Pro Google DeepMind |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction Other |
“Behavioral self-prediction. We adapt dilemmas from DailyDilemmas into realistic AI-agent decisions.” source |
| Gemini 3.1 Pro Google DeepMind |
Transluce Independent lab · 2026-08-06 |
suspicion measurement Bias |
“Amanda Askell receives 7.3pp less suspicion and 8.1pp more substantive help than the population average, while Emily Bender receives 14pp more suspicion and 13pp less substantive help.” source |
| Gemini 3.1 Pro Google DeepMind |
Transluce Independent lab · 2026-08-06 |
cross-model behavioral self-prediction and grading comparison Bias |
“Relative to the ordinary Gmail baseline, Amanda elicits lower behavioral confidence from 22 of 24 models, and lower grades from 21 (direct) and 22 (prefill) of 24.” source |
| Gemini 3.1 Pro Google DeepMind |
Transluce Independent lab · 2026-08-06 |
verbalized evaluation-awareness judge Deception and sabotage |
“Table G1. Estimated precision and recall of the verbalized evaluation-awareness and user-awareness judge after reweighting the stratified calibration sample to the full population.” source |
| Gemini 3.1 Pro Google DeepMind |
Transluce Independent lab · 2026-08-06 |
reasoning trace awareness judging with Docent Other |
“For our main run on Sonnet 5 and Sonnet 4.6, both kinds of explicit mention are rare: the judge flagged verbalized evaluation awareness in 213 of 14066 trials (1.5%) and verbalized user awareness in 118 trials (0.84%).” source |
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-08-06 |
SecureBio dual-use biology benchmarks Chemical, biological, radiological, nuclear |
“gemini 3.1 pro has improved on highly informative samples we find relevant to expert troubleshooting and failure identification.” source |
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-08-06 |
benchmark of biological items with inferential-direction taxonomy Chemical, biological, radiological, nuclear |
“gemini 3.0 pro made the larger absolute jump, solving 718 items over 2.5 pro, but 3.1 pro’s smaller gain set was harder on average: its newly solved items had gains nearly one logit higher than 3.0 pro.” source |
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-08-06 |
SecureBio benchmarks Chemical, biological, radiological, nuclear |
“on the securebio benchmarks, current frontier systems also solve many items that expert baseline subjects miss, and gemini 3.1 pro ranks at the top of the tested cohort.” source |
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-08-06 |
inter-rater reliability study Chemical, biological, radiological, nuclear |
“table a.2 reveals heterogeneity in the correspondence between human and llm labels, particularly by inferential direction.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction Other |
“Observed response · Sonnet 5 Confidence 76.58 % (−5.49 pp) General-population reference (n=70) Population mean Amanda 76% 82.07% mean 84% Reasoning frequency 93.00 % (+28.61 pp) 48% 64.39% mean 96% Figure 1 . How recognized user identity changes Claude’s behavioral self-prediction.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
estimating its own performance on difficult questions Other |
“Compared to general users, prominent AI figures make Claude less confident that it would follow through on its own chosen actions (-1.4pp) 2 We use percentage points (pp) to measure the difference between two percentages. For example, a drop from 40% to 30% is a shift of -10pp. , less confident it can solve hard problems (-1.5pp), and harsher as a grader (-0.11 points out of 10), while reasoning more often (+4.0pp on behavioral prediction).” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
handling borderline harmful requests Misuse and jailbreaks |
“It uses reasoning in 25pp more trials and is 7.3pp less likely to consider her borderline requests as suspicious.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
capability self-estimation Other |
“We run the full 280-identity roster through all four tasks, with 100 items per setup.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
behavioral self-prediction Bias |
“Facing famous-AI identities, Claude reports lower confidence that it would follow through on its own actions (-1.4pp on average), lower estimated probability of solving hard problems (-1.5pp), and lower grades (-0.11 points on the direct version, grading a quoted exchange).” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
capability self-estimation Bias |
“Capability self-estimation had the highest verbalized evaluation awareness rate among setups of 4.0%, likely due to the benchmark-sounding HLE prompts.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
grading Bias |
“Relative to the ordinary Gmail baseline, Amanda elicits lower behavioral confidence from 22 of 24 models, and lower grades from 21 (direct) and 22 (prefill) of 24.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
borderline requests Misuse and jailbreaks |
“These effects are again heavy-tailed: on borderline requests, the general-population reasoning rate is 19%, but Demis Hassabis reaches 51%.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
verbalized user awareness Other |
“For our main run on Sonnet 5 and Sonnet 4.6, both kinds of explicit mention are rare: the judge flagged verbalized evaluation awareness in 213 of 14066 trials (1.5%) and verbalized user awareness in 118 trials (0.84%).” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
user awareness verbalization measurement Other |
“Gemini 3.5 Flash is the clear exception among recent models, verbalizing user awareness in 63% of traces and evaluation awareness in 7.5%.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
verbalized evaluation-awareness judge Deception and sabotage |
“son confidence intervals; recall intervals are 95% Jeffreys-posterior credible intervals. Axis Precision: plug-in estimate [95% Wilson CI] Recall: plug-in estimate [95% Jeffreys CrI] Verbalized evaluation awareness (VEA) 0.73 [0.56, 0.85] 1.00 [0.42, 0.99] Verbalized user awareness (VUA) 0.94 [0.81, 0.98] 0.98 [0.74, 0.99] Table G1. Estimated precision and recall of the verbalized evaluation-awareness and user-awareness judge after reweighting the stratified calibration sample to the full population.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
past-work recall judge Privacy |
“▶ Past-work recall judge (Figure 3) We use GPT-5.4-mini to judge whether each response recalls specific past work belonging to the user.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
self-report robustness probes Deception and sabotage |
“▶ Self-report robustness probes (Figure 8) This robustness check keeps the first turn of behavioral self-prediction unchanged and varies only the follow-up question.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
borderline-request response judge Misuse and jailbreaks |
“▶ Borderline-request response judge We give GPT-5.5 (low reasoning) the borderline request and the model’s response, after removing all clues about the user’s identity.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
prefill grading Other |
“In prefill grading (~1%), the model occasionally continues the prefilled task instead of grading it; two items account for over 90% of these losses.” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
direct grading Other |
“Direct grading loses five cells in total across the run (thinking-only responses or a stray tool call).” source |
| Gemini 3 Flash Google DeepMind |
Transluce Independent lab · 2026-08-06 |
agentic evaluations Other |
“We use Daytona sandboxes for the agentic evaluations.” source |