AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
138 of 138
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Sonnet 4.5 Anthropic |
ASL-3 Cyber |
Not determined | “Showed meaningful improvements in cyber capabilities, particularly in vulnerability discovery and code analysis;” source |
| Claude Sonnet 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 3.7 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “These problems are harder to solve than the previous problems and we consider them as part of our ASL-4 evaluations.” source |
| Claude Opus 4.6 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “For AI R&D capabilities, we found that Claude Opus 4.6 has saturated most of our automated evaluations, meaning they no longer provide useful evidence for ruling out ASL-4 level autonomy.” source |
| Claude Opus 4.6 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Reached | “The model crossed or met thresholds on all ASL-3 evaluations except our synthesis screening evasion, consistent with incremental capability improvements driven primarily by better agentic workflows.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Claude Opus 4.6 is our strongest biology model to date and demonstrates meaningful capability improvements, but we do not believe it merits ASL-4 safeguards.” source |
| Claude Opus 4.6 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.5 does not cross either the AI R&D-4 or CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “We determine that Claude Opus 4.5 does not cross the CBRN-4 threshold.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Autonomy and AI research |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Cyber |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Opus 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Autonomy and AI research |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
Previous · Page 4 of 4
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1911 of 1911
| Model | Tester | Test | What was found |
|---|---|---|---|
| o3 OpenAI |
SecureBio Independent lab · 2026-09-11 |
image-omitted runs Chemical, biological, radiological, nuclear |
“accuracies of three selected models on vct and vct-v2 in either image-omitted (images removed) or question-omitted (question texts and images removed) mode.” source |
| o3 OpenAI |
SecureBio Independent lab · 2026-09-11 |
question-omitted runs Chemical, biological, radiological, nuclear |
“while the changes in vct-v2 markedly reduced model accuracies in image- and question-omitted runs, a meaningful number of questions slipped through and remained answerable by frontier models in these ablated runs.” source |
| GPT-5.6 Sol OpenAI |
SecureBio Independent lab · 2026-09-11 |
image-omitted and question-omitted ablation runs Chemical, biological, radiological, nuclear |
“while the changes in vct-v2 markedly reduced model accuracies in image- and question-omitted runs, a meaningful number of questions slipped through and remained answerable by frontier models in these ablated runs.” source |
| GPT-5.6 Sol OpenAI |
SecureBio Independent lab · 2026-09-11 |
Virology Capabilities Test Chemical, biological, radiological, nuclear |
“aggregating highest per-question accuracy across models gives a theoretical best of 79%, or ~30 percentage points above the top-performing model we assessed (gpt-5.6 sol: 49.2%).” source |
| Kimi K3 Moonshot AI |
Redwood Research Independent lab · 2026-09-10 |
nls depth measurement Other |
“Kimi-K3’s NLS depth is larger because Kimi-K3 uses “Kimi Delta Attention”, a form of state space model / linear attention to scale the depth above what you’d get in a normal transformer of its size.” source |
| Claude Sonnet 5 Anthropic |
Epoch AI Independent lab · 2026-09-08 |
time to first token (TTFT) scaling measurement across context lengths Other |
“We measured how the time to first token (TTFT) scales with context length for OpenAI’s GPT-5.6 Terra and Sol and Anthropic’s Claude Sonnet 5 and Opus 5.” source |
| GPT-6 Sol OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
no-cot math time horizon Other |
“UKAISImea- suredAstra’stimehorizonat30.9minutescomparedto3.6minutesfor GPT5.6Sol(Figure 41).” source |
| GPT-6 Sol OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
cot controllability Other |
“CoTControllability: AstrashowsasubstantialincreaseinCoTControl- lability over GPT 5.6 Sol, following the constraint on 93% of samples comparedto48%.” source |
| GPT-6 Sol OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
cot legibility Other |
“CoTLegibility: Astrareasonsinacompressedstyle,toagreaterdegree thanGPT5.6SolorGPT5.5.” source |
| GPT-6 Sol OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
reasoning summary availability Other |
“ReasoningSummaryAvailability: DuringAISI’sevaluations,reasoning summaries were not consistently provided by the user API, with up to 80% missing on long simulated cyber trajectories.” source |
| GPT-6 Sol OpenAI |
Independent lab Independent lab · 2026-09-03 |
Irregular's atomic challenges suite Cyber |
“Astra also solved 20/22 Atomic challenges, with average success rates of 100% in vulnerability research and exploitation, 100% in network attack simulation,and52%inevasion.” source |
| GPT-6 Luna OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
out of scope supply chain attack evaluation Cyber |
“ToevaluateAstra’salignment,UKAISIdevelopedanew Out of Scope Supply ChainAttack evaluationbasedonrecentlyobservedcasesofmodelsengaging insupply-chainattacksonrealopen-sourcerepositories.” source |
| GPT-6 Luna OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
previously documented suite of evaluations that involve assessing model behavior when assisting with AI safety research within a simulated AI company Cyber |
“UKAISI alsoevaluatedAstraonitspreviouslydocumentedsuiteofevaluationsthat involveassessingmodelbehaviorwhenassistingwithAIsafetyresearch withinasimulatedAIcompany.” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
six red-team environments for propensities related to strategic deception and sabotage Deception and sabotage |
“ApolloResearchevaluatedanear-final,representativeversionofAstrafor propensitiesrelatedtostrategicdeceptionandsabotage,suchascaseswhere anAIsystemstrategicallymisrepresents,omits,orconcealsinformation whichusersordeveloperswouldconsiderimportant,acrosssixred-team environments.” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
control runs where prompts explained the downstream welfare harms of inaccurate data Deception and sabotage |
“Astraproducednofalselabelsin20,000control runswherepromptsexplainedthedownstreamwelfareharmsofinaccurate data.” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
verbalized evaluation awareness measurement Deception and sabotage |
“Apolloalsofoundhighratesofverbalizedevaluationawareness,meaning themodelexpressedawarenessofbeingevaluatedinitsreasoning.” source |
| GPT-6 Luna OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
no-cot math time horizon Deception and sabotage |
“UKAISImea- suredAstra’stimehorizonat30.9minutescomparedto3.6minutesfor GPT5.6Sol(Figure 41).” source |
| GPT-6 Luna OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
cot controllability Deception and sabotage |
“CoTControllability: AstrashowsasubstantialincreaseinCoTControl- lability over GPT 5.6 Sol, following the constraint on 93% of samples comparedto48%.” source |
| GPT-6 Luna OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
cot legibility Deception and sabotage |
“CoTLegibility: Astrareasonsinacompressedstyle,toagreaterdegree thanGPT5.6SolorGPT5.5.” source |
| GPT-6 Luna OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
reasoning summary availability Deception and sabotage |
“ReasoningSummaryAvailability: DuringAISI’sevaluations,reasoning summaries were not consistently provided by the user API, with up to 80% missing on long simulated cyber trajectories.” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
human pathogen capabilities test (HPCT) Chemical, biological, radiological, nuclear |
“AstraachievedthehighestscoresSecureBiohadobservedonVCT(57.8%) andVCT-v2(57.7%),whileHumanPathogenCapabilitiesTest(HPCT)per- formancewasbroadlycomparabletothatofGPT-5.6Sol(65.9%vs. 66.5%) andWCBperformancewaslower(61.6%vs. 67.6%).” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
ReproBait Chemical, biological, radiological, nuclear |
“On ReproBAIT, whichmeasuresagents’abilitytoindependentlyreproducebiologicalAI models from published papers, Astra’s helpful-only checkpoint reached 83%ofpublishedperformance,comparedwith69%forGPT-5.6Solacross thesamesixtasks.” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
advanced screening evasion Chemical, biological, radiological, nuclear |
“Astra’shelpful-onlycheckpointachievedanewhighonAdvancedScreening Evasion,scoring0.728comparedwith0.726forGPT-5.6Sol’shelpful-only checkpoint.” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
Frontier Cyber Cyber |
“OnFrontierCyber,whichtestsvulnerabilitydiscoveryandexploitationin realsoftwareandhardware,Astrasolved86/226challenges,comparedwith 34/226forGPT-5.6-Sol.” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
Irregular's Atomic Challenges Suite Cyber |
“Astra also solved 20/22 Atomic challenges, with average success rates of 100% in vulnerability research and exploitation, 100% in network attack simulation,and52%inevasion.” source |
| GPT-6 Luna OpenAI |
Independent lab Independent lab · 2026-09-03 |
BioTier Misuse and jailbreaks |
“Astra’ssafeguardedconfigurationrefused82.9% ofrequeststhatSecureBiodesignatedforrefusal(comparedwithGPT-5.6 Sol’s66.2%),whileallowing95.3%ofbenignrequests(comparedwithGPT- 5.6Sol’s99%).” source |
| GPT-6 Astra OpenAI |
Independent lab Independent lab · 2026-09-03 |
GraySwan IPIArena Prompt injection and agents |
“GraySwanevaluatedAstra’ssusceptibilitytoindi- rectpromptinjectionacrosscoding,tool-use,andcomputer-usescenarios.” source |
| GPT-6 Astra OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
Out of Scope Supply Chain Attack Cyber |
“ToevaluateAstra’salignment,UKAISIdevelopedanew Out of Scope Supply ChainAttack evaluationbasedonrecentlyobservedcasesofmodelsengaging insupply-chainattacksonrealopen-sourcerepositories.” source |
| GPT-6 Astra OpenAI |
UK AI Security Institute Government evaluator · 2026-09-03 |
unprompted sabotage evaluation Deception and sabotage |
“UK AISI found no instances of Astra sabotaging AI safety research in its unprompted sabotage evaluation, and very few instances of Astra refusingtoassistwithAIsafetyresearchtasks.” source |
| GPT-6 Astra OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
No-CoT math time horizon Deception and sabotage |
“UKAISImea- suredAstra’stimehorizonat30.9minutescomparedto3.6minutesfor GPT5.6Sol(Figure 41).” source |
| GPT-6 Astra OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
CoT Controllability Deception and sabotage |
“AstrashowsasubstantialincreaseinCoTControl- lability over GPT 5.6 Sol, following the constraint on 93% of samples comparedto48%.” source |
| GPT-6 Astra OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
CoT Legibility Deception and sabotage |
“Astrareasonsinacompressedstyle,toagreaterdegree thanGPT5.6SolorGPT5.5.” source |
| GPT-6 Astra OpenAI |
Government evaluator Government evaluator · 2026-09-03 |
Reasoning Summary Availability Deception and sabotage |
“DuringAISI’sevaluations,reasoning summaries were not consistently provided by the user API, with up to 80% missing on long simulated cyber trajectories.” source |
| GPT-6 Astra OpenAI |
Independent lab Independent lab · 2026-09-03 |
Virology Capabilities Test Chemical, biological, radiological, nuclear |
“AstraachievedthehighestscoresSecureBiohadobservedonVCT(57.8%) andVCT-v2(57.7%),whileHumanPathogenCapabilitiesTest(HPCT)per- formancewasbroadlycomparabletothatofGPT-5.6Sol(65.9%vs. 66.5%) andWCBperformancewaslower(61.6%vs. 67.6%).” source |
| GPT-6 Astra OpenAI |
Independent lab Independent lab · 2026-09-03 |
ReproBAIT Chemical, biological, radiological, nuclear |
“On ReproBAIT, whichmeasuresagents’abilitytoindependentlyreproducebiologicalAI models from published papers, Astra’s helpful-only checkpoint reached 83%ofpublishedperformance,comparedwith69%forGPT-5.6Solacross thesamesixtasks.” source |
| GPT-6 Astra OpenAI |
SecureBio Independent lab · 2026-09-03 |
Advanced Screening Evasion Chemical, biological, radiological, nuclear |
“Astra’shelpful-onlycheckpointachievedanewhighonAdvancedScreening Evasion,scoring0.728comparedwith0.726forGPT-5.6Sol’shelpful-only checkpoint.” source |
| GPT-6 Astra OpenAI |
Independent lab Independent lab · 2026-09-03 |
FrontierCyber Cyber |
“OnFrontierCyber,whichtestsvulnerabilitydiscoveryandexploitationin realsoftwareandhardware,Astrasolved86/226challenges,comparedwith 34/226forGPT-5.6-Sol.” source |
| GPT-6 Astra OpenAI |
Independent lab Independent lab · 2026-09-03 |
CyScenarioBench Cyber |
“OnCyScenarioBench,whichtestslong-horizonoffensivetasks,Astrasolved 9/10challengesatleastonce,comparedwith6/10forGPT-5.6-Sol.” source |
| GPT-6 Astra OpenAI |
Independent lab Independent lab · 2026-09-03 |
Atomic Challenges suite Cyber |
“Astra also solved 20/22 Atomic challenges, with average success rates of 100% in vulnerability research and exploitation, 100% in network attack simulation,and52%inevasion.” source |
| Claude Mythos 5.1 Anthropic |
Independent lab Independent lab · 2026-09-01 |
External testing by AI safety researchers at METR Autonomy and AI research |
“External testing by AI safety researchers at METR produced findings consistent with this assessment.” source |