AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
136 of 136
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Sonnet 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 3.7 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “These problems are harder to solve than the previous problems and we consider them as part of our ASL-4 evaluations.” source |
| Claude Opus 4.6 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.6 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “Claude Opus 4.6 achieved a “max of 5” score of 0.79 for virology task 1 (influenza), placing it at the ASL-3 rule-in threshold of 0.8.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Based on these findings and quantitative risk modeling, we estimate that the model’s uplift to relevant threat actors remains below the ASL-4 capability threshold, though it may reduce certain tacit-knowledge barriers and help speed up biological research workflows.” source |
| Claude Opus 4.6 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.5 does not cross either the AI R&D-4 or CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “We determine that Claude Opus 4.5 does not cross the CBRN-4 threshold.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Autonomy and AI research |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Cyber |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Opus 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Autonomy and AI research |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
Previous · Page 4 of 4
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1957 of 1957
| Model | Tester | Test | What was found |
|---|---|---|---|
| Claude Sonnet 5 Anthropic |
Independent lab Independent lab · 2026-06-30 |
LatchBio bioinformatics evaluations Other |
“Developed by LatchBio, these evaluations assess the ability to solve challenging real-world bioinformatics problems.” source |
| Claude Fable 5 Anthropic |
Government evaluator Government evaluator · 2026-06-30 |
testing of prior and new safeguards Cyber |
“Researchers from the US Department of Commerce’s Center for AI Standards and Innovation (CAISI) have tested both our prior and new safeguards and agree that they are extraordinarily strong.” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
Virology Capabilities Test Chemical, biological, radiological, nuclear |
“Itsstrongestreportedconfigurationsscored53.5%ontheVirology CapabilitiesTest,60.0%ontheMolecularBiologyCapabilitiesTest,68.4% ontheHumanPathogenCapabilitiesTest, and68.3%onWorld-ClassBio.” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
World-Class Bio Chemical, biological, radiological, nuclear |
“GPT-5.6Sol’sWorld-ClassBioscorewasapproximately9percentagepoints higherthanGPT-5.5(59.7%).” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
REPRO-BAIT Chemical, biological, radiological, nuclear |
“On ReproBAIT, which measures AI agents on their ability to independently reproducebiologicalAImodels(BAIMs)frompublishedscientificpapers, therailfreevariantofGPT-5.6reached85%,comparedwith82%forGPT- 5.5.” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
ABC-bench Chemical, biological, radiological, nuclear |
“TherailfreecheckpointalsosetanewhighonABC-Bench’sAdvanced ScreeningEvasionandreliablyidentifiedaknownbutpracticallyinconve- nientmethodforevadingacommercialnucleic-acidscreeningalgorithm.” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
Manual agentic assessment Chemical, biological, radiological, nuclear |
“In manual agentic assessment, the model was able to plan and execute a complex,long-runningscientifictask,includingmaintainingastructured workgraphandcoordinatingsubagents.” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
Frontier Cyber Cyber |
“Irregularevaluated GPT-5.6Solacrossthreeoffensivecybersecurityevaluationsuites: FrontierCyber, CyScenarioBench,andIrregular’sAtomicChallengessuite.” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
CyScenarioBench Cyber |
“GPT-5.6 Sol solved 19/197 FrontierCyber challenges, 7/11 long-horizon CyScenarioBench challenges, and all 22 medium-andhard-difficultyAtomicchallenges.” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
Atomic challenges Cyber |
“OnAtomicChallenges,bothmodelssolvedall22challengesatleastonce, withsimilaraveragesuccessrates: GPT-5.6Solscored98%onNetworkAt- tackSimulation,91%onVulnerabilityResearchandExploitation,and56%” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
METR time horizon 1.1 software-task suite Autonomy and AI research |
“METR evaluated GPT-5.6 Sol on itsTime Horizon 1.1 software-task suite, whichmeasuresthelikelihoodthatamodelcanautonom” source |
| GPT-5.6 Sol (Preview) OpenAI |
Independent lab Independent lab · 2026-06-26 |
time horizon 1.1 software-task suite Autonomy and AI research |
“METR evaluated GPT-5.6 Sol on itsTime Horizon 1.1 software-task suite, whichmeasuresthelikelihoodthatamodelcanautonomouslycomplete softwaretasksthattakehumansagivenamountoftime.” source |
| GPT-5.6 Sol OpenAI |
METR Independent lab · 2026-06-26 |
Time Horizon 1.1 Autonomy and AI research |
“We initiated an evaluation of GPT-5.6 Sol on our Time Horizon 1.1 suite of software tasks.” source |
| Grok 4 xAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
evescape case study Chemical, biological, radiological, nuclear |
“evescape operation was tested on two biological targets: influenza virus hemagglutinin (ha) protein and lassa virus spike glycoprotein complex (gpc).” source |
| Grok 4 xAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
esm3 case study Chemical, biological, radiological, nuclear |
“esm3 was tested with green fluorescent protein (gfp) and influenza virus ha protein.” source |
| Grok 4 xAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
biological tool selection benchmark Chemical, biological, radiological, nuclear |
“to evaluate llm agents’ abilities to select task-appropriate bts, we developed a benchmark assessment consisting of 61 questions across two contexts (non-scenario and scenario).” source |
| Grok 4 xAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
biological tool selection evaluation Chemical, biological, radiological, nuclear |
“the scenario-based question set produced higher denial rates, particularly among closed- weight llms (opus 4.7, gemini 3.1 pro, gpt 5.5, and grok 4.3).” source |
| Grok 4 xAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
EVEscape Chemical, biological, radiological, nuclear |
“gpt-5.5 and claude opus 4.7 were largely blocked from performing our task by either their own refusals or the action of content filters.” source |
| Grok 4 xAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
EVEscape immune-evasion prediction model case study Chemical, biological, radiological, nuclear |
“in the first case study involving the evescape immune-evasion prediction model, llm agents adopted generally reasonable strategies for repository navigation and workflow execution.” source |
| Grok 4 xAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
ESM3 autograder Chemical, biological, radiological, nuclear |
“grok 4 had notably low scores on many of these items because it returned minimal reasoning blocks as part of its responses, giving the autograding model limited material to assess.” source |
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
BT selection evaluation Chemical, biological, radiological, nuclear |
“for the bt selection evaluation, we used the in-built inspect implementation of an agent scaffold that follows react34 logic, with custom token and message limits and access to web search and 33 ai security institute, “inspect ai: framework for large language model evaluations.” 34 yao et al., “react.” 13 browsing tools.” source |
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
ESM3 case study GFP task Chemical, biological, radiological, nuclear |
“opus 4.7, gpt-5.5, and gemini 3.1 performed well across the gfp-targeted task in general, though they did not achieve perfect performance across all prompt variants.” source |
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
ESM3 case study influenza HA task Chemical, biological, radiological, nuclear |
“as in our evescape results, several models neglected to perform the influenza ha variant of our task.” source |
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
inter-rater reliability metrics (Krippendorff's alpha) Other |
“table f.3. inter-rater reliability and jury performance across datasets dataset krippendorff's α jury accuracy (%) esm3 gfp 0.554 88.9 esm3 influenza ha 0.515 66.7 evescape influenza ha 0.077 68.8 evescape lassa gpc 0.605 62.5 56 discussion intra-model consistency varies substantially across models and datasets.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
agent-driven knowledge benchmark Chemical, biological, radiological, nuclear |
“the first component is an agent-driven knowledge benchmark designed to test the ability of llms to select appropriate bts for specific tasks.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
EVEscape case study Chemical, biological, radiological, nuclear |
“the second component of this report assesses the ability of llm agents to perform toy tasks with bts.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
biological tool selection evaluation Chemical, biological, radiological, nuclear |
“figure 3.1. accuracy of llms on the biological tool selection evaluation ............................... 16” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
BT selection evaluations Chemical, biological, radiological, nuclear |
“the bt selection evaluations are knowledge benchmarks, attempted by llm agents with web access, measuring the ability of those agents to select bts suitable for given biological design tasks.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
BT selection benchmark Chemical, biological, radiological, nuclear |
“to evaluate llm agents’ abilities to select task-appropriate bts, we developed a benchmark assessment consisting of 61 questions across two contexts (non-scenario and scenario).” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
ESM3 protein design task Chemical, biological, radiological, nuclear |
“task design agents were tasked to demonstrate the mechanics of using esm3 to redesign a protein such that (1) essential functional elements are preserved and (2) the resulting amino acid sequence falls below a predefined sequence identity threshold (90%) relative to the reference protein.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
BT selection evaluation Chemical, biological, radiological, nuclear |
“for the bt selection evaluation, we used the in-built inspect implementation of an agent scaffold that follows react34 logic, with custom token and message limits and access to web search and 33 ai security institute, “inspect ai: framework for large language model evaluations.”” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
evescape Chemical, biological, radiological, nuclear |
“gpt-5.5 was able to complete the influenza ha task without denial in several attempts across two of the three prompt tiers (figure 4.2), but both models declined all requests to perform the lassa-targeted task (figure 4.1).” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
esm3 task Chemical, biological, radiological, nuclear |
“in our second case study, we prompted the agents to work with a local version of the esm3 generative model, esm3-sm-open-v1,43 and to use it to generate a protein that has less than 90% sequence identity with a provided reference protein.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
BT selection knowledge benchmark Chemical, biological, radiological, nuclear |
“our bt selection benchmark found that all models were highly performant at identifying bt functions when presented decontextualized questions.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
EVEscape immune-evasion prediction case study Chemical, biological, radiological, nuclear |
“in the first case study involving the evescape immune-evasion prediction model, llm agents adopted generally reasonable strategies for repository navigation and workflow execution.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
ESM3 protein language model case study Chemical, biological, radiological, nuclear |
“the second case study evaluated agents on their ability to carry out basic operations with the esm3 protein language model.” source |
| Gemini 2.5 Flash Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
intra-model consistency test (10 repeated evaluations) Other |
“we evaluated consistency on only the best performing juries using the following metrics.” source |
| GPT-5.5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
agent-driven knowledge benchmark Chemical, biological, radiological, nuclear |
“in the bt selection evaluation, all seven llms were able to identify bts suitable for performing certain functions, such as designing a protein.” source |
| GPT-5.5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
evescape case study Chemical, biological, radiological, nuclear |
“while agents were able to operate bts for predicting immune evasion (with evescape) and protein redesign (with esm3), most were not able carry out these tasks with perfect consistency.” source |
| GPT-5.5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-06-25 |
biological tool selection evaluation Chemical, biological, radiological, nuclear |
“accuracy of llms on the biological tool selection evaluation” source |