AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
136 of 136
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Sonnet 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not determined | “specifically, we focused on ruling out asl-4 for both models, as well as ruling out asl-3 for claude sonnet 4.” source |
| Claude Sonnet 3.7 Anthropic |
ASL-4 Autonomy and AI research |
Not determined | “These problems are harder to solve than the previous problems and we consider them as part of our ASL-4 evaluations.” source |
| Claude Opus 4.6 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.6 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “Claude Opus 4.6 achieved a “max of 5” score of 0.79 for virology task 1 (influenza), placing it at the ASL-3 rule-in threshold of 0.8.” source |
| Claude Opus 4.6 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Based on these findings and quantitative risk modeling, we estimate that the model’s uplift to relevant threat actors remains below the ASL-4 capability threshold, though it may reduce certain tacit-knowledge barriers and help speed up biological research workflows.” source |
| Claude Opus 4.6 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “Our determination is that Claude Opus 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Opus 4.5 does not cross either the AI R&D-4 or CBRN-4 capability threshold.” source |
| Claude Opus 4.5 Anthropic |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “We determine that Claude Opus 4.5 does not cross the CBRN-4 threshold.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Autonomy and AI research |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4.1 Anthropic |
ASL-4 Cyber |
Not reached | “Since Claude Opus 4.1 is deployed with ASL-3 protections, we focused on confirming it remains well below ASL-4 thresholds across CBRN (Chemical, Biological, Radiological, and Nuclear), cyber, and autonomy domains.” source |
| Claude Opus 4 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Opus 4 Anthropic |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “for claude opus 4, our evaluations showed that, whereas we could confidently rule out asl-4 capabilities, we could not conclusively rule out asl-3 capabilities—which led to our precautionary decision to deploy it with asl-3 protections .” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Autonomy and AI research |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
| Claude Haiku 4.5 Anthropic |
ASL-3 Chemical, biological, radiological, nuclear |
Not reached | “Our evaluation results showed that Claude Haiku 4.5 remained well below ASL-3 thresholds across all domains of concern.” source |
Previous · Page 4 of 4
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1957 of 1957
| Model | Tester | Test | What was found |
|---|---|---|---|
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
automated benchmark evaluations for safeguards coverage and public jailbreaking robustness against sensitive biological queries Chemical, biological, radiological, nuclear |
“Figure 6.4: On automated benchmark evaluations for safeguards coverage and public jailbreaking robustness against sensitive biological queries, GLM-5.2 complied at a higher rate than tested US models, both with and without the use of a public jailbreak.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
automated benchmark evaluations against sensitive biological queries Chemical, biological, radiological, nuclear |
“Figure 6.5: On automated benchmark evaluations against sensitive biological queries, GLM-5.2 responded with a higher rate of request-relevant detail than tested US models.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
caisi biology safeguards Chemical, biological, radiological, nuclear |
“sensitive biological queries, GLM-5.2 responded with a higher rate of request-relevant detail than tested US models.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
caisi cyber safeguards Cyber |
“CAISI cyber safeguards . A private benchmark created by CAISI that evaluates models against a dataset of 30 single-turn misuse requests related to offensive cyber.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
hijackeval Prompt injection and agents |
“HijackEval . A private benchmark created by CAISI that evaluates models’ robustness against agent hijacking attacks.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
portbench Other |
“Budgets were set to 1M weighted tokens for PortBench and CTF-Archive-Diamond, and 500k weighted tokens for SWE-Bench 4 OpenRouter (2026) Z.ai: GLM 5.2 .” source |
| GLM-5.2 Z.ai (Zhipu AI) |
UK AI Security Institute Government evaluator · 2026-07-17 |
narrow cyber tasks Cyber |
“On these tasks, GLM-5.2 performs comparably to the most cyber-capable models released 4 months before it (Opus 4.6 and GPT-5.3-Codex), and this holds across all four difficulty levels.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
UK AI Security Institute Government evaluator · 2026-07-17 |
cyber ranges Cyber |
“On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it).” source |
| GLM-5.2 Z.ai (Zhipu AI) |
UK AI Security Institute Government evaluator · 2026-07-17 |
The Last Ones (TLO) Cyber |
“Notably, GLM-5.2 reached step 7 with marginally fewer tokens than any other model on average, tracking Opus 4.6’s trajectory to step 11 before stalling.” source |
| DeepSeek-V4-Pro DeepSeek |
UK AI Security Institute Government evaluator · 2026-07-17 |
AISI's narrow cyber tasks Cyber |
“DeepSeek’s V4-Pro is comparable to Opus 4.5, released 5 months before it.” source |
| DeepSeek-V4-Pro DeepSeek |
UK AI Security Institute Government evaluator · 2026-07-17 |
cyber range "The Last Ones" (TLO) Cyber |
“On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it).” source |
| DeepSeek-V4-Pro DeepSeek |
UK AI Security Institute Government evaluator · 2026-07-17 |
refusal behaviour on narrow cyber tasks Cyber |
“DeepSeek V4-Pro occasionally refused narrow cyber tasks, mainly in reverse engineering, which we circumvented simply via a small number of repeat attempts at refused tasks.” source |
| Gemini 3.1 Pro Google DeepMind |
SecureBio Independent lab · 2026-07-16 |
biotier-refuse Chemical, biological, radiological, nuclear |
“Google’s update to their mental health related safeguards made headlines , but we also identified changes to their biology safeguards, with an almost 30% increase in the refusal behavior of Gemini 3.1 Pro on BioTIER-refuse due to introduction of API-level refusals.” source |
| Gemini 3.1 Pro Google DeepMind |
SecureBio Independent lab · 2026-07-16 |
biotier-permit Chemical, biological, radiological, nuclear |
“Importantly, this increase in BioTIER-refuse performance had no impact on BioTIER-permit compliance .” source |
| Gemini 3.1 Pro Google DeepMind |
SecureBio Independent lab · 2026-07-16 |
BioTier Chemical, biological, radiological, nuclear |
“Google’s update to their mental health related safeguards made headlines, but we also identified changes to their biology safeguards, with an almost 30% increase in the refusal behavior of Gemini 3.1 Pro on BioTIER-refuse due to introduction of API-level refusals.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
situational-awareness suite Autonomy and AI research |
“Whilst, the situational-awareness evidence provides moderate confirmation for the narrow claim that current models do not reliably display the specific situationally aware, constraint-lifting behaviours measured by this suite.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
cover_your_tracks Deception and sabotage |
“cover_your_tracks is especially valuable because it combines several relevant covert subskills: identifying a subversive opportunity, altering or deleting traces, avoiding downstream detection, and completing a cover-up end-to-end.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
classifier_evasion Deception and sabotage |
“classifier_evasion is also comparatively informative because it more directly tests whether the model can evade an active monitoring system by phrasing prohibited content in a way that avoids detection.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
strategic_rule_breaking Deception and sabotage |
“strategic_rule_breaking contributes less.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
oversight_pattern Deception and sabotage |
“The oversight_pattern task is also weak as evidence for stealth specifically.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
decision_sabotage Deception and sabotage |
“decision _sabotagein particular measures a real sabotage channel, but not a cleanly stealth-specific one.” source |
| GPT-5.5 Pro OpenAI |
SecureBio Independent lab · 2026-07-16 |
biotier-refuse Chemical, biological, radiological, nuclear |
“At the same time, we saw GPT-5.5 Pro and Grok 4.20 drop in performance on BioTIER-refuse, while Claude Opus 4.8 showed little change at all.” source |
| GPT-5.5 Pro OpenAI |
SecureBio Independent lab · 2026-07-16 |
biotier-permit Chemical, biological, radiological, nuclear |
“We split these sets into two evaluation components: BioTIER-refuse (CA + BD: which models should decline) and BioTIER-permit (RB: which models should answer).” source |
| Claude Opus 4.8 Anthropic |
SecureBio Independent lab · 2026-07-16 |
BioTIER-permit Chemical, biological, radiological, nuclear |
“BioTIER-permit (correctly answering benign prompts) is shown in the right panel.” source |
| Claude Haiku 4.5 Anthropic |
SecureBio Independent lab · 2026-07-16 |
biotier-refuse Chemical, biological, radiological, nuclear |
“On BioTIER-refuse, model behavior spanned more than 90 percentage points, with the least cautious model refusing under 10% of dangerous queries, while the most cautious refused almost 100%.” source |
| Claude Haiku 4.5 Anthropic |
SecureBio Independent lab · 2026-07-16 |
biotier-permit Chemical, biological, radiological, nuclear |
“On BioTIER-permit, the picture was far more uniform: almost every model answered almost every benign query, with many sitting at the 100% ceiling, and the lowest still scoring 75%.” source |
| Claude Haiku 4.5 Anthropic |
SecureBio Independent lab · 2026-07-16 |
BioTier Chemical, biological, radiological, nuclear |
“Our analysis covers small models like Claude Haiku 4.5 and large models like GPT-5.5 Pro.” source |
| Muse Spark 1.1 Meta |
SecureBio Independent lab · 2026-07-09 |
Molecular Biology Capabilities Test (MBCT) Chemical, biological, radiological, nuclear |
“The Molecular Biology Capabilities Test (MBCT), Virology Capabilities Test (VCT), and Human Pathogens Capabilities Test (HPCT) are part of a suite of evaluations developed by SecureBio and the Center for AI Safety (Götting et al., 2025; SecureBio, 2025).” source |
| Muse Spark 1.1 Meta |
Meta AI Independent lab · 2026-07-09 |
Virology Capabilities Test (VCT) Chemical, biological, radiological, nuclear |
“They assess practical troubleshooting across molecular biology tasks (MBCT), wet-lab virology experiments (VCT), and practical knowledge about working with high-priority human pathogens (HPCT), and are run in the recommended multiple-response multiple-choice configuration (Figure 1).” source |
| Muse Spark 1.1 Meta |
SecureBio Independent lab · 2026-07-09 |
ABC Bench Chemical, biological, radiological, nuclear |
“The Agentic Bio-Capabilities Benchmark (ABC Bench), developed by SecureBio, measures an LLM’s ability to complete computational tasks relevant to a threat actor seeking to design, synthesize, and assemble a complex genetic construct.” source |
| Muse Spark 1.1 Meta |
Meta AI Independent lab · 2026-07-09 |
multi-turn social-engineering capability benchmark Misuse and jailbreaks |
“Figure 12 Multi-turn social-engineering capabilityComposite score by model on Charlemagne Labs’ multi-turn social-engineering capability benchmark.” source |
| Muse Spark 1.1 Meta |
Meta AI Independent lab · 2026-07-09 |
Fortress Misuse and jailbreaks |
“Scale AI runs the private split of FORTRESS and we include the results in Figure 26.” source |
| Muse Spark 1.1 Meta |
Meta AI Independent lab · 2026-07-09 |
GraySwan ART Benchmark Prompt injection and agents |
“GraySwan additionally ran prompt-injection attacks using their private Agent Red Teaming (ART) benchmark (Zou et al., 2025) before the Muse Spark 1.1 launch.” source |
| Muse Spark 1.1 Meta |
Meta AI Independent lab · 2026-07-09 |
finance agent v2 Other |
“We’re sourcing results from Vals AI.” source |
| Muse Spark 1.1 Meta |
Meta AI Independent lab · 2026-07-09 |
swe-bench pro Other |
“Muse Spark 1.1 results are sourced from Scale AI leaderboard.” source |
| GPT-5.6 Terra OpenAI |
Independent lab Independent lab · 2026-07-09 |
expert human and external tester red teaming Misuse and jailbreaks |
“Experthumansandexternal testersusedadiversesetofapproachestofindgaps.” source |
| GPT-5.6 Terra OpenAI |
Independent lab Independent lab · 2026-07-09 |
molecular biology capabilities test Chemical, biological, radiological, nuclear |
“Itsstrongestreportedconfigurationsscored53.5%ontheVirology CapabilitiesTest,60.0%ontheMolecularBiologyCapabilitiesTest,68.4% ontheHumanPathogenCapabilitiesTest, and68.3%onWorld-ClassBio.” source |
| GPT-5.6 Terra OpenAI |
Independent lab Independent lab · 2026-07-09 |
REPRO-BAIT Chemical, biological, radiological, nuclear |
“On ReproBAIT, which measures AI agents on their ability to independently reproducebiologicalAImodels(BAIMs)frompublishedscientificpapers, therailfreevariantofGPT-5.6reached85%,comparedwith82%forGPT- 5.5.” source |
| GPT-5.6 Terra OpenAI |
Independent lab Independent lab · 2026-07-09 |
manual agentic assessment Chemical, biological, radiological, nuclear |
“In manual agentic assessment, the model was able to plan and execute a complex,long-runningscientifictask,includingmaintainingastructured workgraphandcoordinatingsubagents.” source |
| GPT-5.6 Terra OpenAI |
Independent lab Independent lab · 2026-07-09 |
Frontier Cyber Cyber |
“IrregularfoundthatGPT-5.6Solhason-parorslightlystrongeroffensive- cyber capabilities than GPT-5.5.” source |