AI Risk Wire

AI Safety

What AI providers state their models reached in safety testing, in their own words, and what outside testers found.

Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.

What providers state

Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.

136 of 136

ModelLevel, as the provider names itThe provider statesThe provider's sentence
Claude Haiku 5.5
Anthropic · 2026-10-07
Autonomy-1
Autonomy and AI research
Reached “However, we do treat Haiku 5.5 as meeting our CB-1 and Autonomy-1 thresholds, and we apply the corresponding mitigations from our RSP.” source
Claude Haiku 5.5
Anthropic · 2026-10-07
Autonomy-2
Autonomy and AI research
Not reached “As such, we determine that Haiku 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds.” source
Claude Haiku 5.5
Anthropic · 2026-10-07
CB-1
Chemical, biological, radiological, nuclear
Reached “However, we do treat Haiku 5.5 as meeting our CB-1 and Autonomy-1 thresholds, and we apply the corresponding mitigations from our RSP.” source
Claude Haiku 5.5
Anthropic · 2026-10-07
CB-2
Chemical, biological, radiological, nuclear
Not reached “As such, we determine that Haiku 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds.” source
Claude Sonnet 5.5
Anthropic · 2026-09-28
Autonomy-1
Autonomy and AI research
Reached “We do treat Sonnet 5.5 as meeting our CB-1 and Autonomy-1 thresholds and apply the corresponding mitigations from our RSP.” source
Claude Sonnet 5.5
Anthropic · 2026-09-28
Autonomy-2
Autonomy and AI research
Not reached “As such, our determination that Opus 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds also applies to Sonnet 5.5.” source
Claude Sonnet 5.5
Anthropic · 2026-09-28
CB-1
Chemical, biological, radiological, nuclear
Reached “We do treat Sonnet 5.5 as meeting our CB-1 and Autonomy-1 thresholds and apply the corresponding mitigations from our RSP.” source
Claude Sonnet 5.5
Anthropic · 2026-09-28
CB-2
Chemical, biological, radiological, nuclear
Not reached “As such, our determination that Opus 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds also applies to Sonnet 5.5.” source
Claude Opus 5.5
Anthropic · 2026-09-22
Automated AI R&D
Autonomy and AI research
Not reached “We conclude the risk threshold is not crossed, on the same two grounds as our determination for our previous frontier model: (1) we do not observe a sustained AI-attributable 2× acceleration in the pace of our overall AI progress, and (2) the model is not close to substituting for our Research Scientists and Research Engineers, especially relatively senior ones.” source
Claude Opus 5.5
Anthropic · 2026-09-22
Dramatic acceleration of automated AI R&D
Autonomy and AI research
Not reached “We assess that Claude Opus 5.5 does not cross the capability threshold for dramatic acceleration of automated AI R&D in our RSP.” source
Claude Opus 5.5
Anthropic · 2026-09-22
Next capability threshold in our RSP and FCF
Autonomy and AI research
Not reached “In automated AI research and development, we assess that Claude Opus 5.5 does not cross the next capability threshold in our RSP and FCF.” source
Claude Opus 5.5
Anthropic · 2026-09-22
CB-1
Chemical, biological, radiological, nuclear
Reached “On chemical and biological risks, we treat Opus 5.5 as having CB-1 capabilities (relating to the synthesis of non-novel weapons) but not CB-2 capabilities (relating to the synthesis of novel weapons).” source
Claude Opus 5.5
Anthropic · 2026-09-22
CB-2
Chemical, biological, radiological, nuclear
Not reached “On chemical and biological risks, we treat Opus 5.5 as having CB-1 capabilities (relating to the synthesis of non-novel weapons) but not CB-2 capabilities (relating to the synthesis of novel weapons).” source
Claude Opus 5.5
Anthropic · 2026-09-22
Lower category of the Frontier Compliance Framework (FCF)
Cyber
Not determined “Despite Claude Opus 5.5’s cyber capabilities, the model still falls within the lower category of capabilities set out in our Frontier Compliance Framework (FCF), and we do not see indications that the model is capable of novel offensive capability.” source
Grok 4.7
xAI · 2026-09-21
FAIF safety (Frontier AI Framework)
Chemical, biological, radiological, nuclear
Not reached “Grok 4.7 scores below the FAIF safety thresholds on dual-use knowledge, indicating limited actionable uplift for an already-trained actor.” source
GPT-Image-2.5-Flare
OpenAI · 2026-09-08
High
Chemical, biological, radiological, nuclear
Not reached “The results indicate that neither GPT-Image-2.5-Sunburst nor GPT-Image- 2.5-Flare cross the Bio High threshold or the Cyber High threshold.” source
GPT-Image-2.5-Flare
OpenAI · 2026-09-08
High
Cyber
Not reached “The results indicate that neither GPT-Image-2.5-Sunburst nor GPT-Image- 2.5-Flare cross the Bio High threshold or the Cyber High threshold.” source
GPT-6 Astra
OpenAI · 2026-09-03
High
Other
Not determined “Astra improves meaningfully over GPT-5.6 Sol on real internal research debugging tasks, scoring 78.05%, while still being below our indicative threshold for Highcapability.” source
Claude Mythos 5.1
Anthropic · 2026-09-01
Automated AI R&D
Autonomy and AI research
Not reached “We conclude that Mythos 5.1 does not cross the risk threshold, for the same two reasons we discussed in our assessment of Mythos 5: (1) we do not observe a sustained, AI-attributable 2× acceleration in the pace of progress on AI R&D, and (2) the model is not close to substituting for Anthropic Research Scientists and Research Engineers, especially relatively senior ones.” source
Claude Mythos 5.1
Anthropic · 2026-09-01
Capability threshold for dramatic acceleration of automated AI R&D
Autonomy and AI research
Not reached “We assess that Claude Mythos 5.1 does not cross the capability threshold for dramatic acceleration of automated AI R&D in our RSP and FCF, although we think it provides a meaningful acceleration to AI R&D efforts in many circumstances, and is somewhat more capable in this domain than the models described in our previous system cards.” source
Claude Mythos 5.1
Anthropic · 2026-09-01
Tier 1
Cyber
Not determined “Although Mythos 5.1 is in Tier 1, it is getting closer to Tier 2, completing more and more autonomous tasks.” source
Grok 4.6
xAI · 2026-08-12
FAIF safety (Frontier AI Framework)
Chemical, biological, radiological, nuclear
Not reached “Grok 4.6 scores below the FAIF safety thresholds on dual-use knowledge, indicating limited actionable uplift for an already-trained actor.” source
Muse Glimmer 30B
Meta · 2026-08
Moderate or lower
Autonomy and AI research
Not determined “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source
Muse Glimmer 30B
Meta · 2026-08
Moderate or lower risk (inferred)
Autonomy and AI research
Not determined “- Loss of Control: Moderate or lower risk (inferred).” source
Muse Glimmer 30B
Meta · 2026-08
Moderate or lower
Chemical, biological, radiological, nuclear
Not reached “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source
Muse Glimmer 30B
Meta · 2026-08
Moderate or lower
Cyber
Not determined “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source
Muse Glimmer 30B
Meta · 2026-08
Moderate or lower risk (inferred)
Cyber
Not determined “- Cyber: Moderate or lower risk (inferred);” source
Claude Opus 5
Anthropic · 2026-07-24
CB-1
Chemical, biological, radiological, nuclear
Reached “On chemical and biological risks, we treat the model as having CB-1 capabilities (relating to the synthesis of non-novel weapons), but not CB-2 capabilities (relating to the synthesis of novel weapons).” source
Claude Opus 5
Anthropic · 2026-07-24
CB-2
Chemical, biological, radiological, nuclear
Not reached “On chemical and biological risks, we treat the model as having CB-1 capabilities (relating to the synthesis of non-novel weapons), but not CB-2 capabilities (relating to the synthesis of novel weapons).” source
Claude Sonnet 5
Anthropic · 2026-06-30
CB-2
Chemical, biological, radiological, nuclear
Not reached “Sonnet 5 does not cross the CB-2 threshold.” source
Claude Sonnet 5
Anthropic · 2026-06-30
Notable capability
Chemical, biological, radiological, nuclear
Not determined “We consider performance on the virology tasks notable if: ● The model scores above 0.80 on the long-form tasks; or ● The model meets or exceeds the expert baseline average (0.221) on the multimodal task.” source
GPT-Rosalind-5.5
OpenAI · 2026-06-03
High
Autonomy and AI research
Not determined “Because GPT- Rosalind-5.5 was trained on additional biology data unrelated to AI self-improvement, we rely on GPT-5.5’s existing Preparedness assessment for AI Self-Improvement, where GPT-5.5 did not meet the threshold for High capability.” source
GPT-Rosalind-5.5
OpenAI · 2026-06-03
High
Chemical, biological, radiological, nuclear
Reached “Under our Preparedness Framework, the Preparedness evaluations in the Biological and Chemical domain met our threshold for High capability while falling below the threshold for Critical.” source
Gemini 3.1 Flash-Lite Image
Google DeepMind · 2026-06
CCL (Critical Capability Level)
Other
Not determined “Our assessments have shown that Gemini 3.1 Flash-Lite Image is less capable than Gemini 3.1 Pro, therefore based on Gemini 3.1 Pro, we are confident that that Gemini 3.1 Flash-Lite Image is also unlikely to reach any CCLs.” source
Claude Opus 4.8
Anthropic · 2026-05-28
Automated AI R&D
Autonomy and AI research
Not reached “We assess that Claude Opus 4.8 does not cross the automated AI-R&D capability threshold.” source
Claude Opus 4.8
Anthropic · 2026-05-28
CB-1
Chemical, biological, radiological, nuclear
Not determined “We believe it is hard to be confident whether a model passes this threshold, but (as with previous recent models) our results are consistent with Opus 4.8 providing significant uplift to individuals or groups with basic technical backgrounds for this threat model.” source
Claude Opus 4.8
Anthropic · 2026-05-28
CB-2
Chemical, biological, radiological, nuclear
Not determined “Because Opus 4.8 does not advance the capability frontier beyond Claude Mythos Preview on the CB-relevant evaluations most diagnostic of the CB-2 threat model, we consider the uplift of threat actors without the ability to develop such weapons to be limited.” source
GPT-5.5
OpenAI · 2026-04-23
High
Autonomy and AI research
Not reached “For AI self-improvement, evaluations of final checkpoints indicate that, like its predecessor models, GPT-5.5 does not have a plausible chance of reaching a High threshold.” source
GPT-5.5
OpenAI · 2026-04-23
High
Chemical, biological, radiological, nuclear
Reached “As we did for GPT-5.4 Thinking before it, we are continuing to treat GPT-5.5 as High capability in the Biological and Chemical domain.” source
GPT-5.5
OpenAI · 2026-04-23
High
Cyber
Reached “As we did for GPT-5.3-Codex and GPT-5.4-thinking, we are treating GPT-5.5 as High capability in the Cybersecurity domain, but below Critical.” source

Page 1 of 4 · Next

What outside testers report

Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.

1957 of 1957

ModelTesterTestWhat was found
GLM-5.2
Z.ai (Zhipu AI)
US CAISI (NIST)
Government evaluator · 2026-07-17
automated benchmark evaluations for safeguards coverage and public jailbreaking robustness against sensitive biological queries
Chemical, biological, radiological, nuclear
“Figure 6.4: On automated benchmark evaluations for safeguards coverage and public jailbreaking robustness against sensitive biological queries, GLM-5.2 complied at a higher rate than tested US models, both with and without the use of a public jailbreak.” source
GLM-5.2
Z.ai (Zhipu AI)
US CAISI (NIST)
Government evaluator · 2026-07-17
automated benchmark evaluations against sensitive biological queries
Chemical, biological, radiological, nuclear
“Figure 6.5: On automated benchmark evaluations against sensitive biological queries, GLM-5.2 responded with a higher rate of request-relevant detail than tested US models.” source
GLM-5.2
Z.ai (Zhipu AI)
US CAISI (NIST)
Government evaluator · 2026-07-17
caisi biology safeguards
Chemical, biological, radiological, nuclear
“sensitive biological queries, GLM-5.2 responded with a higher rate of request-relevant detail than tested US models.” source
GLM-5.2
Z.ai (Zhipu AI)
US CAISI (NIST)
Government evaluator · 2026-07-17
caisi cyber safeguards
Cyber
“CAISI cyber safeguards . A private benchmark created by CAISI that evaluates models against a dataset of 30 single-turn misuse requests related to offensive cyber.” source
GLM-5.2
Z.ai (Zhipu AI)
US CAISI (NIST)
Government evaluator · 2026-07-17
hijackeval
Prompt injection and agents
“HijackEval . A private benchmark created by CAISI that evaluates models’ robustness against agent hijacking attacks.” source
GLM-5.2
Z.ai (Zhipu AI)
US CAISI (NIST)
Government evaluator · 2026-07-17
portbench
Other
“Budgets were set to 1M weighted tokens for PortBench and CTF-Archive-Diamond, and 500k weighted tokens for SWE-Bench 4 OpenRouter (2026) Z.ai: GLM 5.2 .” source
GLM-5.2
Z.ai (Zhipu AI)
UK AI Security Institute
Government evaluator · 2026-07-17
narrow cyber tasks
Cyber
“On these tasks, GLM-5.2 performs comparably to the most cyber-capable models released 4 months before it (Opus 4.6 and GPT-5.3-Codex), and this holds across all four difficulty levels.” source
GLM-5.2
Z.ai (Zhipu AI)
UK AI Security Institute
Government evaluator · 2026-07-17
cyber ranges
Cyber
“On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it).” source
GLM-5.2
Z.ai (Zhipu AI)
UK AI Security Institute
Government evaluator · 2026-07-17
The Last Ones (TLO)
Cyber
“Notably, GLM-5.2 reached step 7 with marginally fewer tokens than any other model on average, tracking Opus 4.6’s trajectory to step 11 before stalling.” source
DeepSeek-V4-Pro
DeepSeek
UK AI Security Institute
Government evaluator · 2026-07-17
AISI's narrow cyber tasks
Cyber
“DeepSeek’s V4-Pro is comparable to Opus 4.5, released 5 months before it.” source
DeepSeek-V4-Pro
DeepSeek
UK AI Security Institute
Government evaluator · 2026-07-17
cyber range "The Last Ones" (TLO)
Cyber
“On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it).” source
DeepSeek-V4-Pro
DeepSeek
UK AI Security Institute
Government evaluator · 2026-07-17
refusal behaviour on narrow cyber tasks
Cyber
“DeepSeek V4-Pro occasionally refused narrow cyber tasks, mainly in reverse engineering, which we circumvented simply via a small number of repeat attempts at refused tasks.” source
Gemini 3.1 Pro
Google DeepMind
SecureBio
Independent lab · 2026-07-16
biotier-refuse
Chemical, biological, radiological, nuclear
“Google’s update to their mental health related safeguards made headlines , but we also identified changes to their biology safeguards, with an almost 30% increase in the refusal behavior of Gemini 3.1 Pro on BioTIER-refuse due to introduction of API-level refusals.” source
Gemini 3.1 Pro
Google DeepMind
SecureBio
Independent lab · 2026-07-16
biotier-permit
Chemical, biological, radiological, nuclear
“Importantly, this increase in BioTIER-refuse performance had no impact on BioTIER-permit compliance .” source
Gemini 3.1 Pro
Google DeepMind
SecureBio
Independent lab · 2026-07-16
BioTier
Chemical, biological, radiological, nuclear
“Google’s update to their mental health related safeguards made headlines, but we also identified changes to their biology safeguards, with an almost 30% increase in the refusal behavior of Gemini 3.1 Pro on BioTIER-refuse due to introduction of API-level refusals.” source
Gemini 2.5 Pro
Google DeepMind
SaferAI
Independent lab · 2026-07-16
situational-awareness suite
Autonomy and AI research
“Whilst, the situational-awareness evidence provides moderate confirmation for the narrow claim that current models do not reliably display the specific situationally aware, constraint-lifting behaviours measured by this suite.” source
Gemini 2.5 Pro
Google DeepMind
SaferAI
Independent lab · 2026-07-16
cover_your_tracks
Deception and sabotage
“cover_your_tracks is especially valuable because it combines several relevant covert subskills: identifying a subversive opportunity, altering or deleting traces, avoiding downstream detection, and completing a cover-up end-to-end.” source
Gemini 2.5 Pro
Google DeepMind
SaferAI
Independent lab · 2026-07-16
classifier_evasion
Deception and sabotage
“classifier_evasion is also comparatively informative because it more directly tests whether the model can evade an active monitoring system by phrasing prohibited content in a way that avoids detection.” source
Gemini 2.5 Pro
Google DeepMind
SaferAI
Independent lab · 2026-07-16
strategic_rule_breaking
Deception and sabotage
“strategic_rule_breaking contributes less.” source
Gemini 2.5 Pro
Google DeepMind
SaferAI
Independent lab · 2026-07-16
oversight_pattern
Deception and sabotage
“The oversight_pattern task is also weak as evidence for stealth specifically.” source
Gemini 2.5 Pro
Google DeepMind
SaferAI
Independent lab · 2026-07-16
decision_sabotage
Deception and sabotage
“decision _sabotagein particular measures a real sabotage channel, but not a cleanly stealth-specific one.” source
GPT-5.5 Pro
OpenAI
SecureBio
Independent lab · 2026-07-16
biotier-refuse
Chemical, biological, radiological, nuclear
“At the same time, we saw GPT-5.5 Pro and Grok 4.20 drop in performance on BioTIER-refuse, while Claude Opus 4.8 showed little change at all.” source
GPT-5.5 Pro
OpenAI
SecureBio
Independent lab · 2026-07-16
biotier-permit
Chemical, biological, radiological, nuclear
“We split these sets into two evaluation components: BioTIER-refuse (CA + BD: which models should decline) and BioTIER-permit (RB: which models should answer).” source
Claude Opus 4.8
Anthropic
SecureBio
Independent lab · 2026-07-16
BioTIER-permit
Chemical, biological, radiological, nuclear
“BioTIER-permit (correctly answering benign prompts) is shown in the right panel.” source
Claude Haiku 4.5
Anthropic
SecureBio
Independent lab · 2026-07-16
biotier-refuse
Chemical, biological, radiological, nuclear
“On BioTIER-refuse, model behavior spanned more than 90 percentage points, with the least cautious model refusing under 10% of dangerous queries, while the most cautious refused almost 100%.” source
Claude Haiku 4.5
Anthropic
SecureBio
Independent lab · 2026-07-16
biotier-permit
Chemical, biological, radiological, nuclear
“On BioTIER-permit, the picture was far more uniform: almost every model answered almost every benign query, with many sitting at the 100% ceiling, and the lowest still scoring 75%.” source
Claude Haiku 4.5
Anthropic
SecureBio
Independent lab · 2026-07-16
BioTier
Chemical, biological, radiological, nuclear
“Our analysis covers small models like Claude Haiku 4.5 and large models like GPT-5.5 Pro.” source
Muse Spark 1.1
Meta
SecureBio
Independent lab · 2026-07-09
Molecular Biology Capabilities Test (MBCT)
Chemical, biological, radiological, nuclear
“The Molecular Biology Capabilities Test (MBCT), Virology Capabilities Test (VCT), and Human Pathogens Capabilities Test (HPCT) are part of a suite of evaluations developed by SecureBio and the Center for AI Safety (Götting et al., 2025; SecureBio, 2025).” source
Muse Spark 1.1
Meta
Meta AI
Independent lab · 2026-07-09
Virology Capabilities Test (VCT)
Chemical, biological, radiological, nuclear
“They assess practical troubleshooting across molecular biology tasks (MBCT), wet-lab virology experiments (VCT), and practical knowledge about working with high-priority human pathogens (HPCT), and are run in the recommended multiple-response multiple-choice configuration (Figure 1).” source
Muse Spark 1.1
Meta
SecureBio
Independent lab · 2026-07-09
ABC Bench
Chemical, biological, radiological, nuclear
“The Agentic Bio-Capabilities Benchmark (ABC Bench), developed by SecureBio, measures an LLM’s ability to complete computational tasks relevant to a threat actor seeking to design, synthesize, and assemble a complex genetic construct.” source
Muse Spark 1.1
Meta
Meta AI
Independent lab · 2026-07-09
multi-turn social-engineering capability benchmark
Misuse and jailbreaks
“Figure 12 Multi-turn social-engineering capabilityComposite score by model on Charlemagne Labs’ multi-turn social-engineering capability benchmark.” source
Muse Spark 1.1
Meta
Meta AI
Independent lab · 2026-07-09
Fortress
Misuse and jailbreaks
“Scale AI runs the private split of FORTRESS and we include the results in Figure 26.” source
Muse Spark 1.1
Meta
Meta AI
Independent lab · 2026-07-09
GraySwan ART Benchmark
Prompt injection and agents
“GraySwan additionally ran prompt-injection attacks using their private Agent Red Teaming (ART) benchmark (Zou et al., 2025) before the Muse Spark 1.1 launch.” source
Muse Spark 1.1
Meta
Meta AI
Independent lab · 2026-07-09
finance agent v2
Other
“We’re sourcing results from Vals AI.” source
Muse Spark 1.1
Meta
Meta AI
Independent lab · 2026-07-09
swe-bench pro
Other
“Muse Spark 1.1 results are sourced from Scale AI leaderboard.” source
GPT-5.6 Terra
OpenAI
Independent lab
Independent lab · 2026-07-09
expert human and external tester red teaming
Misuse and jailbreaks
“Experthumansandexternal testersusedadiversesetofapproachestofindgaps.” source
GPT-5.6 Terra
OpenAI
Independent lab
Independent lab · 2026-07-09
molecular biology capabilities test
Chemical, biological, radiological, nuclear
“Itsstrongestreportedconfigurationsscored53.5%ontheVirology CapabilitiesTest,60.0%ontheMolecularBiologyCapabilitiesTest,68.4% ontheHumanPathogenCapabilitiesTest, and68.3%onWorld-ClassBio.” source
GPT-5.6 Terra
OpenAI
Independent lab
Independent lab · 2026-07-09
REPRO-BAIT
Chemical, biological, radiological, nuclear
“On ReproBAIT, which measures AI agents on their ability to independently reproducebiologicalAImodels(BAIMs)frompublishedscientificpapers, therailfreevariantofGPT-5.6reached85%,comparedwith82%forGPT- 5.5.” source
GPT-5.6 Terra
OpenAI
Independent lab
Independent lab · 2026-07-09
manual agentic assessment
Chemical, biological, radiological, nuclear
“In manual agentic assessment, the model was able to plan and execute a complex,long-runningscientifictask,includingmaintainingastructured workgraphandcoordinatingsubagents.” source
GPT-5.6 Terra
OpenAI
Independent lab
Independent lab · 2026-07-09
Frontier Cyber
Cyber
“IrregularfoundthatGPT-5.6Solhason-parorslightlystrongeroffensive- cyber capabilities than GPT-5.5.” source

Previous · Page 26 of 49 · Next