AI Safety

What AI providers state their models reached in safety testing, in their own words, and what outside testers found.

Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.

What providers state

Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.

136 of 136

Model	Level, as the provider names it	The provider states	The provider's sentence
Muse Spark Meta	Moderate or lower Autonomy and AI research	Not reached	“We assess loss of control risk as “moderate or lower”.” source
Muse Spark Meta	High Chemical, biological, radiological, nuclear	Not determined	“Those evaluations identified elevated risks prior to mitigations, with Chemical and Biological capabilities assessed as likely reaching the “high risk” category under the Advanced AI Scaling Framework before safeguards were applied.” source
Muse Spark Meta	Moderate or lower Chemical, biological, radiological, nuclear	Not reached	“However, after applying mitigations, the residual risk of the model is that of “moderate or lower risk.”” source
Muse Spark Meta	Below the level required to meaningfully enable the cyber threat scenarios defin Cyber	Not reached	“Muse Spark’s offensive cyber capabilities remain below the level required to meaningfully enable the cyber threat scenarios defined in the Advanced AI Scaling Framework.” source
Muse Spark Meta	Moderate or lower Cyber	Not reached	“We assess cybersecurity risk as “moderate or lower” with respect to the threat scenarios outlined in Meta’s Advanced AI Scaling Framework.” source
Gemini 4 Argon Google DeepMind	TCL (Tracked Capability Level) Autonomy and AI research	Not determined	“In light of this, we assess that we cannot rule out the TCL.” source
Gemini 4 Argon Google DeepMind	TCL (Tracked Capability Level) Chemical, biological, radiological, nuclear	Not reached	“We can rule out the TCL for the chemical, radiological, and nuclear domains with reasonable confidence.” source
Gemini 4 Argon Google DeepMind	CCL (Critical Capability Level) Deception and sabotage	Not reached	“we assess its overall capabilities to be below the CCL threshold.” source
Gemini 4 Argon Google DeepMind	CCL (Critical Capability Level) Misuse and jailbreaks	Not reached	“Gemini 4 Argon demonstrates some ability to influence user beliefs and behaviors during one-on-one direct conversations in human behavioral studies, and may provide utility in the planning stage of a malicious influence operation but we assess its overall capabilities to be below the CCL threshold.” source
Gemini 4 Argon Google DeepMind	CCL (Critical Capability Level) Other	Not reached	“Gemini 4 Argon's performance remains below our CCL threshold.” source
Gemini 3.7 Flash Google DeepMind	CCL (Critical Capability Level) Autonomy and AI research	Not reached	“The model does not reach the CCL alert threshold.” source
Gemini 3.7 Flash Google DeepMind	CCL (Critical Capability Level) Chemical, biological, radiological, nuclear	Not reached	“We can rule out the CCL for the CBRN domain with reasonable conﬁdence based on the results from our testing.” source
Gemini 3.7 Flash Google DeepMind	TCL (Tracked Capability Level) Chemical, biological, radiological, nuclear	Not reached	“We can rule out the TCL for the CBRN domain with reasonable conﬁdence based on the results from our testing.” source
Gemini 3.7 Flash Google DeepMind	CCL (Critical Capability Level) Cyber	Not reached	“Gemini 3.7 Flash reaches the alert threshold for this CCL, but not the CCL.” source
Gemini 3.7 Flash Google DeepMind	TCL (Tracked Capability Level) Deception and sabotage	Not reached	“The model does not reach the TCL.” source
Gemini 3.5 Transcribe Live Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that none of Gemini 3.5 Translate, Gemini 3.5 Transcribe, or Gemini 3.5 Transcribe Live have meaningful new T/CCL-relevant capabilities or material increases in performance with respect to Frontier Safety compared to Gemini 3.1 Pro and Gemini 3.7 Flash, therefore we are conﬁdent that none of them are likely to reach any CCLs.” source
Gemini 3.5 Transcribe Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that none of Gemini 3.5 Translate, Gemini 3.5 Transcribe, or Gemini 3.5 Transcribe Live have meaningful new T/CCL-relevant capabilities or material increases in performance with respect to Frontier Safety compared to Gemini 3.1 Pro and Gemini 3.7 Flash, therefore we are conﬁdent that none of them are likely to reach any CCLs.” source
Gemini 3.5 Flash-Lite Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that, while Gemini 3.5 Flash-Lite excels at agents and coding, it does not have meaningful new capabilities or material increases in performance with respect to the domains outlined in our Frontier Safety Framework compared to Gemini 3.1 Pro; therefore, based on Gemini 3.1 Pro results, we are confident that Gemini 3.5 Flash-Lite is also unlikely to reach any CCLs.” source
Gemini 3.5 Audio Google DeepMind	Tracked or critical capability level (t/ccls) Other	Not reached	“they did not reach any Tracked or Critical Capability Levels (T/CCLs) outlined in our Frontier Safety Framework .” source
Gemini 3.1 Pro Google DeepMind	Uplift level 1 Chemical, biological, radiological, nuclear	Not reached	“We found that the model remains below alert thresholds for the CBRN, harmful manipulation, machine learning R&D, and misalignment CCLs.” source
Gemini 3.1 Pro Google DeepMind	Uplift level 1 Cyber	Not reached	“As previous models passed the alert threshold for cyber, we performed more additional testing in this domain on Gemini 3.1 Pro with and without Deep Think mode, and found that the model remains below the cyber CCL.” source
Gemini 3.1 Flash-Lite Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that Gemini 3.1 Flash-Lite is less capable than Gemini 3.1 Pro, therefore based on Gemini 3.1 Pro, we are confident that Gemini 3.1 Flash-Lite is also unlikely to reach any CCLs.” source
Gemini 3.1 Flash Audio Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that Gemini 3.1 Flash Audio is less capable than Gemini 3.1 Pro.” source
Gemini 2.5 Pro Google DeepMind	Machine Learning R&D Uplift Level 1 Autonomy and AI research	Not determined	“given Gemini 2.5 Pro Preview's poor median performance on RE-Bench relative to experts, the model likely lacks the necessary capabilities to automate or signiﬁcantly uplift any signiﬁcant fraction of the research process.” source
Gemini 2.5 Pro Google DeepMind	Instrumental Reasoning Level 1 Deception and sabotage	Not reached	“Instru- mental Reasoning Level 1 ✘ CCL not reached” source
Gemini 2.5 Pro Google DeepMind	Instrumental Reasoning Level 2 Deception and sabotage	Not reached	“Instru- mental Reasoning Level 2 ✘ CCL not reached” source
Gemini 2.5 Deep Think Google DeepMind	Uplift level 1 Chemical, biological, radiological, nuclear	Not determined	“Our assessment is that the model has enough technical knowledge in certain CBRN scenarios and stages to be considered at early warning.” source
Gemini 2.5 Deep Think Google DeepMind	Cyber Uplift Level 1 Cyber	Not determined	“CCL reached? No, but the early warning alert threshold was originally reached by Gemini 2.5 Pro and continues to be met by Gemini 2.5 Deep Think.” source
Gemini 2.5 Deep Think Google DeepMind	Uplift level 1 Cyber	Not determined	“For Cyber Uplift Level 1 , the early warning alert threshold was originally reached by Gemini 2.5 Pro and continues to be met by Gemini 2.5 Deep Think.” source
GPT-5.5 Instant OpenAI	Below high capability Autonomy and AI research	Not reached	“GPT-5.5 Instant is considered below High Capability.” source
GPT-5.5 Instant OpenAI	High Autonomy and AI research	Not reached	“For AI Self-Improvement, evaluations of final checkpoints indicate that, like its predecessor models, GPT-5.5 Instant does not have a plausible chance of reaching a High threshold.” source
GPT-5.5 Instant OpenAI	High Chemical, biological, radiological, nuclear	Reached	“GPT-5.5 Instant is our first Instant model to be treated as High Capability in the Biological and Chemical domain, as well as in the Cybersecurity domain.” source
GPT-5.5 Instant OpenAI	High Cyber	Reached	“this is the first Instant model that we are treating as High capability in our Cybersecurity and Biological & Chemical Preparedness categories, and implementing appropriate safeguards.” source
GPT-5 OpenAI	High Chemical, biological, radiological, nuclear	Reached	“Similarly to ChatGPT agent, we have decided to treat gpt-5-thinking as High capability in the Biological and Chemical domain under our Preparedness Framework, activating the associated safeguards.” source
GPT-5 OpenAI	High cyber risk Cyber	Not reached	“The gpt-5 model series does not meet the threshold for high cyber risk.” source
Claude Sonnet 4.5 Anthropic	AI R&D-4 Autonomy and AI research	Not reached	“All models tested remain below our pre-committed rule-out threshold of 0.6.” source
Claude Sonnet 4.5 Anthropic	ASL-3 Autonomy and AI research	Not determined	“Exhibited improved autonomy capabilities in software engineering and AI research tasks, though still below the ASL-4 rule-out threshold.” source
Claude Sonnet 4.5 Anthropic	ASL-3 Chemical, biological, radiological, nuclear	Not determined	“Remained well below ASL-4 thresholds across all domains of concern;” source
Claude Sonnet 4.5 Anthropic	ASL-4 rule-out Chemical, biological, radiological, nuclear	Not determined	“However, given that this type of logic can have a general frog-boiling effect—where gradual changes mask the crossing of important thresholds—we do not use this evaluation as indicative of a strong ASL-4 rule-out.” source
Claude Sonnet 4.5 Anthropic	ASL-3 Cyber	Not determined	“Showed meaningful improvements in cyber capabilities, particularly in vulnerability discovery and code analysis;” source

Previous · Page 3 of 4 · Next

What outside testers report

Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.

1957 of 1957

Model	Tester	Test	What was found
GLM-5.2 Z.ai (Zhipu AI)	US CAISI (NIST) Government evaluator · 2026-07-17	automated benchmark evaluations for safeguards coverage and public jailbreaking robustness against sensitive biological queries Chemical, biological, radiological, nuclear	“Figure 6.4: On automated benchmark evaluations for safeguards coverage and public jailbreaking robustness against sensitive biological queries, GLM-5.2 complied at a higher rate than tested US models, both with and without the use of a public jailbreak.” source
GLM-5.2 Z.ai (Zhipu AI)	US CAISI (NIST) Government evaluator · 2026-07-17	automated benchmark evaluations against sensitive biological queries Chemical, biological, radiological, nuclear	“Figure 6.5: On automated benchmark evaluations against sensitive biological queries, GLM-5.2 responded with a higher rate of request-relevant detail than tested US models.” source
GLM-5.2 Z.ai (Zhipu AI)	US CAISI (NIST) Government evaluator · 2026-07-17	caisi biology safeguards Chemical, biological, radiological, nuclear	“sensitive biological queries, GLM-5.2 responded with a higher rate of request-relevant detail than tested US models.” source
GLM-5.2 Z.ai (Zhipu AI)	US CAISI (NIST) Government evaluator · 2026-07-17	caisi cyber safeguards Cyber	“CAISI cyber safeguards . A private benchmark created by CAISI that evaluates models against a dataset of 30 single-turn misuse requests related to offensive cyber.” source
GLM-5.2 Z.ai (Zhipu AI)	US CAISI (NIST) Government evaluator · 2026-07-17	hijackeval Prompt injection and agents	“HijackEval . A private benchmark created by CAISI that evaluates models’ robustness against agent hijacking attacks.” source
GLM-5.2 Z.ai (Zhipu AI)	US CAISI (NIST) Government evaluator · 2026-07-17	portbench Other	“Budgets were set to 1M weighted tokens for PortBench and CTF-Archive-Diamond, and 500k weighted tokens for SWE-Bench 4 OpenRouter (2026) Z.ai: GLM 5.2 .” source
GLM-5.2 Z.ai (Zhipu AI)	UK AI Security Institute Government evaluator · 2026-07-17	narrow cyber tasks Cyber	“On these tasks, GLM-5.2 performs comparably to the most cyber-capable models released 4 months before it (Opus 4.6 and GPT-5.3-Codex), and this holds across all four difficulty levels.” source
GLM-5.2 Z.ai (Zhipu AI)	UK AI Security Institute Government evaluator · 2026-07-17	cyber ranges Cyber	“On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it).” source
GLM-5.2 Z.ai (Zhipu AI)	UK AI Security Institute Government evaluator · 2026-07-17	The Last Ones (TLO) Cyber	“Notably, GLM-5.2 reached step 7 with marginally fewer tokens than any other model on average, tracking Opus 4.6’s trajectory to step 11 before stalling.” source
DeepSeek-V4-Pro DeepSeek	UK AI Security Institute Government evaluator · 2026-07-17	AISI's narrow cyber tasks Cyber	“DeepSeek’s V4-Pro is comparable to Opus 4.5, released 5 months before it.” source
DeepSeek-V4-Pro DeepSeek	UK AI Security Institute Government evaluator · 2026-07-17	cyber range "The Last Ones" (TLO) Cyber	“On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it).” source
DeepSeek-V4-Pro DeepSeek	UK AI Security Institute Government evaluator · 2026-07-17	refusal behaviour on narrow cyber tasks Cyber	“DeepSeek V4-Pro occasionally refused narrow cyber tasks, mainly in reverse engineering, which we circumvented simply via a small number of repeat attempts at refused tasks.” source
Gemini 3.1 Pro Google DeepMind	SecureBio Independent lab · 2026-07-16	biotier-refuse Chemical, biological, radiological, nuclear	“Google’s update to their mental health related safeguards made headlines , but we also identified changes to their biology safeguards, with an almost 30% increase in the refusal behavior of Gemini 3.1 Pro on BioTIER-refuse due to introduction of API-level refusals.” source
Gemini 3.1 Pro Google DeepMind	SecureBio Independent lab · 2026-07-16	biotier-permit Chemical, biological, radiological, nuclear	“Importantly, this increase in BioTIER-refuse performance had no impact on BioTIER-permit compliance .” source
Gemini 3.1 Pro Google DeepMind	SecureBio Independent lab · 2026-07-16	BioTier Chemical, biological, radiological, nuclear	“Google’s update to their mental health related safeguards made headlines, but we also identified changes to their biology safeguards, with an almost 30% increase in the refusal behavior of Gemini 3.1 Pro on BioTIER-refuse due to introduction of API-level refusals.” source
Gemini 2.5 Pro Google DeepMind	SaferAI Independent lab · 2026-07-16	situational-awareness suite Autonomy and AI research	“Whilst, the situational-awareness evidence provides moderate confirmation for the narrow claim that current models do not reliably display the specific situationally aware, constraint-lifting behaviours measured by this suite.” source
Gemini 2.5 Pro Google DeepMind	SaferAI Independent lab · 2026-07-16	cover_your_tracks Deception and sabotage	“cover_your_tracks is especially valuable because it combines several relevant covert subskills: identifying a subversive opportunity, altering or deleting traces, avoiding downstream detection, and completing a cover-up end-to-end.” source
Gemini 2.5 Pro Google DeepMind	SaferAI Independent lab · 2026-07-16	classifier_evasion Deception and sabotage	“classifier_evasion is also comparatively informative because it more directly tests whether the model can evade an active monitoring system by phrasing prohibited content in a way that avoids detection.” source
Gemini 2.5 Pro Google DeepMind	SaferAI Independent lab · 2026-07-16	strategic_rule_breaking Deception and sabotage	“strategic_rule_breaking contributes less.” source
Gemini 2.5 Pro Google DeepMind	SaferAI Independent lab · 2026-07-16	oversight_pattern Deception and sabotage	“The oversight_pattern task is also weak as evidence for stealth specifically.” source
Gemini 2.5 Pro Google DeepMind	SaferAI Independent lab · 2026-07-16	decision_sabotage Deception and sabotage	“decision _sabotagein particular measures a real sabotage channel, but not a cleanly stealth-specific one.” source
GPT-5.5 Pro OpenAI	SecureBio Independent lab · 2026-07-16	biotier-refuse Chemical, biological, radiological, nuclear	“At the same time, we saw GPT-5.5 Pro and Grok 4.20 drop in performance on BioTIER-refuse, while Claude Opus 4.8 showed little change at all.” source
GPT-5.5 Pro OpenAI	SecureBio Independent lab · 2026-07-16	biotier-permit Chemical, biological, radiological, nuclear	“We split these sets into two evaluation components: BioTIER-refuse (CA + BD: which models should decline) and BioTIER-permit (RB: which models should answer).” source
Claude Opus 4.8 Anthropic	SecureBio Independent lab · 2026-07-16	BioTIER-permit Chemical, biological, radiological, nuclear	“BioTIER-permit (correctly answering benign prompts) is shown in the right panel.” source
Claude Haiku 4.5 Anthropic	SecureBio Independent lab · 2026-07-16	biotier-refuse Chemical, biological, radiological, nuclear	“On BioTIER-refuse, model behavior spanned more than 90 percentage points, with the least cautious model refusing under 10% of dangerous queries, while the most cautious refused almost 100%.” source
Claude Haiku 4.5 Anthropic	SecureBio Independent lab · 2026-07-16	biotier-permit Chemical, biological, radiological, nuclear	“On BioTIER-permit, the picture was far more uniform: almost every model answered almost every benign query, with many sitting at the 100% ceiling, and the lowest still scoring 75%.” source
Claude Haiku 4.5 Anthropic	SecureBio Independent lab · 2026-07-16	BioTier Chemical, biological, radiological, nuclear	“Our analysis covers small models like Claude Haiku 4.5 and large models like GPT-5.5 Pro.” source
Muse Spark 1.1 Meta	SecureBio Independent lab · 2026-07-09	Molecular Biology Capabilities Test (MBCT) Chemical, biological, radiological, nuclear	“The Molecular Biology Capabilities Test (MBCT), Virology Capabilities Test (VCT), and Human Pathogens Capabilities Test (HPCT) are part of a suite of evaluations developed by SecureBio and the Center for AI Safety (Götting et al., 2025; SecureBio, 2025).” source
Muse Spark 1.1 Meta	Meta AI Independent lab · 2026-07-09	Virology Capabilities Test (VCT) Chemical, biological, radiological, nuclear	“They assess practical troubleshooting across molecular biology tasks (MBCT), wet-lab virology experiments (VCT), and practical knowledge about working with high-priority human pathogens (HPCT), and are run in the recommended multiple-response multiple-choice configuration (Figure 1).” source
Muse Spark 1.1 Meta	SecureBio Independent lab · 2026-07-09	ABC Bench Chemical, biological, radiological, nuclear	“The Agentic Bio-Capabilities Benchmark (ABC Bench), developed by SecureBio, measures an LLM’s ability to complete computational tasks relevant to a threat actor seeking to design, synthesize, and assemble a complex genetic construct.” source
Muse Spark 1.1 Meta	Meta AI Independent lab · 2026-07-09	multi-turn social-engineering capability benchmark Misuse and jailbreaks	“Figure 12 Multi-turn social-engineering capabilityComposite score by model on Charlemagne Labs’ multi-turn social-engineering capability benchmark.” source
Muse Spark 1.1 Meta	Meta AI Independent lab · 2026-07-09	Fortress Misuse and jailbreaks	“Scale AI runs the private split of FORTRESS and we include the results in Figure 26.” source
Muse Spark 1.1 Meta	Meta AI Independent lab · 2026-07-09	GraySwan ART Benchmark Prompt injection and agents	“GraySwan additionally ran prompt-injection attacks using their private Agent Red Teaming (ART) benchmark (Zou et al., 2025) before the Muse Spark 1.1 launch.” source
Muse Spark 1.1 Meta	Meta AI Independent lab · 2026-07-09	finance agent v2 Other	“We’re sourcing results from Vals AI.” source
Muse Spark 1.1 Meta	Meta AI Independent lab · 2026-07-09	swe-bench pro Other	“Muse Spark 1.1 results are sourced from Scale AI leaderboard.” source
GPT-5.6 Terra OpenAI	Independent lab Independent lab · 2026-07-09	expert human and external tester red teaming Misuse and jailbreaks	“Experthumansandexternal testersusedadiversesetofapproachestofindgaps.” source
GPT-5.6 Terra OpenAI	Independent lab Independent lab · 2026-07-09	molecular biology capabilities test Chemical, biological, radiological, nuclear	“Itsstrongestreportedconfigurationsscored53.5%ontheVirology CapabilitiesTest,60.0%ontheMolecularBiologyCapabilitiesTest,68.4% ontheHumanPathogenCapabilitiesTest, and68.3%onWorld-ClassBio.” source
GPT-5.6 Terra OpenAI	Independent lab Independent lab · 2026-07-09	REPRO-BAIT Chemical, biological, radiological, nuclear	“On ReproBAIT, which measures AI agents on their ability to independently reproducebiologicalAImodels(BAIMs)frompublishedscientificpapers, therailfreevariantofGPT-5.6reached85%,comparedwith82%forGPT- 5.5.” source
GPT-5.6 Terra OpenAI	Independent lab Independent lab · 2026-07-09	manual agentic assessment Chemical, biological, radiological, nuclear	“In manual agentic assessment, the model was able to plan and execute a complex,long-runningscientifictask,includingmaintainingastructured workgraphandcoordinatingsubagents.” source
GPT-5.6 Terra OpenAI	Independent lab Independent lab · 2026-07-09	Frontier Cyber Cyber	“IrregularfoundthatGPT-5.6Solhason-parorslightlystrongeroffensive- cyber capabilities than GPT-5.5.” source

Previous · Page 26 of 49 · Next