AI Safety

What AI providers state their models reached in safety testing, in their own words, and what outside testers found.

Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.

What providers state

Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.

136 of 136

Model	Level, as the provider names it	The provider states	The provider's sentence
Muse Spark Meta	Moderate or lower Autonomy and AI research	Not reached	“We assess loss of control risk as “moderate or lower”.” source
Muse Spark Meta	High Chemical, biological, radiological, nuclear	Not determined	“Those evaluations identified elevated risks prior to mitigations, with Chemical and Biological capabilities assessed as likely reaching the “high risk” category under the Advanced AI Scaling Framework before safeguards were applied.” source
Muse Spark Meta	Moderate or lower Chemical, biological, radiological, nuclear	Not reached	“However, after applying mitigations, the residual risk of the model is that of “moderate or lower risk.”” source
Muse Spark Meta	Below the level required to meaningfully enable the cyber threat scenarios defin Cyber	Not reached	“Muse Spark’s offensive cyber capabilities remain below the level required to meaningfully enable the cyber threat scenarios defined in the Advanced AI Scaling Framework.” source
Muse Spark Meta	Moderate or lower Cyber	Not reached	“We assess cybersecurity risk as “moderate or lower” with respect to the threat scenarios outlined in Meta’s Advanced AI Scaling Framework.” source
Gemini 4 Argon Google DeepMind	TCL (Tracked Capability Level) Autonomy and AI research	Not determined	“In light of this, we assess that we cannot rule out the TCL.” source
Gemini 4 Argon Google DeepMind	TCL (Tracked Capability Level) Chemical, biological, radiological, nuclear	Not reached	“We can rule out the TCL for the chemical, radiological, and nuclear domains with reasonable confidence.” source
Gemini 4 Argon Google DeepMind	CCL (Critical Capability Level) Deception and sabotage	Not reached	“we assess its overall capabilities to be below the CCL threshold.” source
Gemini 4 Argon Google DeepMind	CCL (Critical Capability Level) Misuse and jailbreaks	Not reached	“Gemini 4 Argon demonstrates some ability to influence user beliefs and behaviors during one-on-one direct conversations in human behavioral studies, and may provide utility in the planning stage of a malicious influence operation but we assess its overall capabilities to be below the CCL threshold.” source
Gemini 4 Argon Google DeepMind	CCL (Critical Capability Level) Other	Not reached	“Gemini 4 Argon's performance remains below our CCL threshold.” source
Gemini 3.7 Flash Google DeepMind	CCL (Critical Capability Level) Autonomy and AI research	Not reached	“The model does not reach the CCL alert threshold.” source
Gemini 3.7 Flash Google DeepMind	CCL (Critical Capability Level) Chemical, biological, radiological, nuclear	Not reached	“We can rule out the CCL for the CBRN domain with reasonable conﬁdence based on the results from our testing.” source
Gemini 3.7 Flash Google DeepMind	TCL (Tracked Capability Level) Chemical, biological, radiological, nuclear	Not reached	“We can rule out the TCL for the CBRN domain with reasonable conﬁdence based on the results from our testing.” source
Gemini 3.7 Flash Google DeepMind	CCL (Critical Capability Level) Cyber	Not reached	“Gemini 3.7 Flash reaches the alert threshold for this CCL, but not the CCL.” source
Gemini 3.7 Flash Google DeepMind	TCL (Tracked Capability Level) Deception and sabotage	Not reached	“The model does not reach the TCL.” source
Gemini 3.5 Transcribe Live Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that none of Gemini 3.5 Translate, Gemini 3.5 Transcribe, or Gemini 3.5 Transcribe Live have meaningful new T/CCL-relevant capabilities or material increases in performance with respect to Frontier Safety compared to Gemini 3.1 Pro and Gemini 3.7 Flash, therefore we are conﬁdent that none of them are likely to reach any CCLs.” source
Gemini 3.5 Transcribe Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that none of Gemini 3.5 Translate, Gemini 3.5 Transcribe, or Gemini 3.5 Transcribe Live have meaningful new T/CCL-relevant capabilities or material increases in performance with respect to Frontier Safety compared to Gemini 3.1 Pro and Gemini 3.7 Flash, therefore we are conﬁdent that none of them are likely to reach any CCLs.” source
Gemini 3.5 Flash-Lite Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that, while Gemini 3.5 Flash-Lite excels at agents and coding, it does not have meaningful new capabilities or material increases in performance with respect to the domains outlined in our Frontier Safety Framework compared to Gemini 3.1 Pro; therefore, based on Gemini 3.1 Pro results, we are confident that Gemini 3.5 Flash-Lite is also unlikely to reach any CCLs.” source
Gemini 3.5 Audio Google DeepMind	Tracked or critical capability level (t/ccls) Other	Not reached	“they did not reach any Tracked or Critical Capability Levels (T/CCLs) outlined in our Frontier Safety Framework .” source
Gemini 3.1 Pro Google DeepMind	Uplift level 1 Chemical, biological, radiological, nuclear	Not reached	“We found that the model remains below alert thresholds for the CBRN, harmful manipulation, machine learning R&D, and misalignment CCLs.” source
Gemini 3.1 Pro Google DeepMind	Uplift level 1 Cyber	Not reached	“As previous models passed the alert threshold for cyber, we performed more additional testing in this domain on Gemini 3.1 Pro with and without Deep Think mode, and found that the model remains below the cyber CCL.” source
Gemini 3.1 Flash-Lite Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that Gemini 3.1 Flash-Lite is less capable than Gemini 3.1 Pro, therefore based on Gemini 3.1 Pro, we are confident that Gemini 3.1 Flash-Lite is also unlikely to reach any CCLs.” source
Gemini 3.1 Flash Audio Google DeepMind	CCL (Critical Capability Level) Other	Not determined	“Our assessments have shown that Gemini 3.1 Flash Audio is less capable than Gemini 3.1 Pro.” source
Gemini 2.5 Pro Google DeepMind	Machine Learning R&D Uplift Level 1 Autonomy and AI research	Not determined	“given Gemini 2.5 Pro Preview's poor median performance on RE-Bench relative to experts, the model likely lacks the necessary capabilities to automate or signiﬁcantly uplift any signiﬁcant fraction of the research process.” source
Gemini 2.5 Pro Google DeepMind	Instrumental Reasoning Level 1 Deception and sabotage	Not reached	“Instru- mental Reasoning Level 1 ✘ CCL not reached” source
Gemini 2.5 Pro Google DeepMind	Instrumental Reasoning Level 2 Deception and sabotage	Not reached	“Instru- mental Reasoning Level 2 ✘ CCL not reached” source
Gemini 2.5 Deep Think Google DeepMind	Uplift level 1 Chemical, biological, radiological, nuclear	Not determined	“Our assessment is that the model has enough technical knowledge in certain CBRN scenarios and stages to be considered at early warning.” source
Gemini 2.5 Deep Think Google DeepMind	Cyber Uplift Level 1 Cyber	Not determined	“CCL reached? No, but the early warning alert threshold was originally reached by Gemini 2.5 Pro and continues to be met by Gemini 2.5 Deep Think.” source
Gemini 2.5 Deep Think Google DeepMind	Uplift level 1 Cyber	Not determined	“For Cyber Uplift Level 1 , the early warning alert threshold was originally reached by Gemini 2.5 Pro and continues to be met by Gemini 2.5 Deep Think.” source
GPT-5.5 Instant OpenAI	Below high capability Autonomy and AI research	Not reached	“GPT-5.5 Instant is considered below High Capability.” source
GPT-5.5 Instant OpenAI	High Autonomy and AI research	Not reached	“For AI Self-Improvement, evaluations of final checkpoints indicate that, like its predecessor models, GPT-5.5 Instant does not have a plausible chance of reaching a High threshold.” source
GPT-5.5 Instant OpenAI	High Chemical, biological, radiological, nuclear	Reached	“GPT-5.5 Instant is our first Instant model to be treated as High Capability in the Biological and Chemical domain, as well as in the Cybersecurity domain.” source
GPT-5.5 Instant OpenAI	High Cyber	Reached	“this is the first Instant model that we are treating as High capability in our Cybersecurity and Biological & Chemical Preparedness categories, and implementing appropriate safeguards.” source
GPT-5 OpenAI	High Chemical, biological, radiological, nuclear	Reached	“Similarly to ChatGPT agent, we have decided to treat gpt-5-thinking as High capability in the Biological and Chemical domain under our Preparedness Framework, activating the associated safeguards.” source
GPT-5 OpenAI	High cyber risk Cyber	Not reached	“The gpt-5 model series does not meet the threshold for high cyber risk.” source
Claude Sonnet 4.5 Anthropic	AI R&D-4 Autonomy and AI research	Not reached	“All models tested remain below our pre-committed rule-out threshold of 0.6.” source
Claude Sonnet 4.5 Anthropic	ASL-3 Autonomy and AI research	Not determined	“Exhibited improved autonomy capabilities in software engineering and AI research tasks, though still below the ASL-4 rule-out threshold.” source
Claude Sonnet 4.5 Anthropic	ASL-3 Chemical, biological, radiological, nuclear	Not determined	“Remained well below ASL-4 thresholds across all domains of concern;” source
Claude Sonnet 4.5 Anthropic	ASL-4 rule-out Chemical, biological, radiological, nuclear	Not determined	“However, given that this type of logic can have a general frog-boiling effect—where gradual changes mask the crossing of important thresholds—we do not use this evaluation as indicative of a strong ASL-4 rule-out.” source
Claude Sonnet 4.5 Anthropic	ASL-3 Cyber	Not determined	“Showed meaningful improvements in cyber capabilities, particularly in vulnerability discovery and code analysis;” source

Previous · Page 3 of 4 · Next

What outside testers report

Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.

1957 of 1957

Model	Tester	Test	What was found
GPT-5.5 OpenAI	Apollo Research Independent lab · 2026-07-07	LLM judge severity scoring of injected-failure trajectories Other	“GPT-5.4 and GPT-5.5 score 7, weighing the unauthorized production execution most heavily, yet even they stay below the reference.” source
GPT-5.4 mini OpenAI	Apollo Research Independent lab · 2026-07-07	coding-agent trajectory severity scoring (1-10 severity scale across five failure modes) Other	“We evaluate 16 LLMs scoring coding-agent trajectories on a 1-10 severity scale across three datasets and five failure modes to assess their capabilities and calibration as coding agent monitors.” source
GPT-5.4 mini OpenAI	Apollo Research Independent lab · 2026-07-07	LLM judge evaluation against approximate ground truth severity labels Other	“For example, Gemini 3.1 Flash-Lite skips severity grading to score task execution, Haiku 4.5 primarily fails to flag violations at all, and GPT-5.4 Mini identifies violations but discounts them heavily through session-context framing.” source
Claude Sonnet 5 Anthropic	Apollo Research Independent lab · 2026-07-07	evaluating LLM calibration for coding-agent monitoring Other	“We evaluate 16 LLMs scoring coding-agent trajectories on a 1-10 severity scale across three datasets and five failure modes to assess their capabilities and calibration as coding agent monitors.” source
Claude Sonnet 5 Anthropic	Apollo Research Independent lab · 2026-07-07	LLM judge evaluation against approximate ground truth severity labels Other	“Claude models lead on ranking quality: Opus 4.8 is the best single judge, with Sonnet 5 close behind at 40% of the cost.” source
Claude Sonnet 5 Anthropic	Apollo Research Independent lab · 2026-07-07	repeated scoring stability analysis Other	“Resampling gains are also concentrated in smaller models (e.g. Gemini 2.5 Flash, GPT-5.4 Mini), while the biggest models (e.g. Opus 4.8, Sonnet 5, GPT-5.5) gain little from repeated calls.” source
Claude Sonnet 5 Anthropic	Apollo Research Independent lab · 2026-07-07	pairwise spearman rank correlation between judges Other	“Sonnet 5 and Opus 4.8, the strongest pair, agree with each other at 0.93, which is higher than any model's agreement with our approximate ground truth.” source
Claude Sonnet 4.6 Anthropic	Apollo Research Independent lab · 2026-07-07	coding-agent trajectory severity grading (1-10 severity scale across five failure modes) Other	“Some models seem rarely useful across many dimensions as single judges: Gemini 3.1 Flash-Lite, Gemini 2.5 Flash, Gemini 3.5 Flash, Gemini-2.5-pro, GPT-5.4 nano, GPT-5.4 mini, Claude Haiku 4.5, Sonnet-4.6, Opus-4.6 since they are strictly dominated by other models.” source
Claude Sonnet 4.6 Anthropic	Apollo Research Independent lab · 2026-07-07	LLM judge evaluation on injected-failure trajectories (Spearman rank correlation and MAE against reference severity) Other	“As single judges, we recommend against Gemini 3.1 Flash-Lite, Gemini 2.5 Flash, Gemini 3.5 Flash, Gemini 2.5 Pro, GPT-5.4 Mini, Claude Haiku 4.5, Sonnet 4.6, and Opus 4.6; each is dominated by a cheaper or better alternative.” source
Claude Sonnet 4.6 Anthropic	Apollo Research Independent lab · 2026-07-07	Paired native-thinking vs no-thinking judge scoring comparison Other	“Every judge scored the same trajectories under both settings: once with native thinking disabled (reasoning=none) and once at the high setting (reasoning=high for OpenAI, Anthropic, and Gemini 3.x models; a thinking budget of 8192 tokens for Gemini 2.5 models) used throughout this post.” source
Claude Opus 4.8 Anthropic	Apollo Research Independent lab · 2026-07-07	coding-agent trajectory severity grading (1-10 severity scale across five failure modes) Other	“We evaluate 16 LLMs scoring coding-agent trajectories on a 1-10 severity scale across three datasets and five failure modes to assess their capabilities and calibration as coding agent monitors.” source
Claude Opus 4.8 Anthropic	Apollo Research Independent lab · 2026-07-07	coding-agent trajectory severity scoring (1-10 severity scale across five failure modes) Other	“Selected by ρ: Opus 4.8 alone (0.90) → GPT-5.5 + Opus 4.8 (0.91, MAE 0.85) → + Gemini 3.1 Pro (0.92).” source
Claude Opus 4.7 Anthropic	Apollo Research Independent lab · 2026-07-07	severity scoring of coding-agent trajectories on a 1-10 severity scale across three datasets and five failure modes Other	“We evaluate 16 LLMs scoring coding-agent trajectories on a 1-10 severity scale across three datasets and five failure modes to assess their capabilities and calibration as coding agent monitors.” source
Claude Opus 4.6 Anthropic	Apollo Research Independent lab · 2026-07-07	severity scoring of coding-agent trajectories on a 1-10 scale Other	“We evaluate 16 LLMs scoring coding-agent trajectories on a 1-10 severity scale across three datasets and five failure modes to assess their capabilities and calibration as coding agent monitors.” source
Claude Haiku 4.5 Anthropic	Apollo Research Independent lab · 2026-07-07	severity scoring of coding-agent trajectories Other	“Some models seem rarely useful across many dimensions as single judges: Gemini 3.1 Flash-Lite, Gemini 2.5 Flash, Gemini 3.5 Flash, Gemini-2.5-pro, GPT-5.4 nano, GPT-5.4 mini, Claude Haiku 4.5, Sonnet-4.6, Opus-4.6 since they are strictly dominated by other models.” source
Claude Haiku 4.5 Anthropic	Apollo Research Independent lab · 2026-07-07	scoring coding-agent trajectories on a 1-10 severity scale across three datasets and five failure modes Other	“We evaluate 16 LLMs scoring coding-agent trajectories on a 1-10 severity scale across three datasets and five failure modes to assess their capabilities and calibration as coding agent monitors.” source
Claude Haiku 4.5 Anthropic	Apollo Research Independent lab · 2026-07-07	judge evaluation on injected-failure trajectories Deception and sabotage	“For example, Gemini 3.1 Flash-Lite skips severity grading to score task execution, Haiku 4.5 primarily fails to flag violations at all, and GPT-5.4 Mini identifies violations but discounts them heavily through session-context framing.” source
Claude Haiku 4.5 Anthropic	Apollo Research Independent lab · 2026-07-07	native thinking vs no-thinking judge comparison Deception and sabotage	“Every judge scored the same trajectories under both settings: once with native thinking disabled (reasoning=none) and once at the high setting (reasoning=high for OpenAI, Anthropic, and Gemini 3.x models; a thinking budget of 8192 tokens for Gemini 2.5 models) used throughout this post.” source
Claude Haiku 4.5 Anthropic	Apollo Research Independent lab · 2026-07-07	per-model calibration curves with thinking disabled Other	“The frontier models are nearly unchanged without thinking, e.g. Sonnet 5 (ρ = 0.89, MAE = 0.96) and Opus 4.8 (0.89, 1.00) remain strong, while the small models degrade visibly: Haiku 4.5 drops to ρ = 0.69, Gemini 3.1 Flash-Lite to 0.67, GPT-5.4 Mini to 0.72, with a more pronounced under-scoring bend below the diagonal.” source
GPT-5.5 OpenAI	Epoch AI Independent lab · 2026-07-01	EBR-bench Other	“AI systems are closer to the random baseline, with GPT-5.5 and Opus 4.8 both incurring 2.1 fatigue per round, enabling about 7.7 rounds of play per day.” source
GPT-5.5 OpenAI	Epoch AI Independent lab · 2026-07-01	max elicitation setting Other	“Namely, they achieve between 2 and 3.5 additional objectives (out of 21), depending on the model.” source
GPT-5 OpenAI	Epoch AI Independent lab · 2026-07-01	ebr-bench Other	“Our latest benchmark, EBR-bench, tests AI systems for this ability by having them play Earthborne Rangers, a complex board game, repeatedly.” source
Claude Opus 5 Anthropic	Gray Swan Independent lab · 2026-07-01	Live bug bounty across surfaces Prompt injection and agents	“As we ﬁrst did for the Claude Opus 4.8 System Card , we worked with Gray Swan to host a live bug bounty in which expert red-teamers competed for a pool of prizes awarded for successful prompt injection attacks against a set of frontier models, including Claude Opus 5.” source
Claude Opus 5 Anthropic	Gray Swan Independent lab · 2026-07-01	live bug bounty hosted with Gray Swan Prompt injection and agents	“Claude Opus 5 (0.08%) shows a small improvement over Claude Opus 4.8 (0.11%), and Claude Fable 5 is our most robust model in these scenarios (0.04%).” source
Claude Opus 5 Anthropic	UK AI Security Institute Government evaluator · 2026-07-01	open-ended misalignment testing Deception and sabotage	“UK AISI was given access to a pre-release checkpoint of Opus 5 to test for behaviors relevant to misalignment risk.” source
Claude Opus 4.8 Anthropic	Epoch AI Independent lab · 2026-07-01	EBR-bench Other	“Our latest benchmark, EBR-bench, tests AI systems for this ability by having them play Earthborne Rangers, a complex board game, repeatedly.” source
Claude Opus 4.1 Anthropic	Epoch AI Independent lab · 2026-07-01	ebr-bench Autonomy and AI research	“Our latest benchmark, EBR-bench, tests AI systems for this ability by having them play Earthborne Rangers, a complex board game, repeatedly.” source
Claude Opus 4.1 Anthropic	Epoch AI Independent lab · 2026-07-01	EBR-bench Other	“In our “max elicitation” setting, we give them detailed information about the game, including all the tactical and strategic advice that one experienced player could think of.” source
Claude Sonnet 5 Anthropic	Dyno Therapeutics Independent lab · 2026-06-30	black-box rna sequence modeling and design challenge Chemical, biological, radiological, nuclear	“We partnered with Dyno Therapeutics on two sequence-to-function evaluations: a black-box RNA sequence modeling and design challenge benchmarked against 57 human participants drawn from the leading edge of the US ML-bio labor market, and an AAV capsid packaging prediction task measuring whether model domain knowledge and machine learning capabilities can outperform pretrained protein language models.” source
Claude Sonnet 5 Anthropic	Gray Swan, UK AI Security Institute Independent lab · 2026-06-30	agent red teaming (art) benchmark Prompt injection and agents	“To be consistent with previous system cards, we report results from the Agent Red Teaming (ART) benchmark, developed by Gray Swan in collaboration with the UK AI Security Institute .” source
Claude Sonnet 5 Anthropic	Gray Swan Independent lab · 2026-06-30	gray swan indirect prompt injection (ipi) benchmark Prompt injection and agents	“For this release, we supplemented the ART prompt injection benchmark—which recent Claude models have nearly saturated—with a new Gray Swan indirect prompt injection (IPI) benchmark that spans coding, computer use, and tool use tasks.” source
Claude Sonnet 5 Anthropic	Gray Swan Independent lab · 2026-06-30	Gray Swan IPI benchmark (Q1 2026) Prompt injection and agents	“In this new benchmark, Claude Sonnet 5 achieves near-parity with Claude Opus 4.8.” source
Claude Sonnet 5 Anthropic	Independent lab Independent lab · 2026-06-30	live bug bounty Prompt injection and agents	“Sonnet 5 tied with Claude Opus 4.8 for the strongest result in the bug bounty, with only 0.19% of unique attacks succeeding against each, a signiﬁcant improvement over Claude Sonnet 4.6 (1.41%) and other frontier models like GPT-5.5 (3.08%) or Gemini 3.5 Flash (6.66%).” source
Claude Sonnet 5 Anthropic	Independent lab Independent lab · 2026-06-30	shade indirect prompt injection attacks in coding environments Prompt injection and agents	“Claude Sonnet 5 showed the strongest robustness to prompt injection in coding environments among all models evaluated, with an attack success rate of 0.31% over all attempts with extended thinking and 0.29% without thinking.” source
Claude Sonnet 5 Anthropic	Independent lab Independent lab · 2026-06-30	shade indirect prompt injection attacks in computer use environments Prompt injection and agents	“In computer use environments, Sonnet 5 also showed a large improvement over Claude Sonnet 4.6, reducing the attack success rate from 12.0% to 2.25% with extended thinking and from 14.4% to 6.04% without thinking.” source
Claude Sonnet 5 Anthropic	Independent lab Independent lab · 2026-06-30	frontiercode Other	“FrontierCode 20 is an agentic coding benchmark of 150 software engineering tasks created by Cognition.” source
Claude Sonnet 5 Anthropic	Independent lab Independent lab · 2026-06-30	cursorbench Other	“Claude Sonnet 5 scored 61.2% compared to Sonnet 4.6 at 49% and Opus 4.8 at 63.8%.” source
Claude Sonnet 5 Anthropic	Independent lab Independent lab · 2026-06-30	GDPval-AA v2 Other	“Claude Sonnet 5 ranks second (ELO 1618), statistically tied with Opus 4.8 (ELO 1615) and trailing only Fable 5 (ELO 1783).” source
Claude Sonnet 5 Anthropic	Independent lab Independent lab · 2026-06-30	AutomationBench Other	“On AutomationBench’s leaderboard, which measures performance on a private held-out evaluation set, Claude Sonnet 5 (max effort) scores 13.5%, a signiﬁcant gain over Claude Sonnet 4.6 (max effort) at 5.3%.” source
Claude Sonnet 5 Anthropic	Independent lab Independent lab · 2026-06-30	AA-Briefcase Other	“Claude Sonnet 5 ranks second (ELO 1393), statistically tied with Opus 4.8 (ELO 1352) and trailing only Fable 5 (ELO 1586).” source

Previous · Page 28 of 49 · Next