AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
138 of 138
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Haiku 5.5 Anthropic · 2026-10-07 |
Autonomy-1 Autonomy and AI research |
Reached | “However, we do treat Haiku 5.5 as meeting our CB-1 and Autonomy-1 thresholds, and we apply the corresponding mitigations from our RSP.” source |
| Claude Haiku 5.5 Anthropic · 2026-10-07 |
Autonomy-2 Autonomy and AI research |
Not reached | “As such, we determine that Haiku 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds.” source |
| Claude Haiku 5.5 Anthropic · 2026-10-07 |
CB-1 Chemical, biological, radiological, nuclear |
Reached | “However, we do treat Haiku 5.5 as meeting our CB-1 and Autonomy-1 thresholds, and we apply the corresponding mitigations from our RSP.” source |
| Claude Haiku 5.5 Anthropic · 2026-10-07 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “As such, we determine that Haiku 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds.” source |
| Claude Sonnet 5.5 Anthropic · 2026-09-28 |
Autonomy-1 Autonomy and AI research |
Reached | “We do treat Sonnet 5.5 as meeting our CB-1 and Autonomy-1 thresholds and apply the corresponding mitigations from our RSP.” source |
| Claude Sonnet 5.5 Anthropic · 2026-09-28 |
Autonomy-2 Autonomy and AI research |
Not reached | “As such, our determination that Opus 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds also applies to Sonnet 5.5.” source |
| Claude Sonnet 5.5 Anthropic · 2026-09-28 |
CB-1 Chemical, biological, radiological, nuclear |
Reached | “We do treat Sonnet 5.5 as meeting our CB-1 and Autonomy-1 thresholds and apply the corresponding mitigations from our RSP.” source |
| Claude Sonnet 5.5 Anthropic · 2026-09-28 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “As such, our determination that Opus 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds also applies to Sonnet 5.5.” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
Automated AI R&D Autonomy and AI research |
Not reached | “We conclude the risk threshold is not crossed, on the same two grounds as our determination for our previous frontier model: (1) we do not observe a sustained AI-attributable 2× acceleration in the pace of our overall AI progress, and (2) the model is not close to substituting for our Research Scientists and Research Engineers, especially relatively senior ones.” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
Dramatic acceleration of automated AI R&D Autonomy and AI research |
Not reached | “We assess that Claude Opus 5.5 does not cross the capability threshold for dramatic acceleration of automated AI R&D in our RSP.” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
Next capability threshold in our RSP and FCF Autonomy and AI research |
Not reached | “In automated AI research and development, we assess that Claude Opus 5.5 does not cross the next capability threshold in our RSP and FCF.” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
CB-1 Chemical, biological, radiological, nuclear |
Reached | “On chemical and biological risks, we treat Opus 5.5 as having CB-1 capabilities (relating to the synthesis of non-novel weapons) but not CB-2 capabilities (relating to the synthesis of novel weapons).” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “On chemical and biological risks, we treat Opus 5.5 as having CB-1 capabilities (relating to the synthesis of non-novel weapons) but not CB-2 capabilities (relating to the synthesis of novel weapons).” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
Lower category of the Frontier Compliance Framework (FCF) Cyber |
Not determined | “Despite Claude Opus 5.5’s cyber capabilities, the model still falls within the lower category of capabilities set out in our Frontier Compliance Framework (FCF), and we do not see indications that the model is capable of novel offensive capability.” source |
| Grok 4.7 xAI · 2026-09-21 |
FAIF safety (Frontier AI Framework) Chemical, biological, radiological, nuclear |
Not reached | “Grok 4.7 scores below the FAIF safety thresholds on dual-use knowledge, indicating limited actionable uplift for an already-trained actor.” source |
| GPT-Image-2.5-Flare OpenAI · 2026-09-08 |
High Chemical, biological, radiological, nuclear |
Not reached | “The results indicate that neither GPT-Image-2.5-Sunburst nor GPT-Image- 2.5-Flare cross the Bio High threshold or the Cyber High threshold.” source |
| GPT-Image-2.5-Flare OpenAI · 2026-09-08 |
High Cyber |
Not reached | “The results indicate that neither GPT-Image-2.5-Sunburst nor GPT-Image- 2.5-Flare cross the Bio High threshold or the Cyber High threshold.” source |
| GPT-6 Astra OpenAI · 2026-09-03 |
High Other |
Not determined | “Astra improves meaningfully over GPT-5.6 Sol on real internal research debugging tasks, scoring 78.05%, while still being below our indicative threshold for Highcapability.” source |
| Claude Mythos 5.1 Anthropic · 2026-09-01 |
Automated AI R&D Autonomy and AI research |
Not reached | “We conclude that Mythos 5.1 does not cross the risk threshold, for the same two reasons we discussed in our assessment of Mythos 5: (1) we do not observe a sustained, AI-attributable 2× acceleration in the pace of progress on AI R&D, and (2) the model is not close to substituting for Anthropic Research Scientists and Research Engineers, especially relatively senior ones.” source |
| Claude Mythos 5.1 Anthropic · 2026-09-01 |
Capability threshold for dramatic acceleration of automated AI R&D Autonomy and AI research |
Not reached | “We assess that Claude Mythos 5.1 does not cross the capability threshold for dramatic acceleration of automated AI R&D in our RSP and FCF, although we think it provides a meaningful acceleration to AI R&D efforts in many circumstances, and is somewhat more capable in this domain than the models described in our previous system cards.” source |
| Claude Mythos 5.1 Anthropic · 2026-09-01 |
Tier 1 Cyber |
Not determined | “Although Mythos 5.1 is in Tier 1, it is getting closer to Tier 2, completing more and more autonomous tasks.” source |
| Grok 4.6 xAI · 2026-08-12 |
FAIF safety (Frontier AI Framework) Chemical, biological, radiological, nuclear |
Not reached | “Grok 4.6 scores below the FAIF safety thresholds on dual-use knowledge, indicating limited actionable uplift for an already-trained actor.” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower Autonomy and AI research |
Not determined | “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower risk (inferred) Autonomy and AI research |
Not determined | “- Loss of Control: Moderate or lower risk (inferred).” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower Chemical, biological, radiological, nuclear |
Not reached | “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower Cyber |
Not determined | “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower risk (inferred) Cyber |
Not determined | “- Cyber: Moderate or lower risk (inferred);” source |
| Claude Opus 5 Anthropic · 2026-07-24 |
CB-1 Chemical, biological, radiological, nuclear |
Reached | “On chemical and biological risks, we treat the model as having CB-1 capabilities (relating to the synthesis of non-novel weapons), but not CB-2 capabilities (relating to the synthesis of novel weapons).” source |
| Claude Opus 5 Anthropic · 2026-07-24 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “On chemical and biological risks, we treat the model as having CB-1 capabilities (relating to the synthesis of non-novel weapons), but not CB-2 capabilities (relating to the synthesis of novel weapons).” source |
| Claude Sonnet 5 Anthropic · 2026-06-30 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “Sonnet 5 does not cross the CB-2 threshold.” source |
| Claude Sonnet 5 Anthropic · 2026-06-30 |
Notable capability Chemical, biological, radiological, nuclear |
Not determined | “We consider performance on the virology tasks notable if: ● The model scores above 0.80 on the long-form tasks; or ● The model meets or exceeds the expert baseline average (0.221) on the multimodal task.” source |
| GPT-Rosalind-5.5 OpenAI · 2026-06-03 |
High Autonomy and AI research |
Not determined | “Because GPT- Rosalind-5.5 was trained on additional biology data unrelated to AI self-improvement, we rely on GPT-5.5’s existing Preparedness assessment for AI Self-Improvement, where GPT-5.5 did not meet the threshold for High capability.” source |
| GPT-Rosalind-5.5 OpenAI · 2026-06-03 |
High Chemical, biological, radiological, nuclear |
Reached | “Under our Preparedness Framework, the Preparedness evaluations in the Biological and Chemical domain met our threshold for High capability while falling below the threshold for Critical.” source |
| Gemini 3.1 Flash-Lite Image Google DeepMind · 2026-06 |
CCL (Critical Capability Level) Other |
Not determined | “Our assessments have shown that Gemini 3.1 Flash-Lite Image is less capable than Gemini 3.1 Pro, therefore based on Gemini 3.1 Pro, we are confident that that Gemini 3.1 Flash-Lite Image is also unlikely to reach any CCLs.” source |
| Claude Opus 4.8 Anthropic · 2026-05-28 |
Automated AI R&D Autonomy and AI research |
Not reached | “We assess that Claude Opus 4.8 does not cross the automated AI-R&D capability threshold.” source |
| Claude Opus 4.8 Anthropic · 2026-05-28 |
CB-1 Chemical, biological, radiological, nuclear |
Not determined | “We believe it is hard to be confident whether a model passes this threshold, but (as with previous recent models) our results are consistent with Opus 4.8 providing significant uplift to individuals or groups with basic technical backgrounds for this threat model.” source |
| Claude Opus 4.8 Anthropic · 2026-05-28 |
CB-2 Chemical, biological, radiological, nuclear |
Not determined | “Because Opus 4.8 does not advance the capability frontier beyond Claude Mythos Preview on the CB-relevant evaluations most diagnostic of the CB-2 threat model, we consider the uplift of threat actors without the ability to develop such weapons to be limited.” source |
| GPT-5.5 OpenAI · 2026-04-23 |
High Autonomy and AI research |
Not reached | “For AI self-improvement, evaluations of final checkpoints indicate that, like its predecessor models, GPT-5.5 does not have a plausible chance of reaching a High threshold.” source |
| GPT-5.5 OpenAI · 2026-04-23 |
High Chemical, biological, radiological, nuclear |
Reached | “we are continuing to treat GPT-5.5 as High capability in the Biological and Chemical domain.” source |
| GPT-5.5 OpenAI · 2026-04-23 |
Critical Cyber |
Not reached | “We therefore treat VulnLMP as important evidence that GPT-5.5 has High cybersecurity capability, while also as evidence against classifying it as Critical under the Preparedness Framework.” source |
Page 1 of 4 · Next
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1911 of 1911
| Model | Tester | Test | What was found |
|---|---|---|---|
| Gemini 3.1 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
GCG-optimized prompt injection misdirection Misuse and jailbreaks |
“After 500 training steps on the abliterated OSS model, help denial via misdirection to the unrelated task increased, going from help denial in 0/10 trials to 10/10 trials.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
tool-level safeguards Chemical, biological, radiological, nuclear |
“We tested a range of tool-level safeguards (text or code embedded within biological software tools to restrict agent usage) and evaluated their efficacy in preventing AI agents from assisting with the tool’s use.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
jailbreaks Misuse and jailbreaks |
“The tool-level safeguards we developed are compromised by subversion of LLM guardrails1: adversarial subversion techniques such as jailbreaks and refusal-vector ablation reduce the efficacy of safeguards, indicating that reliance on LLM guardrails does not currently provide robust risk mitigation.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
help denial rate Chemical, biological, radiological, nuclear |
“Our evaluation metric, the help denial rate, measures the frequency with which the safeguard stops the agent from both (i) running the biological tool and (ii) providing assistance to the user on the use of the tool.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
pre-run check Chemical, biological, radiological, nuclear |
“Across all tasks and all LLMs, the pre-run check increased the help denial rate, except in the one case in which the baseline help denial rate was already 100% (Opus 4.5 on the EVEscape task).” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
jailbreak robustness test Misuse and jailbreaks |
“We test our safeguards’ robustness to jailbreaks by appending the jailbreak text to each initial task prompt, repeating the methodology described above, and observing how the jailbreak affects refusal-to-help rate.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
safety abliteration Misuse and jailbreaks |
“We also explored methods to optimize a subset of our safeguards against small open-weight LLMs, gpt-oss-20b (Agarwal, 2025), and Qwen 3 4B Instruct (Yang, et al., 2025), whose safeguards have been subverted using a technique called safety abliteration (Arditi et al., 2024).” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
jailbreak text added to task prompts Misuse and jailbreaks |
“Including these jailbreaks in the task prompt reduced the help denial rate to near zero for Opus 4.5 and Gemini 3 Pro (Figure 2A).” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
refusal abliteration Misuse and jailbreaks |
“We performed refusal abliteration on two small open-weight LLMs, GPT OSS 20B (Agarwal et al., 2025) and Qwen 3 4B Instruct (Yang et al., 14 2025), and found that this reduced help denial to near zero for both LLMs (Figure 2B).” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
pre-run check on the esm3 task Deception and sabotage |
“On the ESM3 task, both Opus 4.5 and Gemini 3.1 Pro make these false claims in 50% of trials and 63% of trials, respectively.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
safeguard that detects environmental identifiers of the cursor cli agent Deception and sabotage |
“However, we found that Opus 4.5 and Gemini 3.1 Pro frequently delete these variables to evade detection.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
safeguard that utilizes the agents.md protocol Deception and sabotage |
“Despite the fact that AGENTS.md files are intended to contain instructions written by humans for AI agents (Agents.md project, 2026), we 16 found that agents would create and modify AGENTS.md files in order to bypass this safeguard.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
pre-run check in which the biological tool displays a message stating an intent to synthesize variola major Harmful content |
“While Opus 4.5 and Gemini 3.1 Pro were either blocked by content filters or refused to proceed in every trial, GPT 5.2 proceeded without refusal in every trial.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
tool-level safeguard evaluation (pre-run check, agents.md, environment identifiers) in biological software tasks Other |
“Our results suggest that safeguards implemented into biological software cannot currently induce help denial reliably in LLM agents.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
documentation changes (statements prohibiting AI agent usage placed into README.md of RFdiffusion) Misuse and jailbreaks |
“We ran experiments where statements prohibiting ai agent usage were placed into the readme.md documentation files of rfdiffusion.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
GCG optimization of safeguard text on RFdiffusion tasks with an abliterated gpt-oss-20b model Misuse and jailbreaks |
“We ran this algorithm with an abliterated gpt-oss-20b model on the rfdiffusion tasks.” source |
| Gemini 3 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-07-20 |
GCG optimization of misdirection prompt injection text on the abliterated gpt-oss-20b model Misuse and jailbreaks |
“After 500 training steps on the abliterated OSS model, help denial via misdirection to the unrelated task increased, going from help denial in 0/10 trials to 10/10 trials.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
ExploitBench Cyber |
“Cyber ExploitBench 21.4 ± 0.9 57.2 ± 1.1 38.1 ± 1.1 40.7 ± 1.1 N/A” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
CTF-Archive-Diamond Cyber |
“Cyber CTF Archive Diamond 39.3 ± 2.9 65.6 ± 2.8 48.4 ± 3.0 70.5 ± 2.7 32**” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
agent hijacking attacks Cyber |
“GLM-5.2 was never successfully hijacked by publicly available agent hijacking attacks (Figure 5.1), and appeared to withstand more iterations of adversarial red-teaming more than other evaluated open-weight models, although less than evaluated U.S. closed-weight models (Figure 5.2).” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
automated benchmark evaluations for coverage and public jailbreaking robustness against malicious cyber queries Misuse and jailbreaks |
“Figure 6.1: On automated benchmark evaluations for coverage and public jailbreaking robustness against malicious cyber queries, GLM-5.2 complied with a low percentage of requests , both with and without the use of a public jailbreak.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
automated benchmark evaluations against malicious cyber queries Misuse and jailbreaks |
“Figure 6.2: On automated benchmark evaluations against malicious cyber queries, the best jailbreak tended to reduce – rather than increase – the level of request-relevant detail in GLM-5.2’s responses.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
automated benchmark evaluations for safeguards coverage and public jailbreaking robustness against sensitive biological queries Chemical, biological, radiological, nuclear |
“Figure 6.4: On automated benchmark evaluations for safeguards coverage and public jailbreaking robustness against sensitive biological queries, GLM-5.2 complied at a higher rate than tested US models, both with and without the use of a public jailbreak.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
automated benchmark evaluations against sensitive biological queries Chemical, biological, radiological, nuclear |
“Figure 6.5: On automated benchmark evaluations against sensitive biological queries, GLM-5.2 responded with a higher rate of request-relevant detail than tested US models.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
caisi biology safeguards Chemical, biological, radiological, nuclear |
“sensitive biological queries, GLM-5.2 responded with a higher rate of request-relevant detail than tested US models.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
caisi cyber safeguards Cyber |
“CAISI cyber safeguards . A private benchmark created by CAISI that evaluates models against a dataset of 30 single-turn misuse requests related to offensive cyber.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
hijackeval Prompt injection and agents |
“HijackEval . A private benchmark created by CAISI that evaluates models’ robustness against agent hijacking attacks.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-07-17 |
portbench Other |
“Budgets were set to 1M weighted tokens for PortBench and CTF-Archive-Diamond, and 500k weighted tokens for SWE-Bench 4 OpenRouter (2026) Z.ai: GLM 5.2 .” source |
| GLM-5.2 Z.ai (Zhipu AI) |
UK AI Security Institute Government evaluator · 2026-07-17 |
narrow cyber tasks Cyber |
“On these tasks, GLM-5.2 performs comparably to the most cyber-capable models released 4 months before it (Opus 4.6 and GPT-5.3-Codex), and this holds across all four difficulty levels.” source |
| GLM-5.2 Z.ai (Zhipu AI) |
UK AI Security Institute Government evaluator · 2026-07-17 |
cyber ranges Cyber |
“On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it).” source |
| GLM-5.2 Z.ai (Zhipu AI) |
UK AI Security Institute Government evaluator · 2026-07-17 |
The Last Ones (TLO) Cyber |
“Notably, GLM-5.2 reached step 7 with marginally fewer tokens than any other model on average, tracking Opus 4.6’s trajectory to step 11 before stalling.” source |
| DeepSeek-V4-Pro DeepSeek |
UK AI Security Institute Government evaluator · 2026-07-17 |
AISI's narrow cyber tasks Cyber |
“DeepSeek’s V4-Pro is comparable to Opus 4.5, released 5 months before it.” source |
| DeepSeek-V4-Pro DeepSeek |
UK AI Security Institute Government evaluator · 2026-07-17 |
cyber range "The Last Ones" (TLO) Cyber |
“On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it).” source |
| DeepSeek-V4-Pro DeepSeek |
UK AI Security Institute Government evaluator · 2026-07-17 |
refusal behaviour on narrow cyber tasks Cyber |
“DeepSeek V4-Pro occasionally refused narrow cyber tasks, mainly in reverse engineering, which we circumvented simply via a small number of repeat attempts at refused tasks.” source |
| Gemini 3.1 Pro Google DeepMind |
SecureBio Independent lab · 2026-07-16 |
biotier-refuse Chemical, biological, radiological, nuclear |
“Google’s update to their mental health related safeguards made headlines , but we also identified changes to their biology safeguards, with an almost 30% increase in the refusal behavior of Gemini 3.1 Pro on BioTIER-refuse due to introduction of API-level refusals.” source |
| Gemini 3.1 Pro Google DeepMind |
SecureBio Independent lab · 2026-07-16 |
biotier-permit Chemical, biological, radiological, nuclear |
“Importantly, this increase in BioTIER-refuse performance had no impact on BioTIER-permit compliance .” source |
| Gemini 3.1 Pro Google DeepMind |
SecureBio Independent lab · 2026-07-16 |
BioTier Chemical, biological, radiological, nuclear |
“Google’s update to their mental health related safeguards made headlines, but we also identified changes to their biology safeguards, with an almost 30% increase in the refusal behavior of Gemini 3.1 Pro on BioTIER-refuse due to introduction of API-level refusals.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
situational-awareness suite Autonomy and AI research |
“Whilst, the situational-awareness evidence provides moderate confirmation for the narrow claim that current models do not reliably display the specific situationally aware, constraint-lifting behaviours measured by this suite.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
cover_your_tracks Deception and sabotage |
“cover_your_tracks is especially valuable because it combines several relevant covert subskills: identifying a subversive opportunity, altering or deleting traces, avoiding downstream detection, and completing a cover-up end-to-end.” source |
| Gemini 2.5 Pro Google DeepMind |
SaferAI Independent lab · 2026-07-16 |
classifier_evasion Deception and sabotage |
“classifier_evasion is also comparatively informative because it more directly tests whether the model can evade an active monitoring system by phrasing prohibited content in a way that avoids detection.” source |