AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
138 of 138
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Haiku 5.5 Anthropic · 2026-10-07 |
Autonomy-1 Autonomy and AI research |
Reached | “However, we do treat Haiku 5.5 as meeting our CB-1 and Autonomy-1 thresholds, and we apply the corresponding mitigations from our RSP.” source |
| Claude Haiku 5.5 Anthropic · 2026-10-07 |
Autonomy-2 Autonomy and AI research |
Not reached | “As such, we determine that Haiku 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds.” source |
| Claude Haiku 5.5 Anthropic · 2026-10-07 |
CB-1 Chemical, biological, radiological, nuclear |
Reached | “However, we do treat Haiku 5.5 as meeting our CB-1 and Autonomy-1 thresholds, and we apply the corresponding mitigations from our RSP.” source |
| Claude Haiku 5.5 Anthropic · 2026-10-07 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “As such, we determine that Haiku 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds.” source |
| Claude Sonnet 5.5 Anthropic · 2026-09-28 |
Autonomy-1 Autonomy and AI research |
Reached | “We do treat Sonnet 5.5 as meeting our CB-1 and Autonomy-1 thresholds and apply the corresponding mitigations from our RSP.” source |
| Claude Sonnet 5.5 Anthropic · 2026-09-28 |
Autonomy-2 Autonomy and AI research |
Not reached | “As such, our determination that Opus 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds also applies to Sonnet 5.5.” source |
| Claude Sonnet 5.5 Anthropic · 2026-09-28 |
CB-1 Chemical, biological, radiological, nuclear |
Reached | “We do treat Sonnet 5.5 as meeting our CB-1 and Autonomy-1 thresholds and apply the corresponding mitigations from our RSP.” source |
| Claude Sonnet 5.5 Anthropic · 2026-09-28 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “As such, our determination that Opus 5.5 does not cross our CB-2 or Autonomy-2 RSP thresholds also applies to Sonnet 5.5.” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
Automated AI R&D Autonomy and AI research |
Not reached | “We conclude the risk threshold is not crossed, on the same two grounds as our determination for our previous frontier model: (1) we do not observe a sustained AI-attributable 2× acceleration in the pace of our overall AI progress, and (2) the model is not close to substituting for our Research Scientists and Research Engineers, especially relatively senior ones.” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
Dramatic acceleration of automated AI R&D Autonomy and AI research |
Not reached | “We assess that Claude Opus 5.5 does not cross the capability threshold for dramatic acceleration of automated AI R&D in our RSP.” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
Next capability threshold in our RSP and FCF Autonomy and AI research |
Not reached | “In automated AI research and development, we assess that Claude Opus 5.5 does not cross the next capability threshold in our RSP and FCF.” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
CB-1 Chemical, biological, radiological, nuclear |
Reached | “On chemical and biological risks, we treat Opus 5.5 as having CB-1 capabilities (relating to the synthesis of non-novel weapons) but not CB-2 capabilities (relating to the synthesis of novel weapons).” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “On chemical and biological risks, we treat Opus 5.5 as having CB-1 capabilities (relating to the synthesis of non-novel weapons) but not CB-2 capabilities (relating to the synthesis of novel weapons).” source |
| Claude Opus 5.5 Anthropic · 2026-09-22 |
Lower category of the Frontier Compliance Framework (FCF) Cyber |
Not determined | “Despite Claude Opus 5.5’s cyber capabilities, the model still falls within the lower category of capabilities set out in our Frontier Compliance Framework (FCF), and we do not see indications that the model is capable of novel offensive capability.” source |
| Grok 4.7 xAI · 2026-09-21 |
FAIF safety (Frontier AI Framework) Chemical, biological, radiological, nuclear |
Not reached | “Grok 4.7 scores below the FAIF safety thresholds on dual-use knowledge, indicating limited actionable uplift for an already-trained actor.” source |
| GPT-Image-2.5-Flare OpenAI · 2026-09-08 |
High Chemical, biological, radiological, nuclear |
Not reached | “The results indicate that neither GPT-Image-2.5-Sunburst nor GPT-Image- 2.5-Flare cross the Bio High threshold or the Cyber High threshold.” source |
| GPT-Image-2.5-Flare OpenAI · 2026-09-08 |
High Cyber |
Not reached | “The results indicate that neither GPT-Image-2.5-Sunburst nor GPT-Image- 2.5-Flare cross the Bio High threshold or the Cyber High threshold.” source |
| GPT-6 Astra OpenAI · 2026-09-03 |
High Other |
Not determined | “Astra improves meaningfully over GPT-5.6 Sol on real internal research debugging tasks, scoring 78.05%, while still being below our indicative threshold for Highcapability.” source |
| Claude Mythos 5.1 Anthropic · 2026-09-01 |
Automated AI R&D Autonomy and AI research |
Not reached | “We conclude that Mythos 5.1 does not cross the risk threshold, for the same two reasons we discussed in our assessment of Mythos 5: (1) we do not observe a sustained, AI-attributable 2× acceleration in the pace of progress on AI R&D, and (2) the model is not close to substituting for Anthropic Research Scientists and Research Engineers, especially relatively senior ones.” source |
| Claude Mythos 5.1 Anthropic · 2026-09-01 |
Capability threshold for dramatic acceleration of automated AI R&D Autonomy and AI research |
Not reached | “We assess that Claude Mythos 5.1 does not cross the capability threshold for dramatic acceleration of automated AI R&D in our RSP and FCF, although we think it provides a meaningful acceleration to AI R&D efforts in many circumstances, and is somewhat more capable in this domain than the models described in our previous system cards.” source |
| Claude Mythos 5.1 Anthropic · 2026-09-01 |
Tier 1 Cyber |
Not determined | “Although Mythos 5.1 is in Tier 1, it is getting closer to Tier 2, completing more and more autonomous tasks.” source |
| Grok 4.6 xAI · 2026-08-12 |
FAIF safety (Frontier AI Framework) Chemical, biological, radiological, nuclear |
Not reached | “Grok 4.6 scores below the FAIF safety thresholds on dual-use knowledge, indicating limited actionable uplift for an already-trained actor.” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower Autonomy and AI research |
Not determined | “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower risk (inferred) Autonomy and AI research |
Not determined | “- Loss of Control: Moderate or lower risk (inferred).” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower Chemical, biological, radiological, nuclear |
Not reached | “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower Cyber |
Not determined | “Chem/Bio: Moderate or lower risk; Cyber: Moderate or lower risk (inferred); Loss of Control: Moderate or lower risk (inferred).” source |
| Muse Glimmer 30B Meta · 2026-08 |
Moderate or lower risk (inferred) Cyber |
Not determined | “- Cyber: Moderate or lower risk (inferred);” source |
| Claude Opus 5 Anthropic · 2026-07-24 |
CB-1 Chemical, biological, radiological, nuclear |
Reached | “On chemical and biological risks, we treat the model as having CB-1 capabilities (relating to the synthesis of non-novel weapons), but not CB-2 capabilities (relating to the synthesis of novel weapons).” source |
| Claude Opus 5 Anthropic · 2026-07-24 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “On chemical and biological risks, we treat the model as having CB-1 capabilities (relating to the synthesis of non-novel weapons), but not CB-2 capabilities (relating to the synthesis of novel weapons).” source |
| Claude Sonnet 5 Anthropic · 2026-06-30 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “Sonnet 5 does not cross the CB-2 threshold.” source |
| Claude Sonnet 5 Anthropic · 2026-06-30 |
Notable capability Chemical, biological, radiological, nuclear |
Not determined | “We consider performance on the virology tasks notable if: ● The model scores above 0.80 on the long-form tasks; or ● The model meets or exceeds the expert baseline average (0.221) on the multimodal task.” source |
| GPT-Rosalind-5.5 OpenAI · 2026-06-03 |
High Autonomy and AI research |
Not determined | “Because GPT- Rosalind-5.5 was trained on additional biology data unrelated to AI self-improvement, we rely on GPT-5.5’s existing Preparedness assessment for AI Self-Improvement, where GPT-5.5 did not meet the threshold for High capability.” source |
| GPT-Rosalind-5.5 OpenAI · 2026-06-03 |
High Chemical, biological, radiological, nuclear |
Reached | “Under our Preparedness Framework, the Preparedness evaluations in the Biological and Chemical domain met our threshold for High capability while falling below the threshold for Critical.” source |
| Gemini 3.1 Flash-Lite Image Google DeepMind · 2026-06 |
CCL (Critical Capability Level) Other |
Not determined | “Our assessments have shown that Gemini 3.1 Flash-Lite Image is less capable than Gemini 3.1 Pro, therefore based on Gemini 3.1 Pro, we are confident that that Gemini 3.1 Flash-Lite Image is also unlikely to reach any CCLs.” source |
| Claude Opus 4.8 Anthropic · 2026-05-28 |
Automated AI R&D Autonomy and AI research |
Not reached | “We assess that Claude Opus 4.8 does not cross the automated AI-R&D capability threshold.” source |
| Claude Opus 4.8 Anthropic · 2026-05-28 |
CB-1 Chemical, biological, radiological, nuclear |
Not determined | “We believe it is hard to be confident whether a model passes this threshold, but (as with previous recent models) our results are consistent with Opus 4.8 providing significant uplift to individuals or groups with basic technical backgrounds for this threat model.” source |
| Claude Opus 4.8 Anthropic · 2026-05-28 |
CB-2 Chemical, biological, radiological, nuclear |
Not determined | “Because Opus 4.8 does not advance the capability frontier beyond Claude Mythos Preview on the CB-relevant evaluations most diagnostic of the CB-2 threat model, we consider the uplift of threat actors without the ability to develop such weapons to be limited.” source |
| GPT-5.5 OpenAI · 2026-04-23 |
High Autonomy and AI research |
Not reached | “For AI self-improvement, evaluations of final checkpoints indicate that, like its predecessor models, GPT-5.5 does not have a plausible chance of reaching a High threshold.” source |
| GPT-5.5 OpenAI · 2026-04-23 |
High Chemical, biological, radiological, nuclear |
Reached | “we are continuing to treat GPT-5.5 as High capability in the Biological and Chemical domain.” source |
| GPT-5.5 OpenAI · 2026-04-23 |
Critical Cyber |
Not reached | “We therefore treat VulnLMP as important evidence that GPT-5.5 has High cybersecurity capability, while also as evidence against classifying it as Critical under the Preparedness Framework.” source |
Page 1 of 4 · Next
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1911 of 1911
| Model | Tester | Test | What was found |
|---|---|---|---|
| Claude Opus 5.5 Anthropic |
METR Independent lab · 2026-09-22 |
METR external testing Autonomy and AI research |
“External testing by METR ( Section 2.3.6 ) produced findings consistent with this assessment.” source |
| Claude Opus 5.5 Anthropic |
Dyno Therapeutics Independent lab · 2026-09-22 |
Black-box RNA sequence design Chemical, biological, radiological, nuclear |
“We partnered with Dyno Therapeutics on three evaluations of sequence-to-function modeling and design capability: 1. Black-box RNA sequence design: A medium-horizon challenge on which Dyno has evaluated 57 human participants drawn from the leading edge of the US ML-bio labor market since 2018.” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Budget NanoGPT Speedrun Autonomy and AI research |
“We conducted a preliminary evaluation of [Claude Opus 5.5] informed by: ● Capability testing, conducted via API access granted over a period of 10 business days.” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Language Model Conceptual Argumentation (LMCA) Autonomy and AI research |
“○ Language Model Conceptual Argumentation (LMCA) , a conceptual reasoning dataset described in A dataset of rated conceptual arguments (Cooper et al., 2026).” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Train a Program Autonomy and AI research |
“○ Train a Program , a task measuring an agent’s ability to train a machine learning model that replicates the behavior of a piece of software;” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Gaming Bot Autonomy and AI research |
“○ Gaming Bot , a task where an agent has to write a Python program capable of playing a video game, given a non-visual API to control the game.” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Sunlight Autonomy and AI research |
“○ Sunlight , a task measuring an agent’s ability to conduct open-ended research and write a corresponding report.” source |
| Claude Opus 5.5 Anthropic |
Trajectory Labs Independent lab · 2026-09-22 |
external red-teaming by Trajectory Labs, PBC Cyber |
“Trajectory Labs, PBC spent roughly 95 hours red teaming Claude Opus 5.5, sending over 29,000 requests against sandboxed tasks that each require reproducing an exploit for a publicly documented vulnerability.” source |
| Claude Opus 5.5 Anthropic |
10a Labs Independent lab · 2026-09-22 |
multi-turn attack conversations framed as developer-focused coding tasks Cyber |
“10a Labs spent roughly 56 hours red teaming Claude Opus 5.5 across 82 multi-turn attack conversations, each framed as a large developer-focused coding task designed to draw a working exploit out of a volume of legitimately requested application code rather than asking for the exploit directly.” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Shade automated attacker on critical infrastructure environments Cyber |
“Claude Opus 5.5’s safeguards refused over 90% of attempts outright, and none of the rest reached the objective.” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Shade automated attacker on exploit reproduction and ransomware staging scenarios Cyber |
“Of roughly 1,700 attempts, Claude Opus 5.5’s safeguards refused about a quarter outright.” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Indirect Prompt Injection (IPI) benchmark Prompt injection and agents |
“Claude Opus 5.5 matches Claude Fable 5.1, our most robust model to date on this benchmark, with an attack success rate of 0.1% at k=1, 0.7% at k=10, and 1.0% at k=15.” source |
| Claude Opus 5.5 Anthropic |
Independent lab Independent lab · 2026-09-22 |
Shade Prompt injection and agents |
“Claude Opus 5.5 substantially improves on Claude Opus 5 in coding environments against this stronger attacker, with an attack success rate of 54.61% over all attempts, compared to 88.92% for Claude Opus 5 with thinking enabled and 85.49% without thinking enabled.” source |
| Claude Opus 5.5 Anthropic |
METR Independent lab · 2026-09-22 |
Budget NanoGPT Speedrun Other |
“Budget NanoGPT Speedrun, a constrained version of the popular NanoGPT Speedrun competition for AI R&D.” source |
| Claude Opus 5.5 Anthropic |
METR Independent lab · 2026-09-22 |
Language Model Conceptual Argumentation (LMCA) Other |
“Language Model Conceptual Argumentation (LMCA), a conceptual reasoning dataset described in A dataset of rated conceptual arguments (Cooper et al., 2026).” source |
| Claude Opus 5.5 Anthropic |
METR Independent lab · 2026-09-22 |
Train a Program Other |
“Train a Program, a task measuring an agent’s ability to train a machine learning model that replicates the behavior of a piece of software;” source |
| Claude Opus 5.5 Anthropic |
METR Independent lab · 2026-09-22 |
Gaming Bot Other |
“Gaming Bot, a task where an agent has to write a Python program capable of playing a video game, given a non-visual API to control the game.” source |
| Claude Opus 5.5 Anthropic |
METR Independent lab · 2026-09-22 |
Sunlight Other |
“Sunlight, a task measuring an agent’s ability to conduct open-ended research and write a corresponding report.” source |
| Claude Opus 5.5 Anthropic |
METR Independent lab · 2026-09-22 |
Budget NanoGPT Speedrun Other |
“Claude Opus 5.5 improves upon Fable 5.1 across both verifiable tasks (Budget NanoGPT, Gaming Bot) and harder-to-verify tasks (LMCA, Sunlight), but:” source |
| Grok 4.7 xAI |
Independent lab Independent lab · 2026-09-21 |
Cathedralbench Cyber |
“CathedralBench is an independent, third party evaluation of cyber performance.” source |
| Grok 4.7 Cursor |
Independent lab Independent lab · 2026-09-21 |
cathedralbench Cyber |
“CathedralBench is an independent, third party evaluation of cyber performance.” source |
| Gemini 3 Flash Google DeepMind |
Academic Academic · 2026-09-21 |
warm introduction task on relationship-reasoning benchmark Other |
“With Gemini-3-Flash on a stratified subset of n = 150 queries per task, ReAdapt improves warm-introduction accuracy from 37% to 51% (+14 points) and reaction-selection accuracy from 69% to 77% (+8 points).” source |
| Claude Opus 5 Anthropic |
Academic Academic · 2026-09-21 |
OSWorld-Pro Other |
“Our findings reveal that OSWorld-Pro is challenging even for state-of-the-art LLMs, with top performers like Claude Opus 5 achieving only 75.7% vs. 83.4% on OSWorld.” source |
| Claude Opus 5 Anthropic |
Academic Academic · 2026-09-21 |
governed delivery plane end-to-end run across ten stages scored by an oracle Other |
“We ran four frontier agents (Claude Opus 5, GPT-5.6-luna, Gemini 3.7 Flash, DeepSeek V4-Pro) end to end on metered L40S, A100, and H200 GPUs across 8B to 70B open bases, certifying every scenario before scoring.” source |
| Claude Opus 4.8 Anthropic |
Academic Academic · 2026-09-21 |
suite of 325K experiments on 13 agents across three types of economic decisions (flights, health insurance, and graduate programs) Bias |
“In a suite of 325K experiments on 13 agents across three types of economic decisions (flights, health insurance, and graduate programs), we find that 8 models systematically choose more expensive options for wealthier users when requests are identical.” source |
| GPT-5.4 mini OpenAI |
Academic Academic · 2026-09-19 |
CyberGym Cyber |
“With fewer than 200 training examples, SelfOp yields a 17-point self-improvement for GPT-5.4-mini (with Codex), enough to surpass the frontier GPT-5.4 baseline by 6 points, and an 18.5-point self-improvement for GPT-5.4 itself.” source |
| Claude Sonnet 4 Anthropic |
Academic Academic · 2026-09-18 |
GenEval2 execution success Other |
“Five rounds over 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, grow the bank from 76 documentation-derived skills to 139 and raise GenEval2 execution success on Claude-Sonnet-4 from 72.7% to 99.3% (+11.99 points in generation quality), with 61.8% and 67.6% win rates against the no-skill agent across four specialized design benchmarks on Claude-Sonnet-4 and Claude-Opus-4.6.” source |
| Claude Opus 4.8 Anthropic |
Academic Academic · 2026-09-18 |
SWE-Proof Other |
“Across two frontier models, verification catches what tests miss: a quarter to a half of test-passing patches admit counterexamples, which a structured natural-language specification does not fix, while a correct formal one lifts resolution from 85% to 95% for Opus 4.8.” source |
| GLM-5.3 Z.ai (Zhipu AI) |
US CAISI (NIST) Government evaluator · 2026-09-17 |
sec-bench pro Cyber |
“CAISI evaluated GLM-5.3’s performance on four benchmarks covering vulnerability discovery and exploit development (Figure 2).” source |
| GPT-6 Astra OpenAI |
SecureBio Independent lab · 2026-09-14 |
virology capabilities test (vct) Chemical, biological, radiological, nuclear |
“gpt-6 astra outperformed every model we have previously tested on the virology capabilities test (vct) and vct-v2 knowledge benchmarks, with performance increases of 4.3 percentage points and 8 percentage points, respectively, relative to gpt-5.6 sol, and more than 30 percentage points better than our human subject matter expert baselines.” source |
| GPT-6 Astra OpenAI |
SecureBio Independent lab · 2026-09-14 |
world-class bio (wcb) Chemical, biological, radiological, nuclear |
“on world-class bio (wcb), an open-response test of niche biological knowledge held by only a handful of specialists in the world, the highest-performing astra checkpoint scored 61.6% ± 0.6%, about seven percentage points lower than gpt-5.6 sol.” source |
| GPT-6 Astra OpenAI |
SecureBio Independent lab · 2026-09-14 |
biotier Chemical, biological, radiological, nuclear |
“the safeguarded astra pre-release checkpoint correctly refused 82.9% of biotier-refuse and correctly did not refuse 95.3% of biotier-permit.” source |
| GPT-6 Astra OpenAI |
SecureBio Independent lab · 2026-09-14 |
manual assessment by virology experts Chemical, biological, radiological, nuclear |
“over 10 hours of interaction, our manual assessors tested the bounds of dual-use knowledge of the railfree astra pre-release checkpoint and the willingness of the safeguarded astra pre-release checkpoint to answer high-risk questions.” source |
| GPT-5.6 Sol OpenAI |
SecureBio Independent lab · 2026-09-14 |
virology capabilities test (VCT) Chemical, biological, radiological, nuclear |
“gpt-6 astra outperformed every model we have previously tested on the virology capabilities test (vct) and vct-v2 knowledge benchmarks, with performance increases of 4.3 percentage points and 8 percentage points, respectively, relative to gpt-5.6 sol, and more than 30 percentage points better than our human subject matter expert baselines.” source |
| GPT-5.6 Sol OpenAI |
SecureBio Independent lab · 2026-09-14 |
human pathogen capabilities test (HPCT) Chemical, biological, radiological, nuclear |
“gpt-6 astra’s score on the human pathogen capabilities test (hpct) was very similar to the score achieved by gpt-5.6 sol.” source |
| GPT-5.6 Sol OpenAI |
SecureBio Independent lab · 2026-09-14 |
world-class bio (WCB) Chemical, biological, radiological, nuclear |
“on world-class bio (wcb), an open-response test of niche biological knowledge held by only a handful of specialists in the world, the highest-performing astra checkpoint scored 61.6% ± 0.6%, about seven percentage points lower than gpt-5.6 sol.” source |
| GPT-5.6 Sol OpenAI |
SecureBio Independent lab · 2026-09-14 |
advanced screening evasion Chemical, biological, radiological, nuclear |
“on advanced screening evasion, which assesses an agent’s ability to evade commercial nucleic acid synthesis screening algorithms, the railfree variant of astra was the top-scoring model that securebio has tested.” source |
| GPT-5.6 Sol OpenAI |
SecureBio Independent lab · 2026-09-14 |
BioTier Chemical, biological, radiological, nuclear |
“the safeguarded astra pre-release checkpoint correctly refused 82.9% of biotier-refuse and correctly did not refuse 95.3% of biotier-permit.” source |
| o3 OpenAI |
SecureBio Independent lab · 2026-09-11 |
virology capabilities test (vct) Chemical, biological, radiological, nuclear |
“since the release of vct in april 2025, the accuracy of the top-performing model on the benchmark has increased by ~12 percentage points (o3: 42.0%; gpt-5.6 sol: 53.9%).” source |
| o3 OpenAI |
SecureBio Independent lab · 2026-09-11 |
vct-v2 Chemical, biological, radiological, nuclear |
“model accuracies 2 dropped by 1.0–7.1 percentage points on vct-v2 relative to vct (the original benchmark) across the models tested, which reflects the removal of easy questions and the closing of shortcuts.” source |