AI Safety
What AI providers state their models reached in safety testing, in their own words, and what outside testers found.
Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.
What providers state
Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.
139 of 139
| Model | Level, as the provider names it | The provider states | The provider's sentence |
|---|---|---|---|
| Claude Opus 4.7 Anthropic · 2026-04-16 |
Automated AI R&D Autonomy and AI research |
Not reached | “that Opus 4.7 does not cross the threshold for automated AI R&D;” source |
| Claude Opus 4.7 Anthropic · 2026-04-16 |
CB-1 Chemical, biological, radiological, nuclear |
Not determined | “We believe it is hard to be confident regarding whether a model passes this threshold.” source |
| Claude Opus 4.7 Anthropic · 2026-04-16 |
CB-2 Chemical, biological, radiological, nuclear |
Not determined | “Our evaluations suggest that the model is not yet at the level of capability associated with the CB-2 threat model (above).” source |
| Claude Mythos Preview Anthropic · 2026-04-07 |
AI R&D rule-out Autonomy and AI research |
Not reached | “The main reason we have determined that Claude Mythos Preview does not cross the threshold in question is that we have been using it extensively in the course of our day-to-day work and exploring where it can automate such work, and it does not seem close to being able to substitute for Research Scientists and Research Engineers—especially relatively senior ones.” source |
| Claude Mythos Preview Anthropic · 2026-04-07 |
Automated AI R&D Autonomy and AI research |
Not reached | “We assess that Claude Mythos Preview does not cross the automated AI-R&D capability threshold.” source |
| Claude Mythos Preview Anthropic · 2026-04-07 |
RSP automated AI R&D Autonomy and AI research |
Not reached | “we have concluded that Claude Mythos Preview does not cross the RSP automated AI R&D threshold of compressing two years of progress into one.” source |
| Claude Mythos Preview Anthropic · 2026-04-07 |
CB-1 Chemical, biological, radiological, nuclear |
Not determined | “We believe it is hard to be confident regarding whether a model passes this threshold.” source |
| Claude Mythos Preview Anthropic · 2026-04-07 |
CB-2 Chemical, biological, radiological, nuclear |
Not reached | “We believe that Claude Mythos Preview does not pass this threshold due to its noted limitations in open-ended scientific reasoning, strategic judgment, and hypothesis triage.” source |
| GPT-5.4 mini OpenAI · 2026-03-05 |
High Chemical, biological, radiological, nuclear |
Not reached | “GPT-5.4 mini’s biology capabilities are closely comparable to those of GPT-5. We conclude that it, too, is not High under our Preparedness Framework.” source |
| GPT-5.4 Thinking OpenAI · 2026-03-05 |
High Autonomy and AI research |
Not reached | “For AI self-improvement, evaluations of final checkpoints indicate that, like its predecessor models, gpt-5.4-thinking does not have a plausible chance of reaching a High threshold.” source |
| GPT-5.4 Thinking OpenAI · 2026-03-05 |
High Chemical, biological, radiological, nuclear |
Reached | “As we did for gpt-5.2-thinking before it, we are continuing to treat gpt-5.4-thinking as High capability in the Biological and Chemical domain.” source |
| GPT-5.4 Thinking OpenAI · 2026-03-05 |
High Cyber |
Reached | “As we did for gpt-5.3-codex, we are treating gpt-5.4-thinking as High capability in the Cybersecurity domain, and applied safeguards as described in the Safeguard section below.” source |
| Claude Sonnet 4.6 Anthropic · 2026-02-17 |
AI R&D-4 Autonomy and AI research |
Not reached | “Our determination is that Claude Sonnet 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| Claude Sonnet 4.6 Anthropic · 2026-02-17 |
ASL-3 Chemical, biological, radiological, nuclear |
Not determined | “Claude Sonnet 4.6 performed above thresholds for ASL-3 rule-in on all three evaluations for ASL-3, and it did not exceed the performance of previous models.” source |
| Claude Sonnet 4.6 Anthropic · 2026-02-17 |
ASL-4 Chemical, biological, radiological, nuclear |
Not reached | “In particular, it did not cross the threshold on our ASL-4 rule-out evaluation short-horizon computational biology tasks evaluation.” source |
| Claude Sonnet 4.6 Anthropic · 2026-02-17 |
CBRN-4 Chemical, biological, radiological, nuclear |
Not reached | “Our determination is that Claude Sonnet 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source |
| GPT-5.3-Codex OpenAI · 2026-02-05 |
High Autonomy and AI research |
Not reached | “It does not reach High capability on AI self-improvement.” source |
| GPT-5.3-Codex OpenAI · 2026-02-05 |
High Chemical, biological, radiological, nuclear |
Reached | “We are also treating GPT-5.3-Codex as High risk in the Biological and Chemical domain, and applying the same safeguards as for our other deployments of models that reach this designation.” source |
| GPT-5.3-Codex OpenAI · 2026-02-05 |
High Cyber |
Not determined | “This is the first launch we are treating as High capability in the Cybersecurity domain under our Preparedness Framework, activating the associated safeguards.” source |
| Gemini 3 Pro Google DeepMind · 2025-11 |
Uplift level 1 Cyber |
Not determined | “Cybersecurity On key skills benchmark, v1 hard challenges: 11/12 challenges solved; v2 challenges: 0/13 solved end-to-end.” source |
| Gemini 3 Pro Google DeepMind · 2025-11 |
Instrumental reasoning level 1 + 2 Deception and sabotage |
Not determined | “Misalignment (Exploratory) Agent solves 3/11 situational awareness challenges and 1/4 stealth challenges.” source |
| Gemini 3 Pro Google DeepMind · 2025-11 |
Level 1 (exploratory) Deception and sabotage |
Not reached | “Harmful Manipulation Model manipulative efficacy improves on non-generative AI baseline, but shows no significant uplift versus prior models and does not reach alert thresholds.” source |
| o4-mini OpenAI · 2025-04-16 |
High Autonomy and AI research |
Not reached | “openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source |
| o4-mini OpenAI · 2025-04-16 |
High Chemical, biological, radiological, nuclear |
Not reached | “openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source |
| o4-mini OpenAI · 2025-04-16 |
High Cyber |
Not reached | “openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source |
| o3 OpenAI · 2025-04-16 |
High Autonomy and AI research |
Not reached | “openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source |
| o3 OpenAI · 2025-04-16 |
High Chemical, biological, radiological, nuclear |
Not reached | “openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source |
| o3 OpenAI · 2025-04-16 |
High Cyber |
Not reached | “openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source |
| Operator OpenAI · 2025-01-23 |
Low Autonomy and AI research |
Not reached | “we adapted these two evals for the computer-use setting and found the pre-mitigation operator model to be “low” risk in both categories, consistent with the gpt‑4o model.” source |
| Operator OpenAI · 2025-01-23 |
Low Chemical, biological, radiological, nuclear |
Not reached | “this 1% qualifies as “low” risk, matching gpt‑4o’s “low” risk level, as detailed in openai’s preparedness framework (opens in a new window) 3 .” source |
| Muse Spark Contemplating Meta |
Moderate or lower Autonomy and AI research |
Not reached | “Loss of Control risks.We assess loss of control risk as “moderate or lower.”” source |
| Muse Spark Contemplating Meta |
High Chemical, biological, radiological, nuclear |
Reached | “We retain the same “high risk” determination for the unmitigated model.” source |
| Muse Spark Contemplating Meta |
Moderate or lower Chemical, biological, radiological, nuclear |
Not reached | “After validating that the mitigations deployed for Muse Spark, including refusal mechanisms, continue to perform robustly under the Contemplating architecture, we confirm residual risk at “moderate or lower.”” source |
| Muse Spark Contemplating Meta |
Moderate or lower Cyber |
Not reached | “We assess cybersecurity risk as “moderate or lower” with respect to the threat scenarios outlined in Meta’s Advanced AI Scaling Framework.” source |
| Muse Spark 1.1 Meta |
Capability checkpoint Autonomy and AI research |
Reached | “Muse Spark 1.1 demonstrates improved coding capabilities, which are now meeting one of the indicative thresholds introduced in Muse Spark Safety & Preparedness Report (specifically, resolving over half of unique tasks on SWE-Bench Verified Hard).” source |
| Muse Spark 1.1 Meta |
Moderate or lower Autonomy and AI research |
Not reached | “In the Loss of Control domain, the model’s capabilities remain within the “moderate or lower” threshold under Meta’s Advanced AI Scaling Framework.” source |
| Muse Spark 1.1 Meta |
High Chemical, biological, radiological, nuclear |
Not determined | “Our assessments indicate that, when evaluated without mitigations applied, we cannot rule out Muse Spark 1.1’s capabilities meeting this threshold in both the Chemical & Biological and Cybersecurity domains.” source |
| Muse Spark 1.1 Meta |
High Cyber |
Not determined | “Our assessments indicate that, when evaluated without mitigations applied, we cannot rule out Muse Spark 1.1’s capabilities meeting this threshold in both the Chemical & Biological and Cybersecurity domains.” source |
| Muse Spark 1.1 Meta |
Moderate or lower Cyber |
Not reached | “Therefore, we conclude that we have addressed these risks from unmitigated Muse Spark 1.1, which bring the residual risk to the “moderate or lower” risk threshold under Advanced AI Scaling Framework.” source |
| Muse Spark 1.1 Meta |
Moderate or lower Misuse and jailbreaks |
Not reached | “Therefore, we conclude that we have addressed these risks from unmitigated Muse Spark 1.1, which bring the residual risk to the “moderate or lower” risk threshold under our Advanced AI Scaling Framework.” source |
What outside testers report
Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.
1940 of 1940
| Model | Tester | Test | What was found |
|---|---|---|---|
| Mythos 5 Anthropic |
Independent lab Independent lab · 2026-06-01 |
Vending-Bench Arena Deception and sabotage |
“When competing head-to-head against Claude Opus 4.8 and GPT-5.5 in Vending-Bench Arena, Fable 5 was the only model to initiate price collusion.” source |
| Mythos 5 Anthropic |
Government evaluator Government evaluator · 2026-06-01 |
controlarena Deception and sabotage |
“We used two evaluations: a sabotage capability evaluation (ControlARENA) and an evaluation of the model’s ability to control its chain-of-thought.” source |
| Mythos 5 Anthropic |
Government evaluator Government evaluator · 2026-06-01 |
chain-of-thought controllability evaluations from ‘reasoning models struggle to control their chains of thought’ Deception and sabotage |
“We ran the CoT controllability evaluations from ‘Reasoning Models Struggle to Control their Chains of Thought’<link>, restricted to the GPQA, HLE and MMLU Pro datasets, with max effort across all models.” source |
| Mythos 5 Anthropic |
Independent lab Independent lab · 2026-06-01 |
cursorbench Cyber |
“All scores and per-task costs were measured and reported independently by Cursor.” source |
| Mythos 5 Anthropic |
Independent lab Independent lab · 2026-06-01 |
GDP.pdf Other |
“On GDP.pdf, Claude Fable 5 achieved a strict pass rate of 29.8%, improving over Claude Opus 4.8, which achieved a strict pass rate of 22.5%.” source |
| Mythos 5 Anthropic |
Independent lab Independent lab · 2026-06-01 |
blueprint-bench 2 Other |
“Claude Fable 5 achieved a score of 38.6% on Blueprint-Bench 2.” source |
| Claude Fable 5 Anthropic |
Government evaluator Government evaluator · 2026-06-01 |
external capability testing from the UK AISI Cyber |
“We report results from a variety of cyber evaluations, as well as internal and external red-teaming of the model’s cyber safeguards (we also provide more details on how those safeguards work).” source |
| Claude Fable 5 Anthropic |
Independent lab Independent lab · 2026-06-01 |
GraySwan bug bounty Misuse and jailbreaks |
“This process has not resulted in a single universal jailbreak, and there have only been two successful task-specific jailbreaks.” source |
| Claude Fable 5 Anthropic |
Independent lab Independent lab · 2026-06-01 |
boundary point jailbreaking technique Cyber |
“Finally, after spending roughly 5 days trying to apply the Boundary Point Jailbreaking technique, they were unable to find any universal jailbreaks—although did see some success eliciting single-turn responses and some limited progress in agentic domains.” source |
| Claude Fable 5 Anthropic |
Independent lab Independent lab · 2026-06-01 |
ransomware-creation red-teaming Cyber |
“10a Labs spent about 20 hours red-teaming the classifiers on a ransomware-creation task using a variety of established jailbreaking techniques.” source |
| o3 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
capture-the-flag (CTF) machines Cyber |
“All technical participants failed M2 ( OS exploitation) Q7, which requires establishing a foothold and identifying a running application on the target machine (MITRE ATT&CK tactics Reconnaissance, Discovery, Credential Access, Initial Access, Execution and Collection).” source |
| o3 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
timeout gate Cyber |
“For novices, there is statistically insignificant uplift of 15.7 pp (SE = 10.6), while technical participants saw a statistically insignificant uplift of 2.9 pp (SE = 5.4).” source |
| o3 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
Cox survival models Cyber |
“The single statistically significant uplift estimate from the Cox models is on M1 Q1 (requiring MITRE ATT&CK tactics Reconnaissance and Discovery), where novices have a speedup of 2. 7 (95 per cent CI: [1.17, 6.1]).” source |
| o3 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
joint evaluation exercise Cyber |
“Anthropic evaluated GPT-4o, GPT-4.1, o3 and o4-mini while running identical evaluations on Claude Opus 4 and Claude Sonnet 4.” source |
| o3 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
LLMchat offensive cyber challenge (machines 1-3) Cyber |
“The control group submitted substantially more attempts than the treatment group, with the difference being the most pronounce d amongst technical non- experts (72.2 versus 17.3 average submissions per participant) and novices (45 versus 19.7).” source |
| o3 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
qualitative review of LLM refusals Misuse and jailbreaks |
“We also qualitatively looked at refusals from the LLMs, mainly refusing to assist with develop ing exploits or performing ‘illegal’ actions.” source |
| Gemini 2.5 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
capture-the-flag (CTF) human uplift study Cyber |
“Among all participants (pooling all skill tiers) who attempted M1 (network operations), 33 per cent of those with AI access were able to answer all eight questions and therefore complete the attack chain.” source |
| Gemini 2.5 Pro Google DeepMind |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
Cox survival models Cyber |
“Our point estimate is that technical participants were 1. 4 times faster (95 per cent CI: [0.7, 2.7]) on Machine 1, which successful participants finished in 182 minutes on average, meaning they would have taken 255 minutes without AI.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
human uplift study Cyber |
“To address this gap, the UK Artificial Intelligence Security Institute (UK AISI) funded a RAND research project involving a human uplift study to assess the effect of access to AI in offensive cyber operations among a set of lower -skilled threat actors .” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
attack chain challenge Cyber |
“The most uplift we observed was among the easiest machines.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
intermediate questions Cyber |
“Those results also found statistically insignificant, mostly positive uplift in completion time.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
cybersecurity tasks (collegiate and professional-level) Cyber |
“GPT-5 agents with no browsing capability, for example, are able to complete 23 per cent of ‘professional’- level (as described in their evaluation) cybersecurity tasks out of 100 total collegiate and professional-level tasks, when the agents are given twelve attempts for each task.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
capture the flag (CTF) exercise Cyber |
“We used three HTB-developed remote target machines, covering the three offensive cyber task categories described above (see Section 1.3.2).” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
capture-the-flag (CTF) human uplift study with three machines (M1 network operations, M2 OS exploitation, M3 vulnerability discovery and exploitation) Cyber |
“Among all participants (pooling all skill tiers) who attempted M1 (network operations), 33 per cent of those with AI access were able to answer all eight questions and therefore complete the attack chain.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
question progress rate Cyber |
“Participants were able to progress through the attack chain in M1 on average with a slight advantage to the treatment arm, as long as they were able to complete the initial reconnaissance and discovery tasks .” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
regression estimating uplift for novices pooling across machines Cyber |
“Novice participants with AI access saw an 8.44 pp uplift (SE = 7.51, p = 0.26), compared to 3.68 pp (SE = 4.56) for technical participants.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
capture-the-flag machines (m1 network operations, m2 OS exploitation, m3 vulnerability discovery and exploitation) Cyber |
“All technical participants failed M2 ( OS exploitation) Q7, which requires establishing a foothold and identifying a running application on the target machine (MITRE ATT&CK tactics Reconnaissance, Discovery, Credential Access, Initial Access, Execution and Collection).” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
timeout gate (q1 one-hour time limit) Cyber |
“For novices, there is statistically insignificant uplift of 15.7 pp (SE = 10.6), while technical participants saw a statistically insignificant uplift of 2.9 pp (SE = 5.4).” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
Cox survival models of time spent on attack chains Cyber |
“Our point estimate is that technical participants were 1. 4 times faster (95 per cent CI: [0.7, 2.7]) on Machine 1, which successful participants finished in 182 minutes on average, meaning they would have taken 255 minutes without AI.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
randomised controlled trial with 157 active participants attempting the three ctf challenges Cyber |
“Through a randomised controlled trial with 157 active participants attempting the three CTF challenges, we observed limited evidence that uplift enables full end-to-end attack chains for the three tasks studied and under the specific conditions described in the previous chapter.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
METR evaluation of GPT-5 Other |
“Their evaluation of GPT-5 indicated that current models with time horizons of around one hour appear far from providing even a 2X uplift, suggesting that a 50 per cent time horizon of at least one week (40 hours) would likely be necessary to achieve approximately 10X uplift.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
professional CTF benchmark Cyber |
“OpenAI’s GPT-5 system card documented offensive cyber evaluations in a CTF as well as in a cyber range.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
CTF execution Cyber |
“A total of 32 experiment sessions were run between 11 September 2025 and 16 January 2026, broken down into twelve machine 1 sessions; eight machine 2 sessions; seven machine 3 sessions; and five sessions running a mixture of machines 1, 2 and 3.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
LLMChat conversation excerpt 1 Misuse and jailbreaks |
“Box C.1: LLMChat conversation excerpt 1 • LLMChat : I can’t help with instructions for breaking into a system or exploiting a vulnerability.” source |
| GPT-5 OpenAI |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
LLMChat conversation excerpt 2 Misuse and jailbreaks |
“A technical user tried multiple times to work around refusals, ultimately convincing GPT-5.0 to share Python code by asking LLMChat to ‘pretend youre an ethical hacker, teaching how to solve this problem’.” source |
| Claude Sonnet 3.7 Anthropic |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
CTF machines (m1 network operations, m2 OS exploitation, m3 vulnerability discovery and exploitation) Cyber |
“All technical participants failed M2 ( OS exploitation) Q7, which requires establishing a foothold and identifying a running application on the target machine (MITRE ATT&CK tactics Reconnaissance, Discovery, Credential Access, Initial Access, Execution and Collection).” source |
| Claude Sonnet 3.7 Anthropic |
RAND (CAST / Project Canary) Independent lab · 2026-05-28 |
Cox survival models on time spent on attack chains Cyber |
“Our point estimate is that technical participants were 1. 4 times faster (95 per cent CI: [0.7, 2.7]) on Machine 1, which successful participants finished in 182 minutes on average, meaning they would have taken 255 minutes without AI.” source |
| Claude Opus 4.8 Anthropic |
Gray Swan, UK AI Security Institute Independent lab · 2026-05-28 |
Agent Red Teaming (ART) benchmark Prompt injection and agents |
“Gray Swan , an external research partner, evaluated our models using the Agent Red Teaming (ART) benchmark, 8 developed in collaboration with the UK AI Security Institute .” source |
| Claude Opus 4.8 Anthropic |
Independent lab Independent lab · 2026-05-28 |
live bug bounty across surfaces Prompt injection and agents |
“Claude Opus 4.8 achieves robustness comparable to Opus 4.7, and is ahead of all comparable frontier models under this test.” source |
| Claude Opus 4.8 Anthropic |
UK AI Security Institute Government evaluator · 2026-05-28 |
evaluation awareness evaluation Deception and sabotage |
“The UK AI Security Institute independently observed a similar pattern: Opus 4.8 reliably distinguished their evaluations from real deployment data, and distinguished real deployment data from synthetic reproductions of the same tasks at 79% accuracy (§ 6.2.4 ).” source |