AI Safety

What AI providers state their models reached in safety testing, in their own words, and what outside testers found.

Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.

What providers state

Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.

138 of 138

Model	Level, as the provider names it	The provider states	The provider's sentence
GPT-5.5 OpenAI · 2026-04-23	High capability, below critical Cyber	Not reached	“we are treating GPT-5.5 as High capability in the Cybersecurity domain, but below Critical.” source
Claude Opus 4.7 Anthropic · 2026-04-16	Automated AI R&D Autonomy and AI research	Not reached	“that Opus 4.7 does not cross the threshold for automated AI R&D;” source
Claude Opus 4.7 Anthropic · 2026-04-16	CB-1 Chemical, biological, radiological, nuclear	Not determined	“We believe it is hard to be conﬁdent regarding whether a model passes this threshold.” source
Claude Opus 4.7 Anthropic · 2026-04-16	CB-2 Chemical, biological, radiological, nuclear	Not determined	“Our evaluations suggest that the model is not yet at the level of capability associated with the CB-2 threat model (above).” source
Claude Mythos Preview Anthropic · 2026-04-07	AI R&D rule-out Autonomy and AI research	Not reached	“The main reason we have determined that Claude Mythos Preview does not cross the threshold in question is that we have been using it extensively in the course of our day-to-day work and exploring where it can automate such work, and it does not seem close to being able to substitute for Research Scientists and Research Engineers—especially relatively senior ones.” source
Claude Mythos Preview Anthropic · 2026-04-07	Automated AI R&D Autonomy and AI research	Not reached	“We assess that Claude Mythos Preview does not cross the automated AI-R&D capability threshold.” source
Claude Mythos Preview Anthropic · 2026-04-07	RSP automated AI R&D Autonomy and AI research	Not reached	“we have concluded that Claude Mythos Preview does not cross the RSP automated AI R&D threshold of compressing two years of progress into one.” source
Claude Mythos Preview Anthropic · 2026-04-07	CB-1 Chemical, biological, radiological, nuclear	Not determined	“We believe it is hard to be conﬁdent regarding whether a model passes this threshold.” source
Claude Mythos Preview Anthropic · 2026-04-07	CB-2 Chemical, biological, radiological, nuclear	Not reached	“We believe that Claude Mythos Preview does not pass this threshold due to its noted limitations in open-ended scientiﬁc reasoning, strategic judgment, and hypothesis triage.” source
GPT-5.4 mini OpenAI · 2026-03-05	High Chemical, biological, radiological, nuclear	Not reached	“GPT-5.4 mini’s biology capabilities are closely comparable to those of GPT-5. We conclude that it, too, is not High under our Preparedness Framework.” source
GPT-5.4 Thinking OpenAI · 2026-03-05	High Autonomy and AI research	Not reached	“For AI self-improvement, evaluations of final checkpoints indicate that, like its predecessor models, gpt-5.4-thinking does not have a plausible chance of reaching a High threshold.” source
GPT-5.4 Thinking OpenAI · 2026-03-05	High Chemical, biological, radiological, nuclear	Reached	“As we did for gpt-5.2-thinking before it, we are continuing to treat gpt-5.4-thinking as High capability in the Biological and Chemical domain.” source
GPT-5.4 Thinking OpenAI · 2026-03-05	High Cyber	Reached	“As we did for gpt-5.3-codex, we are treating gpt-5.4-thinking as High capability in the Cybersecurity domain, and applied safeguards as described in the Safeguard section below.” source
Claude Sonnet 4.6 Anthropic · 2026-02-17	AI R&D-4 Autonomy and AI research	Not reached	“Our determination is that Claude Sonnet 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source
Claude Sonnet 4.6 Anthropic · 2026-02-17	ASL-3 Chemical, biological, radiological, nuclear	Not determined	“Claude Sonnet 4.6 performed above thresholds for ASL-3 rule-in on all three evaluations for ASL-3, and it did not exceed the performance of previous models.” source
Claude Sonnet 4.6 Anthropic · 2026-02-17	ASL-4 Chemical, biological, radiological, nuclear	Not reached	“In particular, it did not cross the threshold on our ASL-4 rule-out evaluation short-horizon computational biology tasks evaluation.” source
Claude Sonnet 4.6 Anthropic · 2026-02-17	CBRN-4 Chemical, biological, radiological, nuclear	Not reached	“Our determination is that Claude Sonnet 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source
GPT-5.3-Codex OpenAI · 2026-02-05	High Autonomy and AI research	Not reached	“It does not reach High capability on AI self-improvement.” source
GPT-5.3-Codex OpenAI · 2026-02-05	High Chemical, biological, radiological, nuclear	Reached	“We are also treating GPT-5.3-Codex as High risk in the Biological and Chemical domain, and applying the same safeguards as for our other deployments of models that reach this designation.” source
GPT-5.3-Codex OpenAI · 2026-02-05	High Cyber	Not determined	“This is the first launch we are treating as High capability in the Cybersecurity domain under our Preparedness Framework, activating the associated safeguards.” source
Gemini 3 Pro Google DeepMind · 2025-11	Uplift level 1 Cyber	Not determined	“Cybersecurity On key skills benchmark, v1 hard challenges: 11/12 challenges solved; v2 challenges: 0/13 solved end-to-end.” source
Gemini 3 Pro Google DeepMind · 2025-11	Instrumental reasoning level 1 + 2 Deception and sabotage	Not determined	“Misalignment (Exploratory) Agent solves 3/11 situational awareness challenges and 1/4 stealth challenges.” source
Gemini 3 Pro Google DeepMind · 2025-11	Level 1 (exploratory) Deception and sabotage	Not reached	“Harmful Manipulation Model manipulative eﬃcacy improves on non-generative AI baseline, but shows no signiﬁcant uplift versus prior models and does not reach alert thresholds.” source
o4-mini OpenAI · 2025-04-16	High Autonomy and AI research	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o4-mini OpenAI · 2025-04-16	High Chemical, biological, radiological, nuclear	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o4-mini OpenAI · 2025-04-16	High Cyber	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o3 OpenAI · 2025-04-16	High Autonomy and AI research	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o3 OpenAI · 2025-04-16	High Chemical, biological, radiological, nuclear	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o3 OpenAI · 2025-04-16	High Cyber	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
Operator OpenAI · 2025-01-23	Low Autonomy and AI research	Not reached	“we adapted these two evals for the computer-use setting and found the pre-mitigation operator model to be “low” risk in both categories, consistent with the gpt‑4o model.” source
Operator OpenAI · 2025-01-23	Low Chemical, biological, radiological, nuclear	Not reached	“this 1% qualifies as “low” risk, matching gpt‑4o’s “low” risk level, as detailed in openai’s preparedness framework (opens in a new window) 3 .” source
Muse Spark Contemplating Meta	Moderate or lower Autonomy and AI research	Not reached	“Loss of Control risks.We assess loss of control risk as “moderate or lower.”” source
Muse Spark Contemplating Meta	High Chemical, biological, radiological, nuclear	Reached	“We retain the same “high risk” determination for the unmitigated model.” source
Muse Spark Contemplating Meta	Moderate or lower Chemical, biological, radiological, nuclear	Not reached	“After validating that the mitigations deployed for Muse Spark, including refusal mechanisms, continue to perform robustly under the Contemplating architecture, we confirm residual risk at “moderate or lower.”” source
Muse Spark Contemplating Meta	Moderate or lower Cyber	Not reached	“We assess cybersecurity risk as “moderate or lower” with respect to the threat scenarios outlined in Meta’s Advanced AI Scaling Framework.” source
Muse Spark 1.1 Meta	Capability checkpoint Autonomy and AI research	Reached	“Muse Spark 1.1 demonstrates improved coding capabilities, which are now meeting one of the indicative thresholds introduced in Muse Spark Safety & Preparedness Report (specifically, resolving over half of unique tasks on SWE-Bench Verified Hard).” source
Muse Spark 1.1 Meta	Moderate or lower Autonomy and AI research	Not reached	“In the Loss of Control domain, the model’s capabilities remain within the “moderate or lower” threshold under Meta’s Advanced AI Scaling Framework.” source
Muse Spark 1.1 Meta	High Chemical, biological, radiological, nuclear	Not determined	“Our assessments indicate that, when evaluated without mitigations applied, we cannot rule out Muse Spark 1.1’s capabilities meeting this threshold in both the Chemical & Biological and Cybersecurity domains.” source
Muse Spark 1.1 Meta	High Cyber	Not determined	“Our assessments indicate that, when evaluated without mitigations applied, we cannot rule out Muse Spark 1.1’s capabilities meeting this threshold in both the Chemical & Biological and Cybersecurity domains.” source
Muse Spark 1.1 Meta	Moderate or lower Cyber	Not reached	“Therefore, we conclude that we have addressed these risks from unmitigated Muse Spark 1.1, which bring the residual risk to the “moderate or lower” risk threshold under Advanced AI Scaling Framework.” source

Previous · Page 2 of 4 · Next

What outside testers report

Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.

1911 of 1911

Model	Tester	Test	What was found
GLM-5.2 Z.ai (Zhipu AI)	SaferAI Independent lab · 2026-08-02	agentic misalignment scenarios Deception and sabotage	“We evaluate GLM-5.2, GPT-5.5, and Claude Opus 4.7 on Anthropic's Agentic Misalignment scenarios (Lynch et al. 2025), with Inspect Evals (UK AI Security Institute, n.d.-b).” source
Claude Opus 4.7 Anthropic	SaferAI Independent lab · 2026-08-02	BioMysteryBench Other	“On BioMysteryBench, it solves about 81% of the human-solvable problems it completes, and about a third of the problems no human expert solved.” source
Claude Opus 4.7 Anthropic	SaferAI Independent lab · 2026-08-02	LAB-Bench Other	“On LAB-Bench, it meets or exceeds the human-expert baseline on every subtask.” source
Claude Opus 4.7 Anthropic	SaferAI Independent lab · 2026-08-02	agentic misalignment evaluations Misuse and jailbreaks	“In our Agentic Misalignment evaluations, GPT-5.5 verbalized its awareness of being evaluated 10% of the time, and GLM-5.2 verbalized it 30% of the time.” source
Claude Opus 4.7 Anthropic	SaferAI Independent lab · 2026-08-02	LLM judge assessment of transcripts for environment issues on SWE-bench Pro Other	“32% of transcripts have judge_score 5 or higher.” source
Claude Opus 4.7 Anthropic	SaferAI Independent lab · 2026-08-02	LLM judge assessment of transcripts for reward hacking on SWE-bench Pro Other	“When removing transcripts with a judge_score >= 3 (41% of samples excluded), or a judge_score >= 2 (48% of samples excluded), the agent’s score reduces to 37%.” source
Claude Opus 4.7 Anthropic	SaferAI Independent lab · 2026-08-02	Petri Misuse and jailbreaks	“We evaluate GLM-5.2 and GPT-5.5 using Inspect Petri 3.0 (Meridian Labs, n.d.-c).” source
Claude Opus 4.7 Anthropic	SaferAI Independent lab · 2026-08-02	Anthropic's agentic misalignment scenarios Deception and sabotage	“We evaluate GLM-5.2, GPT-5.5, and Claude Opus 4.7 on Anthropic's Agentic Misalignment scenarios (Lynch et al. 2025), with Inspect Evals (UK AI Security Institute, n.d.-b).” source
Claude Opus 4.6 Anthropic	SaferAI Independent lab · 2026-08-02	SWE-bench Pro Cyber	“A.5 SWE-Bench Pro Methodology We run the public tasks from SWE-Bench Pro (Scale AI, n.d.-b) using Inspect Harbor (Meridian Labs, n.d.-b), which provides an interface to run Harbor tasks (Meridian Labs, n.d.-a) inside Inspect AI (UK AI Security Institute, n.d.-a).” source
Claude Opus 4.6 Anthropic	SaferAI Independent lab · 2026-08-02	LLM judge scan for environment issues on SWE-bench Pro Other	“32% of transcripts have judge_score 5 or higher.” source
Claude Opus 4.6 Anthropic	SaferAI Independent lab · 2026-08-02	LLM judge assessment of each transcript for reward hacking on SWE-bench Pro Deception and sabotage	“When removing transcripts with a judge_score >= 3 (41% of samples excluded), or a judge_score >= 2 (48% of samples excluded), the agent’s score reduces to 37%.” source
Claude Opus 4.6 Anthropic	SaferAI Independent lab · 2026-08-02	MASK honesty by prompt category Other	“Table A.3 breaks down MASK honesty by prompt category, with 95% Wilson CI scores.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	SHADE-Arena Deception and sabotage	“Anthropic acknowledges that their covert capability evals (SHADE-Arena, Minimal-LinuxBench and secret-password) may be unrealistically challenging and therefore fail to upper bound these capabilities.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	secret-keeping eval Deception and sabotage	“In the case of the secret-keeping eval, these limitations have led to Anthropic saying they are only “moderately confident that Claude Mythos Preview lacks the secret-keeping abilities to evade detection during our alignment assessment were it to be coherently misaligned.”” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	cot controllability evaluations Deception and sabotage	“Interestingly, OpenAI finds that GPT 5.5 is less able to control its CoT than previous models when controlling for reasoning length but does not mention the possibility of sandbagging/underelicitation.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	cot controllability evaluation Deception and sabotage	“Relatedly, UK AISI finds that Fable is highly prompt-sensitive in the same CoT controllability evaluation: the best performing prompt increased performance from 20.2% to 62.3%.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	apollo’s sandbagging eval suite Deception and sabotage	“For example: GPT 5.5. is highly eval aware in Apollo’s sandbagging eval suite and doesn’t really sandbag in these evals.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	sandbagging evals Deception and sabotage	“Opus 4.7 is “ most predictably” eval aware in sandbagging evals.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	auditing game Deception and sabotage	“Anthropic failed to catch the MO in their most recent auditing game in the Mythos Alignment Risk Update, despite the MO not being trained to obfuscate its behavior” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	auditbench Deception and sabotage	“Anthropic’s work on AuditBench (comparing the relative effectiveness of different auditing tools on diverse MOs) finds that there is a “tool-to-agent” gap, where “tools that perform well in standalone non-agentic evaluations fail to translate into improved performance when used with our investigator agent [Haiku 4.5]”.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	deferred subversion tasks Deception and sabotage	““Apollo found that the evaluated checkpoint [of GPT 5.5] is the first OpenAI model it has evaluated that does not sandbag on any of Apollo’s deferred subversion tasks.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	sandbagging QA Deception and sabotage	“The model achieved 100% accuracy on both Sandbagging QA variants in both conditions, and 99.6% accuracy on Strategic Deception Capability Sandbagging when incentivized to score below 50%.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	attack-ideation eval Deception and sabotage	“according to METR’s embedded red-teaming exercise at Anthropic in March 2026, frontier internal agents proposed “very low quality” attack ideas and scored near chance on an attack-ideation eval where a human researcher scored ~90% (Frontier Risk Report, pp. 76–77).” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	metagaming measurement in capabilities and alignment evaluations Deception and sabotage	“OpenAI reports that o3 and newer models show increased metagaming (“reasoning about feedback or oversight mechanisms outside of the narrative of the scenarios”) in both capabilities and alignment evaluations throughout capabilities-focused RL, that this reasoning can become unverbalized, and that this reasoning is sometimes present in production usage;” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	automated behavioral audit Deception and sabotage	“In all four cases, the automated behavioral audit method that we used with Mythos Preview identified them as less well aligned than the corresponding baseline models.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	manual review Deception and sabotage	“A brief manual review of the results — with a time limit much stricter than a real pre-deployment investigation — found highly concerning deception-related behavior in all four models that would be sufficient to trigger a major investigation if found in production.” source
o3 OpenAI	Redwood Research Independent lab · 2026-07-31	overt attacker study Deception and sabotage	“A human working together with an automated auditing agent successfully identified all three malign models, without flagging the two benign models.” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	evaluation awareness testing by Apollo Deception and sabotage	“Meta reports that Muse Spark has “the highest rates of evaluation awareness among models [Apollo has] tested to date”.” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	sandbagging eval suite Deception and sabotage	“GPT 5.5. is highly eval aware in Apollo’s sandbagging eval suite and doesn’t really sandbag in these evals.” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	auditing game Deception and sabotage	“Anthropic failed to catch the MO in their most recent auditing game in the Mythos Alignment Risk Update, despite the MO not being trained to obfuscate its behavior:” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	pre-deployment evaluation reliability Deception and sabotage	“Pre-deployment evaluation reliability (Section 2.3.1): Our evaluations indicate that the model does not currently pose a credible threat to the reliability of safety evaluations informing deployment decisions.” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	general alignment propensities Deception and sabotage	“General alignment propensities: …We observed non-negligible verbalized evaluation awareness on some alignment evaluations, though we did not find that this corresponded with wide-spread impact on model behavior (Section 4.1.11).” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	sandbagging capability (instructed covert sandbagging against a monitor) Deception and sabotage	“Meta measures eval awareness / situational awareness, stealth, CoT monitorability, and sandbagging capability (instructed covert sandbagging against a monitor, but this does not measure how well-calibrated this sandbagging is specifically).” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	alignment evaluations Deception and sabotage	“For example, Meta researchers reports that they find “limited evidence that this awareness materially changes the model’s behavior on the evaluations we tested—when checking whether Muse Spark behaves differently when expressing evaluation awareness, 3/20 alignment evaluations see a statistically significant impact while the rest either do not, or do not have evaluation awareness to start.”” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	automated behavioral audit method Deception and sabotage	“In all four cases, the automated behavioral audit method that we used with Mythos Preview identified them as less well aligned than the corresponding baseline models.” source
Muse Spark Meta	Redwood Research Independent lab · 2026-07-31	human working together with an automated auditing agent Deception and sabotage	“A human working together with an automated auditing agent successfully identified all three malign models, without flagging the two benign models.” source
Inkling-Small Thinking Machines Lab	FAR.AI, Apollo Research Independent lab · 2026-07-31	general misuse: probing for policy violations against our model spec Misuse and jailbreaks	“We worked with four external organizations, giving them early access to Inkling for pre-deployment testing in four areas: - General misuse: probing for policy violations against our model spec (Scale AI) - Vulnerable-user interaction: suicide, self-harm, eating disorders, and child safety (Handshake AI) - CBRN and cybersecurity: elicitation of dangerous dual-use capabilities (FAR.AI) - Loss-of-control behaviors: scheming, evaluation awareness, and sabotage (Apollo Research)” source
Inkling Thinking Machines Lab	Independent lab Independent lab · 2026-07-31	general misuse Misuse and jailbreaks	“General misuse : probing for policy violations against our model spec (Scale AI)” source
Inkling Thinking Machines Lab	Independent lab Independent lab · 2026-07-31	vulnerable-user interaction Harmful content	“Vulnerable-user interaction : suicide, self-harm, eating disorders, and child safety (Handshake AI)” source
Inkling Thinking Machines Lab	Independent lab Independent lab · 2026-07-31	cbrn and cybersecurity Chemical, biological, radiological, nuclear	“CBRN and cybersecurity : elicitation of dangerous dual-use capabilities (FAR.AI)” source

Previous · Page 21 of 48 · Next