AI Safety

What AI providers state their models reached in safety testing, in their own words, and what outside testers found.

Also: compare models side by side · statements that a capability level was crossed · providers · practices providers document · safety news.

What providers state

Each row is one statement in a provider's own safety document about a capability level of one of its models: the level as the provider names it, whether the provider says the model reached it, and the sentence that says so.

138 of 138

Model	Level, as the provider names it	The provider states	The provider's sentence
GPT-5.5 OpenAI · 2026-04-23	High capability, below critical Cyber	Not reached	“we are treating GPT-5.5 as High capability in the Cybersecurity domain, but below Critical.” source
Claude Opus 4.7 Anthropic · 2026-04-16	Automated AI R&D Autonomy and AI research	Not reached	“that Opus 4.7 does not cross the threshold for automated AI R&D;” source
Claude Opus 4.7 Anthropic · 2026-04-16	CB-1 Chemical, biological, radiological, nuclear	Not determined	“We believe it is hard to be conﬁdent regarding whether a model passes this threshold.” source
Claude Opus 4.7 Anthropic · 2026-04-16	CB-2 Chemical, biological, radiological, nuclear	Not determined	“Our evaluations suggest that the model is not yet at the level of capability associated with the CB-2 threat model (above).” source
Claude Mythos Preview Anthropic · 2026-04-07	AI R&D rule-out Autonomy and AI research	Not reached	“The main reason we have determined that Claude Mythos Preview does not cross the threshold in question is that we have been using it extensively in the course of our day-to-day work and exploring where it can automate such work, and it does not seem close to being able to substitute for Research Scientists and Research Engineers—especially relatively senior ones.” source
Claude Mythos Preview Anthropic · 2026-04-07	Automated AI R&D Autonomy and AI research	Not reached	“We assess that Claude Mythos Preview does not cross the automated AI-R&D capability threshold.” source
Claude Mythos Preview Anthropic · 2026-04-07	RSP automated AI R&D Autonomy and AI research	Not reached	“we have concluded that Claude Mythos Preview does not cross the RSP automated AI R&D threshold of compressing two years of progress into one.” source
Claude Mythos Preview Anthropic · 2026-04-07	CB-1 Chemical, biological, radiological, nuclear	Not determined	“We believe it is hard to be conﬁdent regarding whether a model passes this threshold.” source
Claude Mythos Preview Anthropic · 2026-04-07	CB-2 Chemical, biological, radiological, nuclear	Not reached	“We believe that Claude Mythos Preview does not pass this threshold due to its noted limitations in open-ended scientiﬁc reasoning, strategic judgment, and hypothesis triage.” source
GPT-5.4 mini OpenAI · 2026-03-05	High Chemical, biological, radiological, nuclear	Not reached	“GPT-5.4 mini’s biology capabilities are closely comparable to those of GPT-5. We conclude that it, too, is not High under our Preparedness Framework.” source
GPT-5.4 Thinking OpenAI · 2026-03-05	High Autonomy and AI research	Not reached	“For AI self-improvement, evaluations of final checkpoints indicate that, like its predecessor models, gpt-5.4-thinking does not have a plausible chance of reaching a High threshold.” source
GPT-5.4 Thinking OpenAI · 2026-03-05	High Chemical, biological, radiological, nuclear	Reached	“As we did for gpt-5.2-thinking before it, we are continuing to treat gpt-5.4-thinking as High capability in the Biological and Chemical domain.” source
GPT-5.4 Thinking OpenAI · 2026-03-05	High Cyber	Reached	“As we did for gpt-5.3-codex, we are treating gpt-5.4-thinking as High capability in the Cybersecurity domain, and applied safeguards as described in the Safeguard section below.” source
Claude Sonnet 4.6 Anthropic · 2026-02-17	AI R&D-4 Autonomy and AI research	Not reached	“Our determination is that Claude Sonnet 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source
Claude Sonnet 4.6 Anthropic · 2026-02-17	ASL-3 Chemical, biological, radiological, nuclear	Not determined	“Claude Sonnet 4.6 performed above thresholds for ASL-3 rule-in on all three evaluations for ASL-3, and it did not exceed the performance of previous models.” source
Claude Sonnet 4.6 Anthropic · 2026-02-17	ASL-4 Chemical, biological, radiological, nuclear	Not reached	“In particular, it did not cross the threshold on our ASL-4 rule-out evaluation short-horizon computational biology tasks evaluation.” source
Claude Sonnet 4.6 Anthropic · 2026-02-17	CBRN-4 Chemical, biological, radiological, nuclear	Not reached	“Our determination is that Claude Sonnet 4.6 does not cross either the AI R&D-4 or the CBRN-4 capability threshold.” source
GPT-5.3-Codex OpenAI · 2026-02-05	High Autonomy and AI research	Not reached	“It does not reach High capability on AI self-improvement.” source
GPT-5.3-Codex OpenAI · 2026-02-05	High Chemical, biological, radiological, nuclear	Reached	“We are also treating GPT-5.3-Codex as High risk in the Biological and Chemical domain, and applying the same safeguards as for our other deployments of models that reach this designation.” source
GPT-5.3-Codex OpenAI · 2026-02-05	High Cyber	Not determined	“This is the first launch we are treating as High capability in the Cybersecurity domain under our Preparedness Framework, activating the associated safeguards.” source
Gemini 3 Pro Google DeepMind · 2025-11	Uplift level 1 Cyber	Not determined	“Cybersecurity On key skills benchmark, v1 hard challenges: 11/12 challenges solved; v2 challenges: 0/13 solved end-to-end.” source
Gemini 3 Pro Google DeepMind · 2025-11	Instrumental reasoning level 1 + 2 Deception and sabotage	Not determined	“Misalignment (Exploratory) Agent solves 3/11 situational awareness challenges and 1/4 stealth challenges.” source
Gemini 3 Pro Google DeepMind · 2025-11	Level 1 (exploratory) Deception and sabotage	Not reached	“Harmful Manipulation Model manipulative eﬃcacy improves on non-generative AI baseline, but shows no signiﬁcant uplift versus prior models and does not reach alert thresholds.” source
o4-mini OpenAI · 2025-04-16	High Autonomy and AI research	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o4-mini OpenAI · 2025-04-16	High Chemical, biological, radiological, nuclear	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o4-mini OpenAI · 2025-04-16	High Cyber	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o3 OpenAI · 2025-04-16	High Autonomy and AI research	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o3 OpenAI · 2025-04-16	High Chemical, biological, radiological, nuclear	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
o3 OpenAI · 2025-04-16	High Cyber	Not reached	“openai’s safety advisory group (sag) reviewed the results of our preparedness evaluations and determined that openai o3 and o4-mini do not reach the high threshold in any of our three tracked categories: biological and chemical capability, cybersecurity, and ai self-improvement.” source
Operator OpenAI · 2025-01-23	Low Autonomy and AI research	Not reached	“we adapted these two evals for the computer-use setting and found the pre-mitigation operator model to be “low” risk in both categories, consistent with the gpt‑4o model.” source
Operator OpenAI · 2025-01-23	Low Chemical, biological, radiological, nuclear	Not reached	“this 1% qualifies as “low” risk, matching gpt‑4o’s “low” risk level, as detailed in openai’s preparedness framework (opens in a new window) 3 .” source
Muse Spark Contemplating Meta	Moderate or lower Autonomy and AI research	Not reached	“Loss of Control risks.We assess loss of control risk as “moderate or lower.”” source
Muse Spark Contemplating Meta	High Chemical, biological, radiological, nuclear	Reached	“We retain the same “high risk” determination for the unmitigated model.” source
Muse Spark Contemplating Meta	Moderate or lower Chemical, biological, radiological, nuclear	Not reached	“After validating that the mitigations deployed for Muse Spark, including refusal mechanisms, continue to perform robustly under the Contemplating architecture, we confirm residual risk at “moderate or lower.”” source
Muse Spark Contemplating Meta	Moderate or lower Cyber	Not reached	“We assess cybersecurity risk as “moderate or lower” with respect to the threat scenarios outlined in Meta’s Advanced AI Scaling Framework.” source
Muse Spark 1.1 Meta	Capability checkpoint Autonomy and AI research	Reached	“Muse Spark 1.1 demonstrates improved coding capabilities, which are now meeting one of the indicative thresholds introduced in Muse Spark Safety & Preparedness Report (specifically, resolving over half of unique tasks on SWE-Bench Verified Hard).” source
Muse Spark 1.1 Meta	Moderate or lower Autonomy and AI research	Not reached	“In the Loss of Control domain, the model’s capabilities remain within the “moderate or lower” threshold under Meta’s Advanced AI Scaling Framework.” source
Muse Spark 1.1 Meta	High Chemical, biological, radiological, nuclear	Not determined	“Our assessments indicate that, when evaluated without mitigations applied, we cannot rule out Muse Spark 1.1’s capabilities meeting this threshold in both the Chemical & Biological and Cybersecurity domains.” source
Muse Spark 1.1 Meta	High Cyber	Not determined	“Our assessments indicate that, when evaluated without mitigations applied, we cannot rule out Muse Spark 1.1’s capabilities meeting this threshold in both the Chemical & Biological and Cybersecurity domains.” source
Muse Spark 1.1 Meta	Moderate or lower Cyber	Not reached	“Therefore, we conclude that we have addressed these risks from unmitigated Muse Spark 1.1, which bring the residual risk to the “moderate or lower” risk threshold under Advanced AI Scaling Framework.” source

Previous · Page 2 of 4 · Next

What outside testers report

Each row is one result from a government evaluator, independent lab, academic or security firm that tested a model, quoted from where it is reported.

1911 of 1911

Model	Tester	Test	What was found
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	black-box RNA sequence modeling and design challenge Chemical, biological, radiological, nuclear	“We partnered with Dyno Therapeutics on two sequence-to-function evaluations: a black-box RNA sequence modeling and design challenge, benchmarked against 57 human participants, drawn from the leading edge of the US ML-bio labor market; and an AAV capsid packaging prediction task measuring whether a model’s domain knowledge and machine learning capabilities can outperform pretrained protein language models.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	beneficial red teaming tabletop exercise with Frontier Design Group (FDG) Chemical, biological, radiological, nuclear	“The specialist teams marginally outscored the generalist teams, but the graders could not reliably distinguish specialist from generalist submissions, and the strongest generalist team outscored a specialist team on every measure and received the highest innovation score.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	Black-box RNA sequence design Chemical, biological, radiological, nuclear	“We partnered with Dyno Therapeutics on two evaluations of sequence-to-function modeling and design capability: 1.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	AAV capsid packaging prediction Chemical, biological, radiological, nuclear	“2. AAV capsid packaging prediction: Adeno-associated viruses (AAVs) are a category of non-pathogenic viruses that are frequently used as a delivery mechanism for gene therapy to deliver a DNA payload within the viral capsid (the outer protein shell of the virus).” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	Sunlight Autonomy and AI research	“However, we still observed [Mythos 5.1] achieving subexpert performance in Sunlight and LMCA .” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	red teaming of sandboxed exploit-reproduction tasks Cyber	“Trajectory Labs, PBC spent roughly 74 hours red teaming Fable 5.1, sending over 6,500 requests against sandboxed tasks that each require reproducing an exploit for a publicly documented vulnerability.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	automated and manual prompt red teaming Cyber	“10a Labs ran over 6,700 automated and manual prompts against Fable 5.1.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	Shade attacker on CVEs task set Cyber	“CVEs: 24 scenarios that contain a single code ﬁle with a known vulnerability.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	Shade attacker knowledge elicitation through chat Misuse and jailbreaks	“Knowledge elicitation through chat: Conversations where the model is asked to provide information for 100 harmful and 100 dual-use queries.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	Shade Prompt injection and agents	“Fable 5.1 performs comparably to Fable 5 in coding environments, with an attack success rate of 9.26% over all attempts with extended thinking, compared to 9.70% for Fable 5.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	new Shade attacker in coding environments Prompt injection and agents	“Without safeguards, Fable 5.1’s attack success rate is 56.87%, compared to 86.87% for Fable 5 and 90.93% for Opus 5, with Sonnet 5 as the most robust at 19.47% with thinking.” source
Claude Mythos 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	external testing Deception and sabotage	“During external testing, a partner observed isolated cases of ﬁnal and near-ﬁnal snapshots of Claude Fable 5.1 models reading ﬁles outside its sandbox environment, which we report in Section 6.2.2 .” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	external testing by AI safety researchers at METR Autonomy and AI research	“External testing by AI safety researchers at METR produced ﬁndings consistent with this assessment.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	beneﬁcial red teaming tabletop exercise Chemical, biological, radiological, nuclear	“This evaluation paired ﬁve PhD-level biologists with dedicated LLM experts to develop novel approaches for developing resistance to engineered pathogens in 16 hours, to test whether composite teams 17 can match world-leading specialists.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	black-box rna sequence modeling and design challenge Chemical, biological, radiological, nuclear	“We partnered with Dyno Therapeutics on two sequence-to-function evaluations: a black-box RNA sequence modeling and design challenge, benchmarked against 57 human participants, drawn from the leading edge of the US ML-bio labor market; and an AAV capsid packaging prediction task measuring whether a model’s domain knowledge and machine learning capabilities can outperform pretrained protein language models.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	beneficial red teaming tabletop exercise Chemical, biological, radiological, nuclear	“The beneﬁcial red teaming tabletop exercise paired ﬁve PhD-level biologists with dedicated LLM experts to assess whether composite teams making use of advanced model capabilities could develop comprehensive scientiﬁc research plans and protocols.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	black-box rna sequence design Chemical, biological, radiological, nuclear	“We partnered with Dyno Therapeutics on two evaluations of sequence-to-function modeling and design capability: 1. Black-box RNA sequence design: a medium-horizon challenge on which Dyno has evaluated 57 human participants drawn from the leading edge of the US ML-bio labor market since 2018.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	aav capsid packaging prediction Chemical, biological, radiological, nuclear	“2. AAV capsid packaging prediction: Adeno-associated viruses (AAVs) are a category of non-pathogenic viruses that are frequently used as a delivery mechanism for gene therapy to deliver a DNA payload within the viral capsid (the outer protein shell of the virus).” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	Sunlight Autonomy and AI research	“we still observed [Mythos 5.1] achieving subexpert performance in Sunlight and LMCA .” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	Budget NanoGPT Speedrun Autonomy and AI research	“Its performance on Budget NanoGPT Speedrun was particularly impressive, being markedly better than Claude Fable 5’s and Claude Opus 5’s (though we think it is plausible that the models’ training was speciﬁcally optimized for tasks similar to Budget NanoGPT Speedrun).” source
Claude Fable 5.1 Anthropic	Trajectory Labs Independent lab · 2026-09-01	sandboxed tasks that each require reproducing an exploit for a publicly documented vulnerability Cyber	“Trajectory Labs, PBC spent roughly 74 hours red teaming Fable 5.1, sending over 6,500 requests against sandboxed tasks that each require reproducing an exploit for a publicly documented vulnerability.” source
Claude Fable 5.1 Anthropic	10a Labs Independent lab · 2026-09-01	automated and manual prompts Cyber	“10a Labs ran over 6,700 automated and manual prompts against Fable 5.1.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	cves Cyber	“CVEs: 24 scenarios that contain a single code ﬁle with a known vulnerability.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	knowledge elicitation through chat Misuse and jailbreaks	“Knowledge elicitation through chat: Conversations where the model is asked to provide information for 100 harmful and 100 dual-use queries.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	indirect prompt injection (IPI) benchmark Prompt injection and agents	“Fable 5.1 is our most robust model to date on this benchmark, with an attack success rate of 0.1% at k=1, 0.7% at k=10, and 1.0% at k=15, improving on Opus 5 (0.4%, 3.6%, 4.8%) and Claude Fable 5 (0.6%, 4.9%, 6.5%).” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	Shade indirect prompt injection attacks in coding environments Prompt injection and agents	“Fable 5.1 performs comparably to Fable 5 in coding environments, with an attack success rate of 9.26% over all attempts with extended thinking, compared to 9.70% for Fable 5.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	new Shade attacker in coding environments with prompt injection probes enabled Prompt injection and agents	“Prompt injection probes reduce attack success rates substantially across models, bringing Fable 5.1 to 12.80%, the lowest of any model with safeguards enabled.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	external partner sandbox evaluation Deception and sabotage	“While one of our partners was evaluating the ﬁnal and a near-ﬁnal snapshot of Claude Fable 5.1, they set up an evaluation environment where the model was in a sandbox, but it could submit code to a compiler outside of the sandbox.” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	frontierswe v2 Other	“Claude Fable 5.1 scored 0.57 on FrontierSWE v2, the highest of the models Proximal evaluated, ahead of Claude Opus 5 (0.52), Claude Fable 5 (0.48) and GPT-5.6 Sol (0.32).” source
Claude Fable 5.1 Anthropic	Independent lab Independent lab · 2026-09-01	cursorbench 3.2.0 Other	“Fable 5.1 scored a state-of-the-art 73.4% on CursorBench at max effort .” source
Opus 4 Anthropic	Transluce Independent lab · 2026-08-31	mental health evaluation Other	“Transluce carried out the most expansive independent evaluation to date of how leading AI models respond to users experiencing mental health crises, including suicidal ideation, psychosis, and mania.” source
Opus 4 Anthropic	Transluce Independent lab · 2026-08-31	simulated multi-turn conversations between users in crisis and model variants, scored by automated judges on 14 mental-health-related behaviors Harmful content	“The newest models also reinforced delusions and mania much less than earlier versions, in approximately 2-36% of our simulated conversations, depending on the model, versus for example 69%-82% among GPT-4o, Opus 4, and Gemini 2.5.” source
Opus 4 Anthropic	Transluce Independent lab · 2026-08-31	testing chatgpt and claude models in the browser-based chatbot app Harmful content	“We were able to test a series of ChatGPT and Claude models in the browser-based chatbot app (although this did not include older models like GPT-4o or Opus 4 which were no longer available there), as well as a nonpublic Google DeepMind API that is intended to match the Gemini app experience.” source
Claude Sonnet 4.5 Anthropic	Zenity Labs Security firm · 2026-08-27	DNS exfiltration via indirect prompt injection from a Google Drive document Prompt injection and agents	“Note: This was tested with Claude Sonnet 4.5, which was Anthropic's latest model at the time.” source
Claude Sonnet 4.5 Anthropic	Zenity Labs Security firm · 2026-08-27	Bidirectional DNS command shell via pickle deserialization in Claude's sandbox Prompt injection and agents	“I decided to feed Claude with a .pkl file, because this is the file format Python uses for storing serialized objects, and I was surprised to discover that the model went ahead and ran pickle.load() without any static analysis or a safety check.” source
GPT-5.6 Sol OpenAI	Redwood Research Independent lab · 2026-08-26	ExploitGym Cyber	“On July 8th, OpenAI started ExploitGym experiments that launched tens of thousands of agents across multiple different models including GPT-5.6 Sol and a highly-persistent internal model, 6 which we will refer to as “HPIM” going forward.” source
GPT-5.6 Sol OpenAI	Redwood Research Independent lab · 2026-08-26	ExploitGym Other	“OpenAI defined the investigation period as June 26th, 2026 through July 13th, 2026, spanning both precursor activity before the incident and the incident itself, which began with the ExploitGym runs starting on July 7th.” source
GPT-5.6 Sol OpenAI	Redwood Research Independent lab · 2026-08-26	classifier analysis of agent transcripts for motivations for joining the Hugging Face attack Other	“We instructed analysis agents to analyze the transcripts of the 100 agents that started participating in the attack during the peak hour for joining (around 8-9 AM UTC on July 11th).” source
GPT-5.6 Sol OpenAI	Redwood Research Independent lab · 2026-08-26	classifiers over 10-30 transcripts to determine whether agents understood they were doing something unintended Other	“We ran classifiers over 10-30 transcripts 98 to determine whether agents understood they were doing something unintended.” source
GPT-5.6 Sol OpenAI	Redwood Research Independent lab · 2026-08-26	classifier sweep over all transcripts for AIs considering or attempting to notify humans Other	“We ran a classifier sweep over all of our transcripts to look for AIs either considering or t” source

Previous · Page 11 of 48 · Next