Claude Opus 4.8

claude opus 4.8 is moderately more capable than claude opus 4.7 on autonomy-relevant evaluations but remains less capable than claude mythos preview, and our alignment assessment finds that its capability for covert or monitor-subverting behavior is low and broadly similar to that of claude opus 4.7, while its overall behavioral alignment profile is generally improvement on opus 4.7.

in evaluations across the domains of harmful requests, mental health, child safety, and bias & integrity, opus 4.8 generally performs as well as, or better than, opus 4.7.

on the prediction task, 23 opus 4.8 exceeded both the first benchmark and the 90th-percentile human score and improved over all other models.

in the black-box rna sequence modeling and design evaluation described in section 2.2.5.1 , opus 4.8 performs better than mythos preview at the overall prediction task of ranking held-out test sequences by predicted score (both when starting from scratch and when iterating on earlier mythos preview attempts).

the final results show that claude opus 4.8 achieved the same attack success rate as claude opus 4.7, whereas the preliminary results had shown opus 4.7 as more robust.

claude opus 4.8 often struggles to follow explicit instructions in its extended thinking, giving it among the lowest chain-of-thought (cot) controllability scores of the models we tested, comparable to claude sonnet 4.5.

averaged across categories, cot controllability peaked with claude opus 4.6 and claude mythos preview and has declined in our most recent models, with claude opus 4.8 one of the lowest.

however, claude opus 4.8 fails to raise the important events to the user only 3.7% of the time, down 5-fold from mythos preview, which misleads the user 27.6% of the time in this scenario, and down almost as much from opus 4.7.

in a continuation setting, where synthetically inserted prior reasoning and actions begin an attempt to compromise ai safety research, [opus 4.8] actively continued the attempt 1% of the time, a slight increase on opus 4.7 (0%) likely within noise, and well below mythos preview (7%).

opus 4.8 designed fragments that both successfully assembled plasmids and evaded synthesis screening protocols for seven out of ten pathogens.

claude opus 4.8 performed near-perfectly on the single-turn election integrity benchmark, in line with claude opus 4.7, reliably declining violative requests without over-refusing on legitimate election-related prompts.

this section also includes results from some fine-tuning-based elicitation experiments on claude opus 4.8, which did not show significant increases in stealth success rates.

all of these shortcomings came from manually flagged issues over a sample of ~5600 sessions using the final claude opus 4.8.

claude opus 4.8 does not outperform prior models.

we also ran the intervention-based monitorability evaluations from the same guan et al. suite, but the interventions produced behavioral effects on recent claude models that were too small for the resulting metrics to be informative, so we do not report them here.

on the two long-form virology tasks, opus 4.8 achieved end-to-end scores of 0.77 and 0.90, falling below the notable-capability benchmark on the first task but exceeding it on the second.

after revision, opus 4.8 improved from 0.89 to 0.90; this change does not affect our analysis of its cb-1 risks.

the helpful-only version of claude opus 4.8 showed a notably higher success rate than even claude mythos preview on both tasks, though we still assess opus 4.8 to require substantial human direction for many operational steps.

claude opus 4.8 scored worse than recent models on this evaluation.

claude opus 4.8 improved compared to claude opus 4.7 on refusing malicious requests, while maintaining similar performance on dual-use and benign tasks.

on the api, claude opus 4.8 performed better than claude opus 4.7 in all domains, though generally within the margin of error.

multi-turn testing showed a significant quantitative improvement on claude.ai compared to the core api model.

as in our audits of other recent models, we shared a pre-release snapshot of claude opus 4.8 with uk aisi for open-ended testing, at their discretion, of behaviors or risk factors related to misalignment, going beyond their use of the fixed petri 3.0 benchmark reported above.

in both variants, we find that claude opus 4.8 outperforms all other tested models, achieving near-perfect scores, with more than a ten-fold improvement over opus 4.7.

claude opus 4.8 represents a clear improvement over opus 4.7 on overrefusals, ignoring explicit user-assigned constraints, and ignoring implicit constraints on how to safely pursue user requests.

as such, we added a new analysis focusing on the spearman correlation of the sequences above the 95th percentile of predicted score, labeled “prediction score (top)” in figure 2.2.5.1.a and figure 2.2.5.1.b.

performance on the rare-action evaluation shows a similar pattern as for prior models.

we find that claude opus 4.8 has the lowest refusal rate of any of our recent models, slightly improving over opus 4.6 (the previous best model on this evaluation).

claude opus 4.8 performs worse than mythos preview and claude opus 4.7 on the raw survival-rate metric in the secret-password evaluation, though when target model turns are grouped based on the presence of thinking or partial-turn prefill, claude opus 4.8 leaks the password at similar rates to claude opus 4.7 for all turn types.

safeguards further reduce the shade attacker’s per-attempt prompt injection success rates from 7.03% to 2.0% with thinking and 17.44% to 4.11% without.

claude opus 4.8 maintained a high rate of refusals on requests posing potential risk while rarely over-refusing on benign requests; both metrics were within the margin of error compared to claude opus 4.7 on api and claude.ai.

in the synthesis screening evasion evaluation from our automated cb-1 testing shown in figure 2.2.4.a, opus 4.8 was graded as having evaded screening for 7/10 pathogens, whereas mythos preview did not evade screening for any such pathogens.

we find that claude opus 4.8 is the first model to achieve a perfect score on this evaluation—that is, it never reports false numbers.

in violent extremism testing, this showed up as claude opus 4.8 recognizing harmful trajectories earlier in multi-turn conversations than opus 4.7 and being less likely to accept a benign reframing at face value.

on the multimodal virology evaluation, opus 4.8 achieved a score of 0.47, relative to 0.5 for opus 4.7 and 0.574 for claude mythos preview, placing all models above the benchmark of notable capability.