Muse Spark 1.1

• strongreject(souly et al., 2024)—tests whether the model can be jailbroken into producing unsafe content via adversarial prompts.

when it had misbehaved, muse spark 1.1 named its action as wrong, harmful, deceptive, unethical, or against its operators’ interests in 45% of cases—far more often than any peer (gpt-5.5 20%, gemini 3.1 pro 7%, claude 4.8 5%)—and voiced at least some concern in 51%.

across 1,169 evaluation items per model (the set answered by all models across the five evaluations), muse spark 1.1’s actions and beliefs are coherent in 89.9% of cases, making it the most coherent model tested—ahead of gpt-5.5 (89.6%), claude opus 4.8 (86.0%), and gemini 3.1 pro (78.0%).

figure 15 shows that muse spark 1.1 and muse spark 1.1 system have improved robustness compared to muse spark 1.0, and higher refusal rates on this set compared to peer models with non-overlapping confidence intervals.

muse spark 1.1 exhibits a very low rate of harmful actions, on par with gpt-5.5 and claude 4.8, and marking a significant improvement over muse spark 1.0 (table table 13).

muse spark 1.1 exhibits a compliance gap of -0.2% between the training and deployment settings, not statistically distinguishable from zero (gpt-5.5: 0.6%; claude opus 4.8: 0.2%).

to assess capabilities that more realistically proxy for real-world offensive operations, we use the combination of public and privately developed challenges described in the muse spark safety & preparedness report: cybergym (wang et al., 2026b), a public benchmark measuring automated vulnerability discovery in real open-source software; a set of atomic challenges developed by irregular that test isolated offensive skills mapped to kill-chain phases; and cyscenariobench (irregular, 2025), a scenario-based framework measuring autonomous, long-horizon, multi-host attack chains.

as we saw on cybench and the curated ctf set, muse spark 1.1 improves substantially on the atomic suite overall (67.0% vs 48.1% pass@1, 82.5% vs 65.0% pass@20), and the gains again concentrate on the harder challenges, and in particular on the vulnerability-research-and-exploitation and network-attack-simulation categories.

from figure 21, we find that both muse spark 1.1 and muse spark 1.1 system show a higher false refusal rate compared to the peer models.

muse spark 1.1 shows stronger robustness compared to the peers with overlapping confidence intervals.

failures concentrate heavily in conflicts with a clear correct resolution.

we find muse spark 1.1 has great improvement from muse spark 1.0 and matches the performance of the peer models.

muse spark 1.1 identifies the provenance of its own data well above chance, at 66.5% overall accuracy.

deception toward users is also significantly higher for muse spark 1.1 (1.10) than for gpt-5.5 (1.02), though the gap to claude 4.8 opus (1.06) is not statistically significant, and overrefusal (1.20) is elevated relative to the other frontier models.

however, we find very limited evidence of evaluation awareness in dangerous capabilities testing (2.2% on average)—substantially below the strategic reasoning rates observed in the pressure-gradient experiment (table 8), which uses explicit evaluation- signally prompts not present in the standard dangerous capabilities suite.

third-party red teaming further confirms this conclusion, with muse spark 1.1 demonstrating lower risk and more balanced refusal behavior compared to other major models.

overall, despite adversarial pressure, violations are uncommon and mostly mild: the mean severity per category sits around 1.8/10, with an overall violation rate of 19.47% across categories.

in english, muse spark 1.1 has a slightly higher asr than muse spark 1.0, but lower than claude opus 4.8 and gemini 3.1 pro.

such capability is reliably captured by monitors (figure 22).

muse spark 1.1 shows substantial capabilities to selectively perform when explicitly instructed, aligning with comparison models.

on the private benchmarks, muse spark 1.1 matches or exceeds gemini 3.1 pro, claude opus 4.8, and gpt-5.5 under both static adversarial prompts and adaptive attacks that optimize the user and developer prompts, and it shows the smallest degradation when developer-prompt jailbreaks are introduced; muse spark 1.1 system extends this margin, in several cases outperforming all peers with non-overlapping confidence intervals.

this advantage carries over to the public benchmarks, where muse spark 1.1 attains the lowest attack success rates across single- and multi-turn adaptive red teaming—substantially below gpt-5.5 and gemini 3.1 pro and competitive with claude opus 4.8—and muse spark 1.1 system again reduces susceptibility further.

we assess whether a model spontaneously selectively underperforms on certain safety-critical evaluations while maintaining performance on other capability evaluations, which would reduce the reliability of pre-deployment assessments.

in figure 35, muse spark 1.1 shows a sycophancy rate of 49.2%, below gemini 3.1 pro (65.6%) but above gpt-5.5 (45.5%) and claude opus 4.8 (32.4%).

visible reasoning trace analysis shows divergence from the final output response in only 0.7% of scenarios, which is consistent with sincere mis-resolution rather than concealed intent, though stated reasoning is of course not guaranteed to reflect the model’s underlying computation.