AI Risk Wire

Cases › produced harmful content › LR-2026-0751

Anthropic reports blocking scientists' attempts to use Claude in ways that could support biological weapons development

Preliminary Why: 1 report: CBS News · subject to change · 1 independent report

Status: fix reported

Severity minor
Reported 10 Sep 2026
Developer and model Anthropic · Claude

No source states: loss or records, agent tool, could do, deployer, industry.

Severity minor: test: contained to the provider's own platform with accounts banned; the source states no third party was touched and no harm materialised.

Anthropic reported that scientists used its Claude AI models in ways that could support biological weapons development and that it blocked them. The cases were part of a threat report covering December 2025 to August 2026, and five involved activity with potential to support biological weapons. The people involved are working scientists, and the company said it cannot assert that they intended harm. When detected and investigated, the users' accounts were banned and findings were fed into safeguards, enforcement and threat intelligence. SFist reported that in one state-linked case the same actors re-established accounts and continued research within days. The company said the biological capability was shown but not that it would be used for real weapons. Other misuse in the report, such as influence operations and surveillance, is not part of this record.

What happened

1 · What it was doingScientists used Claude AI models in ways that could support biological weapons development, according to Anthropic.
2 · What went wrongAnthropic identified five cases of activity with potential to support biological weapons development.
3 · How it was caughtAnthropic detected and investigated the cases and banned the users' accounts.

What was done

Anthropic fed its investigative findings into safeguards, enforcement and threat intelligence processes.
Fix reported

“Banned the users' accounts and incorporated investigative findings into frontier model safeguards, enforcement and threat intelligence processes”

Same developer, same risk, after the fix: LR-2026-0080

Risks

Harmful content and jailbreaks
Primary
84 incidents
Rising: 6 vs 3.8 a week

Sorted by rule from the case's fields; not yet read from the sources.

Controls

Model

Effective Abuse monitoring and account enforcement

Control failure chain

L1 · Model EffectiveAnthropic
L2 · Harness Not stated in sourcesHarness not named
L3 · Application Not stated in sourcesDeployer not named
L4 · Infrastructure / target Not stated in sourcesOrganisation hit

L1 · ModelAnthropicModel provider

EffectiveAbuse monitoring and account enforcement“When we detected and investigated these cases, we banned the users' accounts and incorporated our investigative findings into our frontier model safeguards, enforcement, and threat intelligence processes to better prevent, detect, and disrupt these activities in the future," the report said.”CBS News

Model version not linked to a model card.

Other controls for this risk at this layer, not mentioned in the sources: Safety training and refusal, Content safety classifiers, Outside red-teaming.

L2 · HarnessHarness not namedAgent framework vendor

No source mentions a control at this layer.

L3 · ApplicationDeployer not namedDeployer

No source mentions a control at this layer.

Other controls for this risk at this layer, not mentioned in the sources: Output filtering and redaction.

L4 · Infrastructure / targetOrganisation hit

No source mentions a control at this layer.

EffectiveNot effectiveNot effective (bypassed)Did not existNot stated in sourcesNot yet checkedAttacker's ownOrder: model · harness · application · target

Controls for this risk: result in this incident

ControlPart of the systemResult
Abuse monitoring and account enforcementL1 ModelEffective
Content safety classifiersL1 ModelNot stated in sources
Safety training and refusalL1 ModelNot stated in sources
Outside red-teamingL1 ModelNot stated in sources
Output filtering and redactionL3 ApplicationNot stated in sources

Similar incidents

All cases of this kind ›

Evidence

No source states: what it reached or did, when it happened, when it was found, how many records were exposed, the direct loss, a count of the other parties affected, any court case, the cause, what set it off.

Show the full account: 3 items from the primary sources

What happened, in order

  1. December 2025 to August 2026 The cases occurred in this period.

Not dated in the sources:

  • After one activity was intercepted, the same actors re-established accounts and continued the research within days.
  • Anthropic detected and investigated the cases, banned the users' accounts and updated its safeguards and enforcement processes.
Show every field, with its source and grade

The AI system

Model
Claude
C one secondary source states it
Developer
Anthropic
C one secondary source states it

What happened

Earliest source we hold
2026-09-10
D inferred by us from the described behaviour, reason shown
What was done
Banned the users' accounts and incorporated investigative findings into frontier model safeguards, enforcement and threat intelligence processes
C one secondary source states it
What went wrong
produced harmful content
D inferred by us from the described behaviour, reason shown
control chain
layer: L1; party: provider; control id: D7; outcome: held; quote: When we detected and investigated these cases, we banned the users' accounts and incorporated our investigative findings into our frontier model safeguards, enforcement, and threat intelligence processes to better prevent, detect, and disrupt these activities in the future," the report said.; source index: 0
C one secondary source states it

Impact

Severity
minor
D inferred by us from the described behaviour, reason shown
Why this severity
test: contained to the provider's own platform with accounts banned; the source states no third party was touched and no harm materialised.
D inferred by us from the described behaviour, reason shown

Notes on gaps

The Claude Fable 5 safeguards remediation concerns a different, newer model and is not confirmed as a fix for these cases. Other misuse in the report (influence operations, surveillance) is excluded because it is not part of this bio-misuse incident.

LR-2026-0751 · version 1 · published 2026-11-01 after automatic checks; not yet checked field by field by a person · JSON