{"schema_version":"0.10.0","id":"LR-2026-0833","record_version":1,"title":"UMT5 decoder causal-mask regression allowed attending to future tokens in Transformers","verification":{"human_verified":false,"method":"automated_gate","auto_verified":true,"score":51.8,"score_breakdown":{"source_authority":12.1,"independent_corroboration":7.0,"implicated_party_confirmed":0.0,"citation_integrity":14.0,"specificity":6.0,"completeness":5.12,"unhedged_language":5.0},"checks":{"schema_valid":true,"every_field_cited":true,"quotes_verbatim":true,"source_fetched":true,"distinct_incident":true,"dates_plausible":true,"sources_agree":true,"source_describes_incident":true,"severe_backed_by_source":true,"record_integrity":true,"eligible":true,"about_this_case":true},"gate_version":"gate-2026-10-18.7","cross_check_model":null,"cross_check_agreement":null,"duplicate_of":null,"verified_by":null,"verified_at":"2026-11-04","extraction_run_id":null,"evidence_level":{"level":"confirmed","reason":"Hugging Face's own report","original_document":{"url":"https://huggingface.co/blog/transformers-mic","organisation":"Hugging Face","role":"company"},"reports_total":1,"reports_independent":1,"reports_derivative":0,"developing":true,"disputed":false,"rules_version":"evidence-2026-10-29.1","computed_at":"2026-11-04","history":[{"date":"2026-11-04","from_level":null,"to_level":"confirmed","reason":"Hugging Face's own report"}]},"sources_hash":"a6f54d325d31634f891659b5f3c8f6d5a46072f111d8835330333b2acd151981"},"system":{"model_name":"UMT5","model_version":null,"developer":"unknown","harness":"Transformers","tools_granted":[],"permissions_scope":null,"autonomy_level":"unknown","guardrails_present":[],"deployment_context":"production","tier":null,"lifecycle_stage":"production","integration_type":null,"intended_use":null,"safeguard_state":null,"production_intended":null},"event":{"date_occurred":"2026-07-30","date_disclosed":"2026-11-03","incident_class":"performance_drift","failure_class":null,"agent_involved":null,"secondary_failure_classes":[],"trigger":"unknown","duration":"P60D","detection_method":"automated_monitor","summary":"Transformers enabled PyTorch's scaled dot-product attention (SDPA) for T5, and as a side effect UMT5's decoder could attend to future tokens, so during training it could peek at the answer it was supposed to predict. Nothing crashed, and the bug stayed in main for almost two months. It was surfaced when the team replayed frozen Model Integration Contracts over five months of Transformers commits: a small UMT5 fixture running on CPU flagged the regression on every affected commit, with label log-probabilities differing from the reviewed baseline by 1.22 and training loss by about 0.20. The contract used SDPA, the default attention path; running only the eager attention variant missed the bug. The regression was fixed on September 28, 2026, and reverting the fix on a proof branch reproduced the same differences.","tags":["attention-mask-regression","training-data-leakage","model-integration-contract","ci-regression-detection","transformers-library"],"date_detected":null,"near_miss":null,"root_cause":"integration","root_cause_detail":"Enabling SDPA for T5 in Transformers changed the attention path so that UMT5's decoder could attend to future tokens.","failure_class_detail":null,"responsible_party":"provider","adversarial_actor":null,"input_provenance":null,"time_to_contain":null,"operations_paused":null,"remediation":null,"precursor_events":null},"impact":{"direct_loss_usd":null,"remediation_hours":null,"records_exposed":null,"third_parties_affected_count":null,"severity":"minor","severity_because":"The regression was contained to the library's main branch with no crash and no third party affected, meeting the minor rung.","harm_class":[],"affected_description":null,"protected_class_involved":null,"data_categories_exposed":[],"loss_range_usd":null,"reversibility":"reversible","downstream_exposure_count":null},"evidence":{"sources":[{"url":"https://huggingface.co/blog/transformers-mic","publisher":"Hugging Face","date":"2026-11-03","type":"press","title":"Transformers Model Integration Contracts","archived_url":null,"attachment":"first_report","independence":"independent"}],"per_field_confidence":{"system.model_name":"C","system.developer":"D","system.harness":"C","system.autonomy_level":"D","system.deployment_context":"C","system.lifecycle_stage":"C","event.date_occurred":"C","event.date_disclosed":"D","event.trigger":"D","event.duration":"C","event.detection_method":"C","event.summary":"C","event.tags":"C","event.root_cause":"C","event.root_cause_detail":"C","event.responsible_party":"C","event.incident_class":"C","impact.severity":"C","impact.reversibility":"C","impact.severity_because":"C","context.country":"D","context.denominator_available":"D"},"field_citations":{"system.model_name":[{"source_index":0,"quote":"As a side effect, UMT5's decoder could attend to future tokens."}],"system.harness":[{"source_index":0,"quote":"On July 30, 2026, Transformers enabled PyTorch's scaled dot-product attention (SDPA) for T5."}],"system.deployment_context":[{"source_index":0,"quote":"Nothing crashed, and the bug stayed in main for almost two months."}],"system.lifecycle_stage":[{"source_index":0,"quote":"Nothing crashed, and the bug stayed in main for almost two months."}],"event.date_occurred":[{"source_index":0,"quote":"On July 30, 2026, Transformers enabled PyTorch's scaled dot-product attention (SDPA) for T5."}],"event.duration":[{"source_index":0,"quote":"Nothing crashed, and the bug stayed in main for almost two months."},{"source_index":0,"quote":"The small UMT5 fixture flagged the regression from its introduction on July 30, 2026 until its fix on September 28."}],"event.detection_method":[{"source_index":0,"quote":"When we later replayed those commits, a small UMT5 model running on CPU flagged it on every one of them."}],"event.summary":[{"source_index":0,"quote":"On July 30, 2026, Transformers enabled PyTorch's scaled dot-product attention (SDPA) for T5. As a side effect, UMT5's decoder could attend to future tokens."},{"source_index":0,"quote":"During training, it could peek at the answer it was supposed to predict. Nothing crashed, and the bug stayed in main for almost two months."},{"source_index":0,"quote":"When we later replayed those commits, a small UMT5 model running on CPU flagged it on every one of them."},{"source_index":0,"quote":"The small UMT5 fixture flagged the regression from its introduction on July 30, 2026 until its fix on September 28. Label log-probabilities differed from the reviewed baseline by 1.22 and training loss by about 0.20."},{"source_index":0,"quote":"One detail mattered: the contract used SDPA, the default attention path. Running only the eager attention variant missed the bug."}],"event.tags":[{"source_index":0,"quote":"As a side effect, UMT5's decoder could attend to future tokens."},{"source_index":0,"quote":"During training, it could peek at the answer it was supposed to predict."},{"source_index":0,"quote":"The small UMT5 fixture flagged the regression from its introduction on July 30, 2026 until its fix on September 28."}],"event.root_cause":[{"source_index":0,"quote":"On July 30, 2026, Transformers enabled PyTorch's scaled dot-product attention (SDPA) for T5. As a side effect, UMT5's decoder could attend to future tokens."}],"event.root_cause_detail":[{"source_index":0,"quote":"On July 30, 2026, Transformers enabled PyTorch's scaled dot-product attention (SDPA) for T5. As a side effect, UMT5's decoder could attend to future tokens."}],"event.responsible_party":[{"source_index":0,"quote":"On July 30, 2026, Transformers enabled PyTorch's scaled dot-product attention (SDPA) for T5."}],"event.incident_class":[{"source_index":0,"quote":"During training, it could peek at the answer it was supposed to predict. Nothing crashed, and the bug stayed in main for almost two months."}],"impact.severity":[{"source_index":0,"quote":"Nothing crashed, and the bug stayed in main for almost two months."}],"impact.reversibility":[{"source_index":0,"quote":"The small UMT5 fixture flagged the regression from its introduction on July 30, 2026 until its fix on September 28."}],"impact.severity_because":[{"source_index":0,"quote":"Nothing crashed, and the bug stayed in main for almost two months."}]},"notes_on_gaps":"The document is a technical blog post about a test-infrastructure project (transformers-mic) and describes one concrete regression: the UMT5 SDPA causal-mask bug. No monetary loss, record counts, affected third parties, or responsible individuals are stated. date_disclosed is taken from the collector-supplied publication date (2026-11-03), not from the document text. duration is computed from the stated introduction (July 30, 2026) and fix (September 28, 2026) dates, which the document itself gives; the document also says 'almost two months'. detection_method is coded automated_monitor because the fixture flagged the regression during replay, though the document does not describe a live monitoring system. The document also mentions a Whisper assisted-decoding bug and synthetic fault-injection results, but these are research/evaluation findings without a dated concrete incident, so they are not recorded as separate incidents. The LLM-drafted usage code is described as human-reviewed and is not an incident. Extractor proposed values without quotes; dropped: ['context.country', 'context.industry', 'event.adversarial_actor', 'event.agent_involved', 'event.date_disclosed', 'event.trigger', 'system.autonomy_level', 'system.developer']. Single secondary source; corroborate before upgrading any field to B. [2026-11-04: evidence level set: Confirmed (Hugging Face's own report)]"},"context":{"industry":null,"org_size_band":null,"country":"unknown","denominator_available":false,"denominator":null,"related_incident_ids":[],"systemic_exposure":null,"shared_model_id":null,"upstream_incident_id":null,"incident_evidence":null,"regulatory":null,"coverage":null}}