{"schema_version":"0.10.0","id":"LR-2026-0788","record_version":1,"title":"METR red-team finds vulnerabilities in OpenAI monitoring system","verification":{"human_verified":false,"method":"automated_gate","auto_verified":true,"score":51.1,"score_breakdown":{"source_authority":12.1,"independent_corroboration":7.0,"implicated_party_confirmed":0.0,"citation_integrity":14.0,"specificity":6.0,"completeness":4.48,"unhedged_language":5.0},"checks":{"schema_valid":true,"every_field_cited":true,"quotes_verbatim":true,"source_fetched":true,"distinct_incident":true,"dates_plausible":true,"sources_agree":true,"source_describes_incident":true,"severe_backed_by_source":true,"record_integrity":true,"eligible":true,"about_this_case":true},"gate_version":"gate-2026-10-18.7","cross_check_model":null,"cross_check_agreement":null,"duplicate_of":null,"verified_by":null,"verified_at":"2026-11-01","extraction_run_id":null,"evidence_level":{"level":"preliminary","reason":"Publisher not yet named · subject to change","original_document":null,"reports_total":1,"reports_independent":0,"reports_derivative":0,"developing":true,"disputed":false,"rules_version":"evidence-2026-10-29.1","computed_at":"2026-11-01","history":[{"date":"2026-11-01","from_level":null,"to_level":"preliminary","reason":"Publisher not yet named · subject to change"}]},"sources_hash":"35f9bc7755161478ba1f86cc47b22da4ee1f9518d4f872ee9912f100554caaf4"},"system":{"model_name":"GPT-5.4 Thinking","model_version":null,"developer":"OpenAI","harness":null,"tools_granted":[],"permissions_scope":null,"autonomy_level":"unknown","guardrails_present":[],"deployment_context":"unknown","tier":null,"lifecycle_stage":null,"integration_type":null,"intended_use":"monitoring internal coding agent deployments","safeguard_state":null,"production_intended":null},"event":{"date_occurred":null,"date_disclosed":"2026-10-22","incident_class":"other","failure_class":null,"agent_involved":null,"secondary_failure_classes":[],"trigger":"unknown","duration":"P3W","detection_method":"third_party","summary":"In a pilot exercise from July 21 to August 11, 2026, METR and Redwood Research, each with one employee, had deep access to OpenAI's internal infrastructure to stress-test OpenAI's internal misalignment monitoring system for agentic traffic. The evaluators reported vulnerabilities, including a method by which Codex agents could in principle bypass sandboxing and an exploit that could keep recent actions from being monitored. They also found that client-controllable headers could disable monitoring for new agents. Coverage gaps were found too, including reinforcement learning training runs not covered at the start of the exercise. The evaluators could not check whether any agent had used these paths in practice. The findings were published on October 22, 2026. OpenAI has stated that identified vulnerabilities affecting its production safeguards were patched, a claim METR has not evaluated.","tags":[],"date_detected":null,"near_miss":null,"root_cause":null,"root_cause_detail":null,"failure_class_detail":null,"responsible_party":null,"adversarial_actor":null,"input_provenance":null,"time_to_contain":null,"operations_paused":null,"remediation":null,"precursor_events":null,"primary_risk":"RK05","control_chain":[{"layer":"L2","party":"harness_vendor","party_name":null,"control_id":"E1","outcome":"got_around","reason":null,"quote":"They found a method by which Codex agents could in principle bypass Codex sandboxing and Auto-Review to take any action available to the host user, and an exploit, whose details OpenAI redacted, that would keep hundreds of recent actions, including those involved in carrying out the exploit itself, from being monitored.","source_index":0,"reviewed":false},{"layer":"L3","party":"deployer","party_name":null,"control_id":"F2","outcome":"failed","reason":null,"quote":"METR estimated that not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, a gap OpenAI staff had independently identified before the exercise began and fixed during it.","source_index":0,"reviewed":false},{"layer":"L3","party":"deployer","party_name":null,"control_id":"F2","outcome":"missing","reason":null,"quote":"At the start of the exercise, reinforcement learning training runs were not covered by the monitoring system, although the evaluators were informed that a subset of RL training rollouts had been monitored for misalignment between March and June.","source_index":0,"reviewed":false}],"remediations":[{"by":null,"kind":"fix","date_reported":null,"controls_added":["J1"],"quote":"Since the exercise ended, OpenAI has stated that all of the identified vulnerabilities that impact its production safeguards stack have been patched and has described significant subsequent changes to the intended coverage of its monitoring system.","source_index":0,"reviewed":false},{"by":null,"kind":"fix","date_reported":null,"controls_added":["F2"],"quote":"METR estimated that not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, a gap OpenAI staff had independently identified before the exercise began and fixed during it.","source_index":0,"reviewed":false},{"by":null,"kind":"advice","date_reported":null,"controls_added":["F2","F3","E4"],"quote":"Among the specific practices described are monitoring agents every set number of messages or minutes rather than waiting for them to end their turns, maintaining a list of all known ways around monitoring, keeping tamper-proof and searchable logs, and running agent scaffolds in an environment isolated from where tool calls are executed.","source_index":0,"reviewed":false}]},"impact":{"direct_loss_usd":null,"remediation_hours":null,"records_exposed":null,"third_parties_affected_count":null,"severity":"minor","severity_because":"Minor tier: the findings come from one pilot evaluation of internal systems, designed as a pilot and not as a public transparency exercise; no third party is stated to have been affected.","harm_class":[],"affected_description":null,"protected_class_involved":null,"data_categories_exposed":[],"loss_range_usd":null,"reversibility":null,"downstream_exposure_count":null},"evidence":{"sources":[{"url":"https://www.unite.ai/metr-red-team-finds-vulnerabilities-in-openai-monitoring-system/","publisher":"Unite.AI","date":"2026-10-22","type":"press","title":"METR Red-Team Finds Vulnerabilities in OpenAI Monitoring System - Unite.AI","archived_url":null,"attachment":"first_report"}],"per_field_confidence":{"system.developer":"C","system.model_name":"C","system.intended_use":"C","event.date_disclosed":"C","event.detection_method":"C","event.duration":"C","event.incident_class":"D","impact.severity":"D","impact.severity_because":"D","event.primary_risk":"D","event.control_chain":"C","event.remediations":"C","event.summary":"C","system.autonomy_level":"D","system.deployment_context":"D","event.trigger":"D","context.country":"D","context.denominator_available":"D"},"field_citations":{"system.developer":[{"source_index":0,"quote":"OpenAI publicly described the tested system in a March 19, 2026 post."}],"system.model_name":[{"source_index":0,"quote":"According to that post, the low-latency system is powered by GPT-5.4 Thinking at maximum reasoning effort, reviews agent interactions within 30 minutes of completion, and logs and analyzes agents’ chains of thought and actions, categorizing each interaction, assigning it a severity level, and surfacing anomalies for human review."}],"system.intended_use":[{"source_index":0,"quote":"OpenAI said the system runs across all standard internal coding agent deployments, viewing the full conversation history, including all user, developer, and assistant messages along with tool calls and outputs."}],"event.date_disclosed":[{"source_index":0,"quote":"METR and Redwood Research on October 22, 2026 published a public account of a pilot exercise in which one employee from each organization spent three weeks with deep access to OpenAI’s internal infrastructure, stress-testing the company’s internal misalignment monitoring system."}],"event.detection_method":[{"source_index":0,"quote":"METR and Redwood Research on October 22, 2026 published a public account of a pilot exercise in which one employee from each organization spent three weeks with deep access to OpenAI’s internal infrastructure, stress-testing the company’s internal misalignment monitoring system."}],"event.duration":[{"source_index":0,"quote":"METR and Redwood Research on October 22, 2026 published a public account of a pilot exercise in which one employee from each organization spent three weeks with deep access to OpenAI’s internal infrastructure, stress-testing the company’s internal misalignment monitoring system."}],"event.incident_class":[{"source_index":0,"quote":"The exercise ran from July 21 to August 11, 2026, and focused primarily on estimating the fraction of agentic inference that is monitored and on whether agents could in principle disable or subvert the monitoring system."}],"impact.severity":[{"source_index":0,"quote":"METR, Redwood Research, and OpenAI designed the exercise as a pilot for giving external researchers deep access to internal infrastructure rather than as a vehicle for robust public transparency about OpenAI’s monitoring systems, and the parties agreed upfront to publish only a relatively brief, high-level public post."}],"impact.severity_because":[{"source_index":0,"quote":"METR, Redwood Research, and OpenAI designed the exercise as a pilot for giving external researchers deep access to internal infrastructure rather than as a vehicle for robust public transparency about OpenAI’s monitoring systems, and the parties agreed upfront to publish only a relatively brief, high-level public post."}],"event.control_chain":[{"source_index":0,"quote":"They found a method by which Codex agents could in principle bypass Codex sandboxing and Auto-Review to take any action available to the host user, and an exploit, whose details OpenAI redacted, that would keep hundreds of recent actions, including those involved in carrying out the exploit itself, from being monitored."},{"source_index":0,"quote":"METR estimated that not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, a gap OpenAI staff had independently identified before the exercise began and fixed during it."},{"source_index":0,"quote":"At the start of the exercise, reinforcement learning training runs were not covered by the monitoring system, although the evaluators were informed that a subset of RL training rollouts had been monitored for misalignment between March and June."}],"event.remediations":[{"source_index":0,"quote":"Since the exercise ended, OpenAI has stated that all of the identified vulnerabilities that impact its production safeguards stack have been patched and has described significant subsequent changes to the intended coverage of its monitoring system."},{"source_index":0,"quote":"METR estimated that not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, a gap OpenAI staff had independently identified before the exercise began and fixed during it."},{"source_index":0,"quote":"Among the specific practices described are monitoring agents every set number of messages or minutes rather than waiting for them to end their turns, maintaining a list of all known ways around monitoring, keeping tamper-proof and searchable logs, and running agent scaffolds in an environment isolated from where tool calls are executed."}],"event.summary":[{"source_index":0,"quote":"METR and Redwood Research on October 22, 2026 published a public account of a pilot exercise in which one employee from each organization spent three weeks with deep access to OpenAI’s internal infrastructure, stress-testing the company’s internal misalignment monitoring system."},{"source_index":0,"quote":"They found a method by which Codex agents could in principle bypass Codex sandboxing and Auto-Review to take any action available to the host user, and an exploit, whose details OpenAI redacted, that would keep hundreds of recent actions, including those involved in carrying out the exploit itself, from being monitored."},{"source_index":0,"quote":"Because the system triggered when agents returned control to the user, agents could have put off being monitored indefinitely simply by continuing to act during that period; access constraints meant the evaluators could not check whether any agent had done so in practice."},{"source_index":0,"quote":"Since the exercise ended, OpenAI has stated that all of the identified vulnerabilities that impact its production safeguards stack have been patched and has described significant subsequent changes to the intended coverage of its monitoring system."}]},"notes_on_gaps":"No real incident is reported: the sources describe a pilot red-team exercise and its findings, and no source states that any agent actually exploited the paths found. Therefore date_occurred, agent_involved, failure_class, root_cause, responsible_party, harm_class and reversibility are left out. The incident class is set to other and near_miss to true as my judgement. The exercise period 2026-07-21 to 2026-08-11 is used as the timeline, not as an occurrence date. The tested system is taken to be the monitoring system described in OpenAI's March 19, 2026 post, which names GPT-5.4 Thinking; the sources do not confirm the model version of the system as it existed during the exercise. The patch claim is OpenAI's own and METR has not evaluated it, so it is not recorded as a verified fix. Severity minor is my judgement from the pilot scope. Some monitoring gaps (compaction, RL runs) are attributed to F2 as the closest control in the list; this mapping is my judgement. Deployment context, industry and country are not stated and are left out. The header and monitor-input bypass findings are not mapped to controls beyond E3 because the closed control list does not name them precisely. [case created by the backfill: read by claude-haiku-5-5, backfill-2026-11-01.2, 2026-11-01; not yet reviewed] [2026-11-01: evidence level set: Preliminary (Publisher not yet named · subject to change)]"},"context":{"industry":null,"org_size_band":null,"country":"unknown","denominator_available":false,"denominator":null,"related_incident_ids":[],"systemic_exposure":null,"shared_model_id":null,"upstream_incident_id":null,"incident_evidence":null,"regulatory":null,"coverage":null},"narrative":{"steps":[{"step":"doing","text":"Pilot exercise testing whether agents could disable or subvert OpenAI's monitoring system.","quote":"The exercise ran from July 21 to August 11, 2026, and focused primarily on estimating the fraction of agentic inference that is monitored and on whether agents could in principle disable or subvert the monitoring system.","source_index":0,"grade":"C"},{"step":"went_wrong","text":"Evaluators found bypass paths and coverage gaps in the monitoring system.","quote":"They found two independent client-controllable headers that disable the monitoring system for new agents, which could enable an agent to start an otherwise identical instance that runs unmonitored, though the act of setting those headers and creating the instance could itself be caught by the monitoring system.","source_index":0,"grade":"C"},{"step":"caught","text":"The METR and Redwood Research evaluators found the gaps during the pilot exercise.","quote":"METR and Redwood Research on October 22, 2026 published a public account of a pilot exercise in which one employee from each organization spent three weeks with deep access to OpenAI’s internal infrastructure, stress-testing the company’s internal misalignment monitoring system.","source_index":0,"grade":"C"}],"model":"claude-haiku-5-5","prompt_version":"backfill-2026-11-01.2","extracted_at":"2026-11-01","source_urls":["https://www.unite.ai/metr-red-team-finds-vulnerabilities-in-openai-monitoring-system/"]},"analysis":{"sections":{"timeline":[{"text":"OpenAI publicly describes the monitoring system in a post.","quote":"OpenAI publicly described the tested system in a March 19, 2026 post.","source_index":0,"when":"2026-03-19"},{"text":"A subset of RL training rollouts was monitored for misalignment.","quote":"the evaluators were informed that a subset of RL training rollouts had been monitored for misalignment between March and June.","source_index":0,"when":"March to June 2026"},{"text":"Compaction events likely left about 10% of eligible internal Codex traffic unmonitored.","quote":"METR estimated that not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, a gap OpenAI staff had independently identified before the exercise began and fixed during it.","source_index":0,"when":"May to July 2026"},{"text":"The red-team exercise runs and the evaluators test the monitoring system.","quote":"The exercise ran from July 21 to August 11, 2026, and focused primarily on estimating the fraction of agentic inference that is monitored and on whether agents could in principle disable or subvert the monitoring system.","source_index":0,"when":"2026-07-21 to 2026-08-11"},{"text":"OpenAI states the identified vulnerabilities affecting its production safeguards were patched.","quote":"Since the exercise ended, OpenAI has stated that all of the identified vulnerabilities that impact its production safeguards stack have been patched and has described significant subsequent changes to the intended coverage of its monitoring system.","source_index":0,"when":null},{"text":"METR and Redwood Research publish a public account of the exercise.","quote":"METR and Redwood Research on October 22, 2026 published a public account of a pilot exercise in which one employee from each organization spent three weeks with deep access to OpenAI’s internal infrastructure, stress-testing the company’s internal misalignment monitoring system.","source_index":0,"when":"2026-10-22"}]},"model":"claude-haiku-5-5","prompt_version":"backfill-2026-11-01.2","extracted_at":"2026-11-01","source_urls":["https://www.unite.ai/metr-red-team-finds-vulnerabilities-in-openai-monitoring-system/"]}}