{"total":210,"limit":100,"offset":0,"practices":[{"schema_version":"0.1.0","id":"LP-2026-0073","record_version":1,"name":"ACTS (Artifacts in Cipher Testing Suite)","aliases":["ACTS"],"category":"benchmark","summary":"ACTS is a reproducible multi-tier benchmark that isolates cryptanalytic ability in LLMs by progressively depriving models of metadata. Tier-1 provides full metadata, Tier-2 gives filename only, and Tier-3 is completely blind, testing identification from ciphertext alone. Tier-5 tests forced reasoning strategies such as chain-of-thought, code-as-reasoning, and self-correction. The suite also includes a 10-configuration ablation study over 7,000 files using a single 70/30 train-test split for feature-removal analysis, and live API inference on a v2b corpus with randomised padding schemes, unique CSPRNG keys, and unique plaintexts.","problem_addressed":"Evaluating whether LLMs can identify ciphers from ciphertext alone without relying on metadata shortcuts, and measuring the gap between metadata-assisted and blind performance.","mitigates":["other"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The comparison against a classical Random Forest spans different input representations and training paradigms, so the observed gap should be interpreted as an overall capability difference rather than a clean factorial decomposition. The benchmark uses a single 70/30 train-test split for feature-removal analysis.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.26893v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"ACTS: A multi-tier benchmark evaluating LLM cipher identification under controlled blind conditions"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce ACTS (Artifacts in Cipher Testing Suite), a reproducible benchmark that isolates cryptanalytic ability through tiered metadata deprivation (Tier-1: full metadata; Tier-2: filename only; Tier-3: completely blind) and tests forced reasoning (Tier-5: chain-of-thought, code-as-reasoning, self-correction) on ciphertext alone."}],"summary":[{"source_index":0,"quote":"We introduce ACTS (Artifacts in Cipher Testing Suite), a reproducible benchmark that isolates cryptanalytic ability through tiered metadata deprivation (Tier-1: full metadata; Tier-2: filename only; Tier-3: completely blind) and tests forced reasoning (Tier-5: chain-of-thought, code-as-reasoning, self-correction) on ciphertext alone."},{"source_index":0,"quote":"A 10-configuration ablation study on 7,000 files, using a single 70/30 train-test split for feature-removal analysis, provides additional evidence at scale."},{"source_index":0,"quote":"Live API inference on a v2b corpus with fully randomised padding (PKCS7, ISO 10126, and ANSI X9.23 selected per file), unique CSPRNG keys, and unique plaintexts (127 files per model, 381 Tier-1 records, 380 Tier-3 records across three cloud systems, supplemented by 254 autonomous agentic evaluations in Tier-4) yields a combined Tier-3 accuracy of 30.8%, only modestly above the 14.3% random baseline for seven-way classification."}],"problem_addressed":[{"source_index":0,"quote":"We introduce ACTS (Artifacts in Cipher Testing Suite), a reproducible benchmark that isolates cryptanalytic ability through tiered metadata deprivation (Tier-1: full metadata; Tier-2: filename only; Tier-3: completely blind) and tests forced reasoning (Tier-5: chain-of-thought, code-as-reasoning, self-correction) on ciphertext alone."}],"limitations":[{"source_index":0,"quote":"Because this comparison spans different input representations and training paradigms, the gap should be interpreted as an overall capability difference rather than a clean factorial decomposition."},{"source_index":0,"quote":"A 10-configuration ablation study on 7,000 files, using a single 70/30 train-test split for feature-removal analysis, provides additional evidence at scale."}],"maturity":[{"source_index":0,"quote":"A 10-configuration ablation study on 7,000 files, using a single 70/30 train-test split for feature-removal analysis, provides additional evidence at scale."},{"source_index":0,"quote":"Live API inference on a v2b corpus with fully randomised padding (PKCS7, ISO 10126, and ANSI X9.23 selected per file), unique CSPRNG keys, and unique plaintexts (127 files per model, 381 Tier-1 records, 380 Tier-3 records across three cloud systems, supplemented by 254 autonomous agentic evaluations in Tier-4) yields a combined Tier-3 accuracy of 30.8%, only modestly above the 14.3% random baseline for seven-way classification."}],"evidence_strength":[{"source_index":0,"quote":"Live API inference on a v2b corpus with fully randomised padding (PKCS7, ISO 10126, and ANSI X9.23 selected per file), unique CSPRNG keys, and unique plaintexts (127 files per model, 381 Tier-1 records, 380 Tier-3 records across three cloud systems, supplemented by 254 autonomous agentic evaluations in Tier-4) yields a combined Tier-3 accuracy of 30.8%, only modestly above the 14.3% random baseline for seven-way classification."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0057","record_version":1,"name":"AI-powered anomaly detection in cloud environments","aliases":[],"category":"monitoring","summary":"The paper synthesizes a constituent study on AI-powered anomaly detection in cloud environments, which addresses the reliability of real-time monitoring as a point of failure in AI-driven security operations. This monitoring capability is incorporated into the proposed layered reference architecture alongside adversarially hardened models, blockchain-anchored data provenance, and smart-contract-governed multi-agent remediation. The technique uses AI to detect anomalies in cloud environments in real time, supporting the integrity and reliability of security operations across distributed systems.","problem_addressed":"The reliability of real-time monitoring in AI-driven security operations.","mitigates":["security_compromise","performance_drift","operational_error"],"addresses":[],"responds_to":[],"maturity":"proposed","evidence_strength":"none_stated","limitations":"The document states that open problems in scalability, privacy-transparency trade-offs, and the governance of autonomous agents must be resolved before such integrated systems can be trusted in production-critical environments.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28843v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Blockchain-Enabled Artificial Intelligence and AI Agents for Secure Data Sharing and Cybersecurity Applications"}],"field_citations":{"name":[{"source_index":0,"quote":"This paper presents a meta-synthesis that draws together four constituent studies covering adversarial machine learning, AI-powered anomaly detection in cloud environments, automated vulnerability patching by multi-agent large language model (LLM) pipelines, and the broader landscape of securing AI systems across their lifecycle"}],"summary":[{"source_index":0,"quote":"Each constituent study addresses a distinct point of failure in modern AI-driven security operations: the integrity of training data and model behavior, the reliability of real-time monitoring, and the trustworthiness of automated code remediation."}],"problem_addressed":[{"source_index":0,"quote":"Each constituent study addresses a distinct point of failure in modern AI-driven security operations: the integrity of training data and model behavior, the reliability of real-time monitoring, and the trustworthiness of automated code remediation."}],"limitations":[{"source_index":0,"quote":"We conclude by identifying open problems in scalability, privacy-transparency trade-offs, and the governance of autonomous agents that must be resolved before such integrated systems can be trusted in production-critical environments."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0119","record_version":1,"name":"APEXA-Bench","aliases":[],"category":"benchmark","summary":"An evaluation harness of 58 facility tasks (50 base plus an 8-task cross-detector slice) organized by a four-class physical-consequence taxonomy. It separates a wasted compute cycle from a damaged instrument, and includes cross-detector grading against NIST-traceable lattice constants. The benchmark is designed to surface latent pipeline bugs and evaluate agent performance on physically consequential tasks.","problem_addressed":"Chat benchmarks cannot see failure modes where an agent reports a calibration that was never computed, and do not distinguish between wasted compute and physical damage.","mitigates":["operational_error","physical_safety"],"addresses":["silent_wrong_state","deception_or_concealment"],"responds_to":["LR-2026-0051"],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Large-scale agent scoring is left to a full-length study.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24165v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"APEXA: Execution-Integrity Enforcement for Multi-Agent LLM Automation of Synchrotron Data Reduction"}],"field_citations":{"name":[{"source_index":0,"quote":"Second, we release APEXA-Bench, an evaluation harness of 58 facility tasks (50 base plus an 8-task cross-detector slice) organized by a four-class physical-consequence taxonomy, the first benchmark axis we know of separating a wasted compute cycle from a damaged instrument; its cross-detector grading against NIST-traceable lattice constants surfaced two latent pipeline bugs."}],"summary":[{"source_index":0,"quote":"Second, we release APEXA-Bench, an evaluation harness of 58 facility tasks (50 base plus an 8-task cross-detector slice) organized by a four-class physical-consequence taxonomy, the first benchmark axis we know of separating a wasted compute cycle from a damaged instrument; its cross-detector grading against NIST-traceable lattice constants surfaced two latent pipeline bugs."}],"problem_addressed":[{"source_index":0,"quote":"LLM agents promise to collapse it, but driving a real pipeline with a stochastic model creates a failure mode chat benchmarks cannot see: an agent can report a calibration that was never computed."}],"mitigates":[{"source_index":0,"quote":"the first benchmark axis we know of separating a wasted compute cycle from a damaged instrument"}],"maturity":[{"source_index":0,"quote":"Second, we release APEXA-Bench, an evaluation harness of 58 facility tasks (50 base plus an 8-task cross-detector slice) organized by a four-class physical-consequence taxonomy, the first benchmark axis we know of separating a wasted compute cycle from a damaged instrument; its cross-detector grading against NIST-traceable lattice constants surfaced two latent pipeline bugs."}],"evidence_strength":[{"source_index":0,"quote":"its cross-detector grading against NIST-traceable lattice constants surfaced two latent pipeline bugs."}],"limitations":[{"source_index":0,"quote":"Large-scale agent scoring is left to a full-length study."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0181","record_version":1,"name":"AURORA-Bench","aliases":[],"category":"benchmark","summary":"A benchmark for evaluating air-ground co-simulation scenario generation that assesses not only whether generated scenarios execute but whether they faithfully realize the requested interactions. It is designed to surface silent failures that completion-based evaluation overlooks, complementing execution-success metrics with fidelity checks against requested spatial, temporal, communication, and behavioral relationships.","problem_addressed":"Completion-based evaluation overlooks silent failures where scenarios execute but do not realize requested interactions.","mitigates":["operational_error","other"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.19527v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulation"}],"field_citations":{"name":[{"source_index":0,"quote":"We further introduce AURORA-Bench to evaluate not only whether generated scenarios execute, but whether they faithfully realize the requested interactions."}],"summary":[{"source_index":0,"quote":"We further introduce AURORA-Bench to evaluate not only whether generated scenarios execute, but whether they faithfully realize the requested interactions."},{"source_index":0,"quote":"Experiments across multiple language models show that structured execution substantially improves reliability, while runtime verification exposes silent failures that completion-based evaluation overlooks."}],"problem_addressed":[{"source_index":0,"quote":"Experiments across multiple language models show that structured execution substantially improves reliability, while runtime verification exposes silent failures that completion-based evaluation overlooks."}],"maturity":[{"source_index":0,"quote":"Experiments across multiple language models show that structured execution substantially improves reliability, while runtime verification exposes silent failures that completion-based evaluation overlooks."}],"evidence_strength":[{"source_index":0,"quote":"Experiments across multiple language models show that structured execution substantially improves reliability, while runtime verification exposes silent failures that completion-based evaluation overlooks."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0117","record_version":1,"name":"ActGov","aliases":["ActGov-Policy","ActGov-Runtime"],"category":"control_protocol","summary":"ActGov is a runtime enforcement framework that validates each LLM-proposed tool action before it causes external effects. It uses a unified semantic model of authorization, actions, runtime context, and security constraints. The ActGov-Policy component iteratively constructs a policy set from tool specifications, benign tasks, and observed failure traces, with each update verified through SMT-based counterexample checking. At runtime, ActGov-Runtime abstracts each tool call into finite policy records and permits it only if it remains within the task-scoped authorization boundary and satisfies all applicable policies. This per-action enforcement preserves authorization throughout long-horizon, dynamically branching workflows.","problem_addressed":"Untrusted outputs from external tools can influence subsequent LLM agent actions and exceed user authorization, and existing defenses are brittle under dynamic workflows and scale poorly across extensible tool ecosystems.","mitigates":["agent_misalignment","security_compromise"],"addresses":["harmful_compliance","injection_triggered","credential_misuse","data_exfiltration","containment_escape"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24446v2","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"ActGov: Governing LLM Agent Actions via Policy-Constrained Validation"}],"field_citations":{"name":[{"source_index":0,"quote":"In this work, we present ActGov, a runtime enforcement framework that validates each LLM-proposed tool action before it causes external effects."}],"summary":[{"source_index":0,"quote":"Built on a unified semantic model of authorization, actions, runtime context, and security constraints, the ActGov-Policy component iteratively constructs a policy set from tool specifications, benign tasks, and observed failure traces, with each update verified through SMT-based counterexample checking."},{"source_index":0,"quote":"At runtime, ActGov-Runtime abstracts each tool call into finite policy records and permits it only if it remains within the task-scoped authorization boundary and satisfies all applicable policies."},{"source_index":0,"quote":"This per-action enforcement preserves authorization throughout long-horizon, dynamically branching workflows."}],"problem_addressed":[{"source_index":0,"quote":"Large language model (LLM) agents increasingly execute long-horizon workflows through external tools, allowing untrusted outputs to influence subsequent actions and exceed user authorization."},{"source_index":0,"quote":"Existing defenses isolate injected content or constrain execution with predefined plans and static policies, but these approaches are brittle under dynamic workflows and scale poorly across extensible tool ecosystems."}],"mitigates":[{"source_index":0,"quote":"It shows that ActGov consistently reduces the success rate of indirect prompt-injection attacks while preserving task utility, significantly outperforming existing defenses."}],"maturity":[{"source_index":0,"quote":"We evaluate ActGov on the AgentDojo and AgentDyn benchmarks across multiple models and attack configurations."}],"evidence_strength":[{"source_index":0,"quote":"We evaluate ActGov on the AgentDojo and AgentDyn benchmarks across multiple models and attack configurations."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0164","record_version":1,"name":"Action-space design in a coding harness","aliases":[],"category":"other","summary":"A harness component that determines the interface through which the agent acts, varied between predefined tools and a bash-only interface. The paper finds that predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks. Trajectory-level analysis indicates that the action space changes the granularity at which code is written.","problem_addressed":"Matching agent action interfaces to model capability and task type for better performance and cost.","mitigates":["operational_error","performance_drift"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document states that the benefit of predefined tools depends on model bash proficiency and that bash-capable models can operate effectively with a bash-only interface, so the design is not uniformly superior.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.20804v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"An Empirical Study of Harness Design for Coding Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"Predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks."}],"summary":[{"source_index":0,"quote":"Predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks."}],"problem_addressed":[{"source_index":0,"quote":"Predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks."}],"mitigates":[{"source_index":0,"quote":"Predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks."}],"maturity":[{"source_index":0,"quote":"Across four models evaluated on SWE-Bench Verified and Terminal-Bench 2.1, we evaluate 176 matched settings spanning five context-management strategies, four context-window budgets, and targeted ablations of planning and action space."}],"evidence_strength":[{"source_index":0,"quote":"Across four models evaluated on SWE-Bench Verified and Terminal-Bench 2.1, we evaluate 176 matched settings spanning five context-management strategies, four context-window budgets, and targeted ablations of planning and action space."}],"limitations":[{"source_index":0,"quote":"Predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0149","record_version":1,"name":"Active curriculum synthesis with performance-informed calibration","aliases":[],"category":"evaluation","summary":"A curriculum mechanism that uses observed model performance to calibrate and adjust the distribution over generation parameters as model capabilities evolve. Instead of a fixed task set that saturates, the generator shifts the difficulty and composition of generated gyms in response to measured performance, keeping the evaluation frontier moving with the models being trained or assessed.","problem_addressed":"Task sets saturate as models improve and are increasingly exposed to contamination.","mitigates":["performance_drift"],"addresses":["reward_hacking"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not describe the calibration signal's reliability, the update cadence, or whether the curriculum can be gamed by the model being evaluated; no independent evaluation is reported.","organisations":[],"first_seen":"2026-09-18","last_seen":"2026-09-18","sources":[{"url":"https://arxiv.org/abs/2609.22592v1","publisher":"arXiv","date":"2026-09-18","type":"vendor_research","title":"AutoGym: Blueprint-First Generation of Verifiable Agent Gyms"}],"field_citations":{"name":[{"source_index":0,"quote":"Active curriculum synthesis uses performance-informed calibration to adjust the distribution over these parameters as model capabilities evolve."}],"summary":[{"source_index":0,"quote":"Active curriculum synthesis uses performance-informed calibration to adjust the distribution over these parameters as model capabilities evolve."}],"problem_addressed":[{"source_index":0,"quote":"Task sets saturate as models improve and are increasingly exposed to contamination."}],"mitigates":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"maturity":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"evidence_strength":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"limitations":[{"source_index":0,"quote":"Models comparable in capability solve them despite convoluted phrasing, and correctness must be adjudicated post-hoc by unreliable LLM judges."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0079","record_version":1,"name":"Adding a text tool list while retaining the native channel","aliases":[],"category":"control_protocol","summary":"A request-construction technique for tool-use evaluation in which the tool schema is supplied to the model as a text tool list in addition to the native tool-call channel, rather than replacing it. The document reports that this recovers much of the measured fidelity for models the serving stack accepts, whereas a uniform text protocol reduces fidelity for Llama-3.2, which has native tool-call support. It is a way of making the protocol step robust to serving-layer gating without discarding native capability.","problem_addressed":"Serving stacks gate or convert tool requests per model, so a single request format does not measure the same thing across models.","mitigates":["operational_error","performance_drift"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The benefit is not uniform: a uniform text protocol reduces fidelity for Llama-3.2, which has native tool-call support, so the approach must be conditioned on whether the model has native tool-call support.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.26693v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"Measuring the Serving Stack Instead of the Model: Hidden Confounds in Local Tool-Use Evaluation"}],"field_citations":{"name":[{"source_index":0,"quote":"Adding a text tool list while retaining the native channel recovers much of the measured fidelity for accepted models, whereas a uniform text protocol reduces fidelity for Llama-3.2, which has native tool-call support."}],"summary":[{"source_index":0,"quote":"Adding a text tool list while retaining the native channel recovers much of the measured fidelity for accepted models, whereas a uniform text protocol reduces fidelity for Llama-3.2, which has native tool-call support."}],"problem_addressed":[{"source_index":0,"quote":"In Ollama, the default tools= request is gated per model by a static template flag: some models are accepted and return calls as text, some return native tool_calls, while Phi-3 and Gemma-3 are rejected before inference."}],"limitations":[{"source_index":0,"quote":"Adding a text tool list while retaining the native channel recovers much of the measured fidelity for accepted models, whereas a uniform text protocol reduces fidelity for Llama-3.2, which has native tool-call support."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0129","record_version":1,"name":"Adversarial trials for detecting verifier bypasses","aliases":[],"category":"red_teaming","summary":"A validation step in which agents are run adversarially against a benchmark task's verifier to see whether the task can be passed without performing the intended work. Adversarial trials are one of the evidence streams combined in the ordered validity screen for all-fail tasks. In the studied corpus, this step identified 4 tasks that were only passable through verifier bypasses, which were then removed from the certified-unsolved candidate set. The technique targets the gap between a task's stated capability and what its scoring mechanism actually rewards.","problem_addressed":"Benchmark verifiers that can be satisfied by shortcuts, so that a task appears to measure a capability it does not actually test.","mitigates":["other"],"addresses":["reward_hacking"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state whether adversarial trials were exhaustive or how bypasses were judged, and notes that the certified-unsolved label does not prove verifier completeness.","organisations":[],"first_seen":"2026-09-20","last_seen":"2026-09-20","sources":[{"url":"https://arxiv.org/abs/2609.26826v1","publisher":"arXiv","date":"2026-09-20","type":"vendor_research","title":"What Makes a Terminal-Bench Task Hard? Separating Genuine Hardness from Fake-Hardness on an Adjudicated Agentic Corpus"}],"field_citations":{"name":[{"source_index":0,"quote":"For the 125 tasks with no honest pass, we combine task artifacts, reference-solution runs, empty-solution controls, adversarial trials, trajectories, telemetry, and review records, and apply an ordered validity screen."}],"summary":[{"source_index":0,"quote":"The remaining tasks include 14 with broken oracles, 8 dominated by infrastructure failures, 4 that are only passable through verifier bypasses, and 21 whose solvability is not certified by the available evidence."}],"problem_addressed":[{"source_index":0,"quote":"The same zero pass rate can come from a real capability gap, but it can also come from missing context, a broken reference solution, infrastructure failure, or a verifier that can be bypassed."}],"mitigates":[{"source_index":0,"quote":"The remaining tasks include 14 with broken oracles, 8 dominated by infrastructure failures, 4 that are only passable through verifier bypasses, and 21 whose solvability is not certified by the available evidence."}],"maturity":[{"source_index":0,"quote":"For the 125 tasks with no honest pass, we combine task artifacts, reference-solution runs, empty-solution controls, adversarial trials, trajectories, telemetry, and review records, and apply an ordered validity screen."}],"evidence_strength":[{"source_index":0,"quote":"In this paper, we study this issue using a frozen Terminal-Bench 3 / Frontier-Bench 0.1 production record with 1,081 pull requests, 639 scored tasks, 28,801 trials, and $105,933 in logged agent spend."}],"limitations":[{"source_index":0,"quote":"It does not prove intrinsic hardness, verifier completeness, or failure at the intended capability."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0058","record_version":1,"name":"Adversarially hardened models","aliases":[],"category":"alignment_training","summary":"The paper synthesizes a constituent study on adversarial machine learning and proposes coupling adversarially hardened models into a layered reference architecture for securing data sharing and autonomous decision-making. This component addresses the integrity of training data and model behavior as a point of failure in AI-driven security operations. The hardening is intended to make models more robust against adversarial manipulation, and it is combined with blockchain-anchored data provenance, AI-driven anomaly detection, and smart-contract-governed multi-agent remediation.","problem_addressed":"The integrity of training data and model behavior in AI-driven security operations.","mitigates":["security_compromise","harmful_output"],"addresses":["injection_triggered","reward_hacking"],"responds_to":[],"maturity":"proposed","evidence_strength":"none_stated","limitations":"The document states that open problems in scalability, privacy-transparency trade-offs, and the governance of autonomous agents must be resolved before such integrated systems can be trusted in production-critical environments.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28843v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Blockchain-Enabled Artificial Intelligence and AI Agents for Secure Data Sharing and Cybersecurity Applications"}],"field_citations":{"name":[{"source_index":0,"quote":"we propose a layered reference architecture that couples adversarially hardened models, blockchain-anchored data provenance, AI-driven anomaly detection, and smart-contract-governed multi-agent remediation"}],"summary":[{"source_index":0,"quote":"Each constituent study addresses a distinct point of failure in modern AI-driven security operations: the integrity of training data and model behavior, the reliability of real-time monitoring, and the trustworthiness of automated code remediation."}],"problem_addressed":[{"source_index":0,"quote":"Each constituent study addresses a distinct point of failure in modern AI-driven security operations: the integrity of training data and model behavior, the reliability of real-time monitoring, and the trustworthiness of automated code remediation."}],"limitations":[{"source_index":0,"quote":"We conclude by identifying open problems in scalability, privacy-transparency trade-offs, and the governance of autonomous agents that must be resolved before such integrated systems can be trusted in production-critical environments."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0088","record_version":1,"name":"Affected-clean-traffic detection model","aliases":[],"category":"evaluation","summary":"A detection model that measures fault detectability by the volume of affected clean traffic rather than by nominal cell coverage. Across 540 repeated unit-arm rows nested in 20 clusters, affected clean traffic reduced negative log likelihood by 29.3% relative to cell coverage, a gain of 0.1264 nats per row (cluster-bootstrap 95% interval [0.0593, 0.1918]). Adding mask family and its interaction improved log loss by 0.0015 nats per row (one-sided upper bound 0.0066), below the frozen 0.01 practical-sufficiency margin. The approach reframes detection as a function of exposure to affected states rather than of the fraction of cells nominally covered.","problem_addressed":"Nominal cell coverage overstates detectability because a closed-loop policy may rarely visit affected cells, so faults can leave no aggregate trace.","mitigates":["performance_drift","operational_error"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The mask-family addition improved log loss by only 0.0015 nats per row, below the frozen 0.01 practical-sufficiency margin, indicating limited incremental value; evaluation is on a simulated hotel-pricing agent.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.25806v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"When Are Aggregate Agent Traces Diagnosable? Traffic-Governed Interpretation and Calibrated Abstention"}],"field_citations":{"name":[{"source_index":0,"quote":"We calibrate false admission on a disjoint clean stream at the physical-component level and model detection by affected clean traffic rather than nominal cell coverage."}],"summary":[{"source_index":0,"quote":"We calibrate false admission on a disjoint clean stream at the physical-component level and model detection by affected clean traffic rather than nominal cell coverage."},{"source_index":0,"quote":"Across 540 repeated unit-arm rows nested in those 20 clusters, affected clean traffic reduced negative log likelihood by 29.3% relative to cell coverage, a gain of 0.1264 nats per row (cluster-bootstrap 95% interval [0.0593, 0.1918])."},{"source_index":0,"quote":"Adding mask family and its interaction improved log loss by 0.0015 nats per row (one-sided upper bound 0.0066), below the frozen 0.01 practical-sufficiency margin."}],"problem_addressed":[{"source_index":0,"quote":"A fault may leave no aggregate trace when the policy rarely visits affected cells."}],"limitations":[{"source_index":0,"quote":"Adding mask family and its interaction improved log loss by 0.0015 nats per row (one-sided upper bound 0.0066), below the frozen 0.01 practical-sufficiency margin."}],"maturity":[{"source_index":0,"quote":"Across 540 repeated unit-arm rows nested in those 20 clusters, affected clean traffic reduced negative log likelihood by 29.3% relative to cell coverage, a gain of 0.1264 nats per row (cluster-bootstrap 95% interval [0.0593, 0.1918])."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0116","record_version":1,"name":"Agent-specific AI security incident reporting elements","aliases":[],"category":"incident_response","summary":"A proposed set of reporting elements for security incidents involving AI agents, derived by comparing AI systems with AI agents and drawing on input from 23 academic and industry experts. The elements go beyond conventional incident reports to capture agent-specific state and behaviour, including agent memory and memory accesses, actual and potential levels of autonomy, and tool usage. The paper argues that existing incident reporting frameworks must be adapted to these characteristics to support legal compliance, governance, accountability and security, and frames the elements as a basis for further research on efficiently recording incidents and determining whether vulnerabilities and incidents generalize.","problem_addressed":"Existing incident reporting frameworks are not adapted to the unique characteristics of AI agents, leaving agent-specific harms and attack surfaces unreported.","mitigates":["security_compromise","agent_misalignment","operational_error"],"addresses":[],"responds_to":[],"maturity":"proposed","evidence_strength":"author_claim","limitations":"The paper presents the reporting elements as potential and identifies open research questions rather than a validated scheme; expert feedback also highlighted risks of data leakage and attacks targeting the reporting infrastructure itself, creating additional research needs.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24515v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"Beyond Predictable Paths: Redefining AI Security Incident Reporting for Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"Potential reporting elements include, for example, agent memory and memory accesses, actual and potential levels of autonomy, and tool usage."}],"summary":[{"source_index":0,"quote":"In this paper, two editorial authors compare AI systems and AI agents and, drawing on input from 23 experts in academia and industry, identify the information required for reporting incidents where the security of AI agents is harmed."}],"problem_addressed":[{"source_index":0,"quote":"As incident reporting becomes increasingly important for legal compliance, governance, accountability, and security; current frameworks must be adapted to the unique characteristics of AI agents."}],"mitigates":[{"source_index":0,"quote":"identify the information required for reporting incidents where the security of AI agents is harmed"}],"maturity":[{"source_index":0,"quote":"Potential reporting elements include, for example, agent memory and memory accesses, actual and potential levels of autonomy, and tool usage."}],"evidence_strength":[{"source_index":0,"quote":"drawing on input from 23 experts in academia and industry, identify the information required for reporting incidents where the security of AI agents is harmed"}],"limitations":[{"source_index":0,"quote":"Expert feedback also highlighted potential reporting weaknesses, such as risks of data leakage and attacks targeting the reporting infrastructure itself, creating additional research needs."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0201","record_version":1,"name":"Agora append-only Git DAG for shared agent research memory","aliases":["Agora"],"category":"provenance","summary":"Agora stores research-agent contributions as an append-only directed acyclic graph in Git. Each commit records a result, insight, hypothesis, verification, or report and links it to prior work, so separate agent sessions can see what others tried and which results they can build on. Searchable views surface leading results, neglected branches, and verification status, and diversity-aware recommendations suggest experiments beyond the current leaders. In the reported run, 13 language-model workers with no assigned tasks or central planner used Agora to solve a weight-transfer problem, publishing 1,703 contributions and 165 independent reproductions across 95 targets.","problem_addressed":"Research agents working in separate sessions lack shared memory of what others have tried and which results are verified and reusable.","mitigates":["agent_misalignment","operational_error","other"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"piloted","evidence_strength":"author_evaluation","limitations":"The document states that measuring the effect on discovery per unit of compute requires a matched comparison, so the run does not establish that Agora improves discovery efficiency.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18094v2","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"Agora: Git as Shared Memory for Collective AutoResearch"}],"field_citations":{"name":[{"source_index":0,"quote":"Agora stores their contributions as an append-only directed acyclic graph (DAG) in Git."}],"summary":[{"source_index":0,"quote":"Each commit records a result, insight, hypothesis, verification, or report and links it to prior work. Searchable views show leading results, neglected branches, and verification status; diversity-aware recommendations suggest experiments beyond the current leaders."}],"problem_addressed":[{"source_index":0,"quote":"Research agents working in separate sessions need to know what others have tried and which results they can build on."}],"mitigates":[{"source_index":0,"quote":"The run documents how agents reused and verified shared work."}],"maturity":[{"source_index":0,"quote":"We report a run of nearly 12 days in which 13 language-model workers, with no assigned tasks or central planner, used Agora to solve a weight-transfer problem."}],"evidence_strength":[{"source_index":0,"quote":"They published 1,703 contributions and reduced the development evaluator score from 3.39 to 1.899 bits per byte, closing 62% of the gap to a trained GPT-2 124M."}],"limitations":[{"source_index":0,"quote":"Measuring the effect on discovery per unit of compute requires a matched comparison."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0179","record_version":1,"name":"Air-Ground Scenario Graph (AGSG)","aliases":[],"category":"other","summary":"A typed intermediate representation for air-ground co-simulation scenarios that explicitly connects agents, aerial missions, events, communication links, success conditions, and their cross-domain dependencies. It serves as a shared representation enabling simulator-grounded parsing, joint road-airspace grounding, temporal planning, pre-execution feasibility checking, trace-based runtime verification, failure localization, and bounded repair within a unified workflow. The graph is the central artifact through which natural-language scenario requests are compiled and verified.","problem_addressed":"Generated scenarios may execute successfully while failing to realize the spatial, temporal, communication, or behavioral relationships requested by the user.","mitigates":["operational_error","other"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.19527v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulation"}],"field_citations":{"name":[{"source_index":0,"quote":"Central to AURORA is the Air-Ground Scenario Graph (AGSG), a typed intermediate representation that explicitly connects agents, aerial missions, events, communication links, success conditions, and their cross-domain dependencies."}],"summary":[{"source_index":0,"quote":"Central to AURORA is the Air-Ground Scenario Graph (AGSG), a typed intermediate representation that explicitly connects agents, aerial missions, events, communication links, success conditions, and their cross-domain dependencies."},{"source_index":0,"quote":"This shared representation enables simulator-grounded parsing, joint road-airspace grounding, temporal planning, pre-execution feasibility checking, trace-based runtime verification, failure localization, and bounded repair within a unified workflow."}],"problem_addressed":[{"source_index":0,"quote":"More importantly, a generated scenario may execute successfully while failing to realize the spatial, temporal, communication, or behavioral relationships requested by the user."}],"maturity":[{"source_index":0,"quote":"Experiments across multiple language models show that structured execution substantially improves reliability, while runtime verification exposes silent failures that completion-based evaluation overlooks."}],"evidence_strength":[{"source_index":0,"quote":"Experiments across multiple language models show that structured execution substantially improves reliability, while runtime verification exposes silent failures that completion-based evaluation overlooks."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0024","record_version":1,"name":"Approval before state-changing actions","aliases":[],"category":"control_protocol","summary":"A human-in-the-loop gate in which the agent must obtain approval before executing any action that changes state. The orchestrator can propose such actions but cannot complete them unilaterally, keeping consequential steps under user or operator control. The document lists this as part of the architecture but does not report a separate quantitative evaluation of the approval gate itself.","problem_addressed":"Unreviewed consequential actions taken autonomously by an agent.","mitigates":["agent_misalignment","operational_error"],"addresses":["harmful_compliance","covert_sabotage","silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_claim","limitations":"The document does not report a separate evaluation of the approval mechanism, and notes that unreliable failure fallback remained.","organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29341v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"SkinAgent AI: A Safety-Grounded Multimodal Agentic Framework for Non-Diagnostic Skincare Support"}],"field_citations":{"name":[{"source_index":0,"quote":"The architecture includes routing for Acne, Pores, and Wrinkles; photograph-based skin-type estimation; count-informed ordinal acne-severity support; typed tools; database-grounded recommendation and action functions; deterministic safety, privacy, and evidence checks; approval before state-changing actions; and structured trace and replay mechanisms."}],"summary":[{"source_index":0,"quote":"The architecture includes routing for Acne, Pores, and Wrinkles; photograph-based skin-type estimation; count-informed ordinal acne-severity support; typed tools; database-grounded recommendation and action functions; deterministic safety, privacy, and evidence checks; approval before state-changing actions; and structured trace and replay mechanisms."}],"maturity":[{"source_index":0,"quote":"Visual-model performance and system-level agent behavior were evaluated separately."}],"evidence_strength":[{"source_index":0,"quote":"These findings support the feasibility of bounded, database-grounded, and traceable agent orchestration for non-diagnostic skincare assistance."}],"limitations":[{"source_index":0,"quote":"Tool-selection errors, incomplete grounding of product attributes, and unreliable failure fallback nevertheless remained."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0060","record_version":1,"name":"Approval-to-Action Security Benchmark","aliases":[],"category":"benchmark","summary":"A benchmark that binds approval objects and decision-time metadata to post-execution evidence, enabling measurement of whether the durable record of an approved invocation covers the transitive effects its workflow actually exercises. It uses fixed approval-object/trace pairs and fixed-SHA executions to quantify residual records across effect classes, and supports a ladder of record types from explicit fields to command semantics to decision-time metadata. It is used to establish approval laundering as a measurable, recurrent record-coverage failure.","problem_addressed":"Durable approval records name the entry invocation but omit effects exercised by the transitive workflow it activates, a failure the authors call approval laundering.","mitigates":["agent_misalignment","operational_error","security_compromise"],"addresses":["deception_or_concealment","silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document reports results only across 111 fixed approval-object/trace pairs and 11 fixed-SHA executions; it does not claim the benchmark generalizes beyond those prespecified sets.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28586v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Agent Approval Laundering: Transitive Effects Beyond the Approved Invocation"}],"field_citations":{"name":[{"source_index":0,"quote":"The Approval-to-Action Security Benchmark binds approval objects and decision-time metadata to post-execution evidence."}],"summary":[{"source_index":0,"quote":"The Approval-to-Action Security Benchmark binds approval objects and decision-time metadata to post-execution evidence."},{"source_index":0,"quote":"Across 111 fixed approval-object/trace pairs, residual records fall from 40 under explicit fields to 17 with command semantics and 13 with decision-time metadata."}],"problem_addressed":[{"source_index":0,"quote":"We call the resulting record-coverage failure approval laundering: the durable record names the entry invocation but omits effects exercised by its workflow."}],"mitigates":[{"source_index":0,"quote":"We formalize closure-bound approval over six effect classes and derive an information limit: identical policy-visible fields can require different effect-specific decisions, so no record-only policy can guarantee both."}],"maturity":[{"source_index":0,"quote":"Across 111 fixed approval-object/trace pairs, residual records fall from 40 under explicit fields to 17 with command semantics and 13 with decision-time metadata."}],"evidence_strength":[{"source_index":0,"quote":"Across 111 fixed approval-object/trace pairs, residual records fall from 40 under explicit fields to 17 with command semantics and 13 with decision-time metadata."}],"limitations":[{"source_index":0,"quote":"Across 111 fixed approval-object/trace pairs, residual records fall from 40 under explicit fields to 17 with command semantics and 13 with decision-time metadata."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0122","record_version":1,"name":"Attention-difference-based pseudo mask selection","aliases":[],"category":"evaluation","summary":"A weakly supervised localization framework that suppresses false positives by measuring attention changes before and after removing a candidate region. If removing a candidate region produces a large attention difference, the region is selected as a pseudo mask, enabling localization without dense annotations. The learned Action-Slot representations supply the transferable spatio-temporal grounding signals this selection operates over.","problem_addressed":"Weakly supervised localization without dense annotations, and false positives in candidate region selection.","mitigates":["other"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the pseudo mask selection framework.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24127v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"Action-Slot: Structured Action-Centric Representation Learning for Multi-Agent Atomic Activity Understanding"}],"field_citations":{"name":[{"source_index":0,"quote":"We further propose an attention-difference-based pseudo mask selection framework that suppresses false positives by measuring attention changes before and after candidate region removal, enabling weakly supervised localization without dense annotations."}],"summary":[{"source_index":0,"quote":"Beyond recognition, the learned representations encode transferable spatio-temporal grounding signals."},{"source_index":0,"quote":"We further propose an attention-difference-based pseudo mask selection framework that suppresses false positives by measuring attention changes before and after candidate region removal, enabling weakly supervised localization without dense annotations."}],"problem_addressed":[{"source_index":0,"quote":"enabling weakly supervised localization without dense annotations"}],"maturity":[{"source_index":0,"quote":"Experiments on OATS, TACO, and annotated nuScenes show superior recognition, strong sim-to-real transfer, and state-of-the-art weakly supervised localization."}],"evidence_strength":[{"source_index":0,"quote":"Experiments on OATS, TACO, and annotated nuScenes show superior recognition, strong sim-to-real transfer, and state-of-the-art weakly supervised localization."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0107","record_version":1,"name":"Attribute-blocking privacy controls for personal AI agents","aliases":[],"category":"guardrail","summary":"A privacy control mechanism that blocks specific personal attributes (e.g., financial attributes) from being provided to a personal AI agent, intended to prevent the agent from inferring sensitive user characteristics such as wealth and steering recommendations based on them. The document evaluates this control by comparing agent behavior when financial attributes are blocked versus when other attributes are blocked, finding that blocking financial attributes largely removes the wealth-based disparity, but blocking other attributes leaves it unchanged or can increase it by up to 40% for insurance decisions.","problem_addressed":"Personal AI agents inferring wealth from personal context and steering economic recommendations based on inferred wealth, even against user instructions.","mitigates":["bias_disparate_impact","privacy_leakage"],"addresses":["reward_hacking"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Blocking non-financial attributes leaves the disparity unchanged and can increase it by up to 40% for insurance, as agents rely on remaining signals to infer wealth.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24927v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"Et Tu, Brute? Economic Misalignment in Personal AI Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"And it persists under privacy controls that block specific attributes: blocking financial attributes largely removes the disparity, but blocking other attributes leaves it unchanged and can increase it by up to 40% for insurance, as agents rely on the remaining signals to infer wealth."}],"summary":[{"source_index":0,"quote":"And it persists under privacy controls that block specific attributes: blocking financial attributes largely removes the disparity, but blocking other attributes leaves it unchanged and can increase it by up to 40% for insurance, as agents rely on the remaining signals to infer wealth."}],"problem_addressed":[{"source_index":0,"quote":"We show that by simply providing this personal context, the agent steers recommendations based on inferred wealth, without being explicitly instructed to do so."}],"mitigates":[{"source_index":0,"quote":"And it persists under privacy controls that block specific attributes: blocking financial attributes largely removes the disparity, but blocking other attributes leaves it unchanged and can increase it by up to 40% for insurance, as agents rely on the remaining signals to infer wealth."}],"maturity":[{"source_index":0,"quote":"In a suite of 325K experiments on 13 agents across three types of economic decisions (flights, health insurance, and graduate programs), we find that 8 models systematically choose more expensive options for wealthier users when requests are identical."}],"evidence_strength":[{"source_index":0,"quote":"In a suite of 325K experiments on 13 agents across three types of economic decisions (flights, health insurance, and graduate programs), we find that 8 models systematically choose more expensive options for wealthier users when requests are identical."}],"limitations":[{"source_index":0,"quote":"And it persists under privacy controls that block specific attributes: blocking financial attributes largely removes the disparity, but blocking other attributes leaves it unchanged and can increase it by up to 40% for insurance, as agents rely on the remaining signals to infer wealth."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0197","record_version":1,"name":"AutoTuneBench","aliases":[],"category":"benchmark","summary":"AutoTuneBench is a benchmark and measurement protocol for evaluating agent-driven auto-tuning of LLM serving engines. It makes trust architectural by freezing the protocol as code with test-enforced provenance, using a database-level validator to reject out-of-protocol results, running anti-cheat checks outside the agent's modification surface, following pre-registered readouts for comparisons, and anchoring measurements to externally published results. It employs paired-seed statistics with a 5% cross-run coefficient-of-variation cap to ensure reliability. The protocol addresses four failure modes: strawman baselines, non-transferable absolute times, saturated tasks, and infrastructure defects.","problem_addressed":"The measurements behind the closed loop of propose, measure, and keep for tuning GPU kernels and serving engines are not trustworthy, leading to misleading speedups and non-reproducible results.","mitigates":["performance_drift","operational_error","other"],"addresses":["reward_hacking","deception_or_concealment"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not explicitly state limitations of the protocol, but it acknowledges that the pilot corpus revealed failure modes and that the protocol is a response to those; it does not claim to eliminate all measurement issues.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18123v1","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines"}],"field_citations":{"name":[{"source_index":0,"quote":"We present AutoTuneBench, a benchmark and measurement protocol that makes trust architectural."}],"summary":[{"source_index":0,"quote":"The protocol is frozen as code with test-enforced provenance; a database-level validator rejects out-of-protocol results; anti-cheat checks run outside the agent's modification surface; comparisons follow pre-registered readouts; and measurements anchor to externally published results, grounded in paired-seed statistics with a 5\\% cross-run coefficient-of-variation cap."}],"problem_addressed":[{"source_index":0,"quote":"Large language model agents tune GPU kernels and serving engines through a closed loop of propose, measure, and keep, but the measurements behind this loop are not trustworthy."}],"mitigates":[{"source_index":0,"quote":"We characterize four failure modes from a four-day pilot corpus of 619 model calls: strawman baselines manufacture speedups, absolute times do not transfer across machines, saturated tasks nullify comparisons, and infrastructure defects impersonate science."}],"maturity":[{"source_index":0,"quote":"We present AutoTuneBench, a benchmark and measurement protocol that makes trust architectural."}],"evidence_strength":[{"source_index":0,"quote":"Honest measurement rewrites the headlines: our best kernel reads 10.6x against a naive baseline but 2.03x against the honest one; one configuration delivers 1.174x on one machine and 1.0049x on another; a pre-registered on/off comparison nulls at a shared wall (2.4840 vs 2.4957\\,ms); and the KernelBench Level-1 suite admits 51\\% of tasks with median speedup 1.0001x over PyTorch eager."}],"limitations":[{"source_index":0,"quote":"The protocol, the two-engine corpus (vLLM and SGLang), and its audit trail are released as open artifacts."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0108","record_version":1,"name":"BackTrend","aliases":[],"category":"benchmark","summary":"BackTrend is a retrospective benchmark for evaluating scientific weak-signal prediction. Given a mature target topic and a temporal evidence constraint, systems must recover two types of precursors: problem-space signals (underrecognized research problems) and solution-space signals (emerging methods for known problems). The benchmark contains 25 mature target topics in AI and machine learning and 66 human-validated weak signals, reconstructed from large-scale literature by grounding each candidate in its 2019-2024 publication-frequency trajectory. Systems are evaluated using semantic matching and coverage-based metrics, with F1 and Coverage10 as headline measures.","problem_addressed":"Existing resources such as trend tracking, citation forecasting, and foresight reports rarely provide validated reference sets that link concrete early precursors to later paradigms.","mitigates":["other"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The strongest system achieves only 10.1% F1, while Coverage10 reaches at most 18.5% of the reference signals. Additional retrieval and web-search evidence can improve performance up to a moderate budget, but does not by itself close the substantial performance gap.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24921v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"BackTrend: Evaluating Scientific Weak-Signal Prediction via Backward Reconstruction"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce BackTrend, a retrospective benchmark in which, given a mature target topic and a temporal evidence constraint, systems must recover two types of precursors: problem-space signals, underrecognized research problems, and solution-space signals, emerging methods for known problems."}],"summary":[{"source_index":0,"quote":"BackTrend contains 25 mature target topics in artificial intelligence and machine learning and 66 human-validated weak signals, reconstructed from large-scale literature by grounding each candidate in its 2019-2024 publication-frequency trajectory."},{"source_index":0,"quote":"We evaluate frontier LLMs, RAG systems, and agentic research systems using semantic matching and coverage-based metrics."}],"problem_addressed":[{"source_index":0,"quote":"Scientific weak signals are early, low-visibility research directions that later become central to mature scientific topics, yet existing resources such as trend tracking, citation forecasting, and foresight reports rarely provide validated reference sets that link concrete early precursors to later paradigms."}],"limitations":[{"source_index":0,"quote":"Current systems often generate plausible but misaligned precursors, exhibiting topic drift, granularity mismatch, near-miss matching, and incomplete coverage; the strongest system achieves only 10.1% F1, while Coverage10 reaches at most 18.5% of the reference signals."},{"source_index":0,"quote":"Our budget analyses show that additional retrieval and web-search evidence can improve performance up to a moderate budget, but does not by itself close the substantial performance gap."}],"maturity":[{"source_index":0,"quote":"We evaluate frontier LLMs, RAG systems, and agentic research systems using semantic matching and coverage-based metrics."}],"evidence_strength":[{"source_index":0,"quote":"We evaluate frontier LLMs, RAG systems, and agentic research systems using semantic matching and coverage-based metrics."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0157","record_version":1,"name":"Benchproofer","aliases":[],"category":"evaluation","summary":"Benchproofer is a pipeline that converts a coding task with a known correct patch into a formally verified instance. It writes a formal specification for the new code, summarizes existing functions called by that code with axioms, and admits the instance only after mechanical and adversarial gates agree. Applied to SWE-bench Verified, it produces SWE-Proof, a benchmark of 500 real issues whose correctness is formally verified rather than tested. The pipeline extends to SWE-bench Pro. It is designed to catch incorrect patches that pass held-out test suites, addressing incompleteness and memorization in test-based evaluation.","problem_addressed":"Held-out test suites for agentic code generation are inherently incomplete and increasingly susceptible to memorization, allowing incorrect patches to pass.","mitigates":["other"],"addresses":["reward_hacking"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Writing the formal specification is the hard part: models that must write their own gain nothing over an unaided baseline, and only 62% of their specifications pass the audit. The usual failure is faithfulness, a specification that constrains part of the required behavior and leaves the rest free. Specification quality still tracks the outcome, failing on 89% of unresolved instances against 47% of resolved ones.","organisations":[],"first_seen":"2026-09-18","last_seen":"2026-09-18","sources":[{"url":"https://arxiv.org/abs/2609.21190v1","publisher":"arXiv","date":"2026-09-18","type":"vendor_research","title":"SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?"}],"field_citations":{"name":[{"source_index":0,"quote":"We present Benchproofer, a pipeline that turns a coding task with a known correct patch into a formally verified one: it writes a specification for the new code, summarizes the existing functions that code calls with axioms, and admits an instance only after mechanical and adversarial gates agree."}],"summary":[{"source_index":0,"quote":"We present Benchproofer, a pipeline that turns a coding task with a known correct patch into a formally verified one: it writes a specification for the new code, summarizes the existing functions that code calls with axioms, and admits an instance only after mechanical and adversarial gates agree."},{"source_index":0,"quote":"Applying it to SWE-bench Verified yields SWE-Proof, 500 real issues whose correctness is formally verified rather than tested, and it extends to SWE-bench Pro."}],"problem_addressed":[{"source_index":0,"quote":"Benchmarks for agentic code generation check correctness with held-out test suites, which are inherently incomplete and increasingly susceptible to memorization."}],"mitigates":[{"source_index":0,"quote":"Formal verification avoids both problems, but existing work covers only standalone tasks whose specifications are given as input, not real issues, which touch large repositories and state intent in vague natural language."}],"maturity":[{"source_index":0,"quote":"Applying it to SWE-bench Verified yields SWE-Proof, 500 real issues whose correctness is formally verified rather than tested, and it extends to SWE-bench Pro."}],"evidence_strength":[{"source_index":0,"quote":"Across two frontier models, verification catches what tests miss: a quarter to a half of test-passing patches admit counterexamples, which a structured natural-language specification does not fix, while a correct formal one lifts resolution from 85% to 95% for Opus 4.8."}],"limitations":[{"source_index":0,"quote":"Writing that specification is the hard part: models that must write their own gain nothing over an unaided baseline, and only 62% of their specifications pass our audit."},{"source_index":0,"quote":"The usual failure is faithfulness, a specification that constrains part of the required behavior and leaves the rest free."},{"source_index":0,"quote":"Specification quality still tracks the outcome, failing on 89% of unresolved instances against 47% of resolved ones, making faithful specification synthesis a concrete open problem."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0195","record_version":1,"name":"Best-practices agentic AI software development workflow with human supervision","aliases":["typical best-practices agentic AI workflow for software development"],"category":"other","summary":"A structured agentic AI software development workflow in which prompts and skills direct the agent and a human supervises and responds to the process. The workflow is applied as a repeatable practice for generating release-quality programs, with the prompts and skills functioning as a specification of the code. The authors report that robust and reliable software generation requires careful practice and human supervision, and that the resulting prompt-and-skill specification supports cost-effective sustainability of the software.","problem_addressed":"Generating robust and reliable software using agentic AI while retaining human control over quality.","mitigates":["operational_error","agent_misalignment"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The authors state that the quality of the code is highly dependent on the prompts and skills used and on how the human directing the process responds, so results are not guaranteed by the workflow alone.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18298v1","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"A Study of the Reliability of Agentic AI-Generated Programs"}],"field_citations":{"name":[{"source_index":0,"quote":"First, we applied a typical best-practices agentic AI workflow for software development."}],"summary":[{"source_index":0,"quote":"In addition, we verified that generating robust and reliable software using agentic AI requires careful practice and human supervision."},{"source_index":0,"quote":"We also demonstrated that using agentic AI workflow for software development (with its prompts and skills) can become a specification of the code that leads to cost-effective sustainability of the software."}],"problem_addressed":[{"source_index":0,"quote":"Agentic-AI based software development offers the promise of faster completion of the software, greater programmer efficiency, and more reliable code."}],"mitigates":[{"source_index":0,"quote":"In addition, we verified that generating robust and reliable software using agentic AI requires careful practice and human supervision."}],"maturity":[{"source_index":0,"quote":"In this project, we attempted to answer this question based on three practices."}],"evidence_strength":[{"source_index":0,"quote":"We also demonstrated that using agentic AI workflow for software development (with its prompts and skills) can become a specification of the code that leads to cost-effective sustainability of the software."}],"limitations":[{"source_index":0,"quote":"The quality of the code is highly dependent on the prompts and skills used, and how the human directing the process responds."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0134","record_version":1,"name":"Black-box trigger probing of LLM explainers","aliases":[],"category":"evaluation","summary":"A black-box audit method for LLM-based runtime oversight of autonomous agents. The agent and its LLM explainer are paired and probed with three triggers: corrupting the observation stream (e.g., by 600 MW per step) to see whether explanations flag the resulting belief shift; selecting timesteps where the agent takes an objectively wrong action to test whether the explainer rationalizes it; and injecting attacker-controlled text into the observation metadata field to test whether the explainer can be steered or induced to exfiltrate data. The method checks the account itself rather than the agent's internal state, and is proposed as a standard part of any audit of an agentic deployment.","problem_addressed":"LLM explainers attached to autonomous agents as runtime oversight can produce fluent but false accounts, and nothing in the explainer architecture checks whether an explanation is true before an operator acts on it.","mitigates":["agent_misalignment","operational_error","security_compromise"],"addresses":["deception_or_concealment","injection_triggered","data_exfiltration"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The authors state that they propose mitigations for each failure but do not evaluate them. The audit itself is run by the authors on their own paired system, not independently.","organisations":[],"first_seen":"2026-09-19","last_seen":"2026-09-19","sources":[{"url":"https://arxiv.org/abs/2609.23215v1","publisher":"arXiv","date":"2026-09-19","type":"vendor_research","title":"Triggers and Diagnostics for LLM-Based Interpretability Failures in Active Inference Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"We audit the account itself, pairing an Active Inference (AIF) agent that tracks German grid demand and adjusts generation with an LLM explainer on three backends (GPT-4o, Claude-3-Opus, Gemini), and probing the pair with three black-box triggers."}],"summary":[{"source_index":0,"quote":"We audit the account itself, pairing an Active Inference (AIF) agent that tracks German grid demand and adjusts generation with an LLM explainer on three backends (GPT-4o, Claude-3-Opus, Gemini), and probing the pair with three black-box triggers."}],"problem_addressed":[{"source_index":0,"quote":"Moreover, nothing in the explainer architecture checks whether an explanation is true before an operator acts on it."}],"mitigates":[{"source_index":0,"quote":"Attacker-controlled text in the observation metadata field steers the explainer, with susceptibility differing by provider and data exfiltration succeeding on all three."}],"maturity":[{"source_index":0,"quote":"We audit the account itself, pairing an Active Inference (AIF) agent that tracks German grid demand and adjusts generation with an LLM explainer on three backends (GPT-4o, Claude-3-Opus, Gemini), and probing the pair with three black-box triggers."}],"evidence_strength":[{"source_index":0,"quote":"We audit the account itself, pairing an Active Inference (AIF) agent that tracks German grid demand and adjusts generation with an LLM explainer on three backends (GPT-4o, Claude-3-Opus, Gemini), and probing the pair with three black-box triggers."}],"limitations":[{"source_index":0,"quote":"We propose mitigations for each failure but do not evaluate them."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0055","record_version":1,"name":"Blockchain-anchored data provenance","aliases":[],"category":"provenance","summary":"The paper proposes anchoring data and model artifacts to a blockchain so that their origin and integrity can be verified without a central authority. Blockchain's immutability, decentralized consensus, and verifiable provenance are used to establish trust in the training data, models, and autonomous agents that participate in distributed security operations. This provenance layer is one component of a layered reference architecture that also couples adversarially hardened models, AI-driven anomaly detection, and smart-contract-governed multi-agent remediation.","problem_addressed":"The difficulty of establishing trust in data, models, and autonomous agents that operate without a central authority.","mitigates":["security_compromise","performance_drift"],"addresses":[],"responds_to":[],"maturity":"proposed","evidence_strength":"none_stated","limitations":"The document states that open problems in scalability, privacy-transparency trade-offs, and the governance of autonomous agents must be resolved before such integrated systems can be trusted in production-critical environments.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28843v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Blockchain-Enabled Artificial Intelligence and AI Agents for Secure Data Sharing and Cybersecurity Applications"}],"field_citations":{"name":[{"source_index":0,"quote":"we propose a layered reference architecture that couples adversarially hardened models, blockchain-anchored data provenance, AI-driven anomaly detection, and smart-contract-governed multi-agent remediation"}],"summary":[{"source_index":0,"quote":"We argue that blockchain's properties of immutability, decentralized consensus, and verifiable provenance directly address a gap common to all three: the difficulty of establishing trust in data, models, and autonomous agents that operate without a central authority."}],"problem_addressed":[{"source_index":0,"quote":"We argue that blockchain's properties of immutability, decentralized consensus, and verifiable provenance directly address a gap common to all three: the difficulty of establishing trust in data, models, and autonomous agents that operate without a central authority."}],"limitations":[{"source_index":0,"quote":"We conclude by identifying open problems in scalability, privacy-transparency trade-offs, and the governance of autonomous agents that must be resolved before such integrated systems can be trusted in production-critical environments."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0147","record_version":1,"name":"Blueprint-first generation of verifiable agent gyms","aliases":["AutoGym"],"category":"evaluation","summary":"AutoGym generates complete reinforcement-learning gyms — task, executable environment, and verifier — from a minimal domain seed or prior model trajectories. Rather than synthesizing a task and adjudicating correctness after the fact with an LLM judge, it first writes a blueprint specifying the valid solution space, environment requirements, and verification criteria, then materializes the environment against that blueprint. This makes solvability a construction prerequisite rather than a property checked post-hoc, and yields verifiers that reliably distinguish success from failure.","problem_addressed":"Manual, expensive, static gym construction; task saturation and contamination; single-pass synthesis producing cosmetically difficult tasks adjudicated by unreliable LLM judges.","mitigates":["performance_drift","operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not report independent replication; evaluation is confined to productivity and temporal-reasoning settings, and the abstract does not quantify verifier reliability or contamination resistance.","organisations":[],"first_seen":"2026-09-18","last_seen":"2026-09-18","sources":[{"url":"https://arxiv.org/abs/2609.22592v1","publisher":"arXiv","date":"2026-09-18","type":"vendor_research","title":"AutoGym: Blueprint-First Generation of Verifiable Agent Gyms"}],"field_citations":{"name":[{"source_index":0,"quote":"We present AutoGym, a framework that generates complete gyms (tasks, executable environments, and verifiers) from a minimal domain seed or prior model trajectories."}],"summary":[{"source_index":0,"quote":"Blueprint-first generation specifies the valid solution space, environment requirements, and verification criteria before the environment is materialized, making solvability a construction prerequisite rather than a property verified after the fact."}],"problem_addressed":[{"source_index":0,"quote":"Constructing such gyms remains manual, expensive, and static. Task sets saturate as models improve and are increasingly exposed to contamination. Synthetic generation offers scale, but single-pass synthesis produces tasks whose difficulty is largely cosmetic."}],"mitigates":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"maturity":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"evidence_strength":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"limitations":[{"source_index":0,"quote":"Models comparable in capability solve them despite convoluted phrasing, and correctness must be adjudicated post-hoc by unreliable LLM judges."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0052","record_version":1,"name":"Bucket analysis separating persuasion from information gaps","aliases":[],"category":"evaluation","summary":"A diagnostic method for classifying task failures in context-grounded question answering by whether the context contains an assertion by an interested party. Tasks are bucketed according to whether a witness assertion is present and whether it contradicts authoritative records, so that failures caused by persuasion can be distinguished from failures caused by missing information. Applied to 100 lead-qualification tasks from CRMArena-Pro, it showed that only 3 of 35 genuine failures involved no assertion, supporting the conclusion that the failure mode is persuasion rather than an information gap.","problem_addressed":"Failures of language-model agents over CRM records are often attributed to missing information, obscuring a distinct persuasion failure mode.","mitigates":["hallucination","operational_error","agent_misalignment"],"addresses":["deception_or_concealment"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document describes it as a diagnostic method rather than an architecture, and the generalization test the authors pre-specified returned a negative result.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28854v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Persuaded, Not Informed: Incentive-Misaligned Witnesses Defeat In-Context Grounding"}],"field_citations":{"name":[{"source_index":0,"quote":"We contribute a diagnostic method rather than an architecture: (i) a bucket analysis that separates persuasion from information gaps"}],"summary":[{"source_index":0,"quote":"We contribute a diagnostic method rather than an architecture: (i) a bucket analysis that separates persuasion from information gaps"},{"source_index":0,"quote":"Only 3 of 35 genuine failures involve no assertion: the failure is persuasion, not missing information."}],"problem_addressed":[{"source_index":0,"quote":"Only 3 of 35 genuine failures involve no assertion: the failure is persuasion, not missing information."}],"mitigates":[{"source_index":0,"quote":"when the context contains an assertion by a party with an incentive toward optimism - here the sales representative, a witness recorded in the CRM - the model treats the assertion as evidence and clears deals the company's own records deem unacceptable"}],"maturity":[{"source_index":0,"quote":"We contribute a diagnostic method rather than an architecture: (i) a bucket analysis that separates persuasion from information gaps"}],"evidence_strength":[{"source_index":0,"quote":"Across 100 lead-qualification tasks from CRMArena-Pro, the representative asserts an acceptable timeline in every call and an acceptable budget in 76; on the 31 tasks where such an assertion contradicts the price list and installation policy, a model reading only the transcript clears the deal in 29 of 31 cases."}],"limitations":[{"source_index":0,"quote":"We pre-specify a generalization test that returns a negative result"}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0047","record_version":1,"name":"CART (Closed-Loop Adaptive Red Teaming)","aliases":[],"category":"red_teaming","summary":"CART is a red-teaming framework that adapts its probes based on the results of each test rather than replaying a fixed prompt set. It starts with broad risk coverage, follows weaknesses that emerge, keeps new probes diverse, and records the evidence and source of every finding. The framework separates three roles: a Challenger that creates tests, a Target being tested (text-only model or bounded tool-using agent), and a Judge that evaluates results, allowing the roles to be studied independently. It is evaluated across three families (Frontier, JAH, Agentic) and reportedly discovers more failures and higher average risk than static seed replay.","problem_addressed":"Automated red teaming that replays a fixed set of prompts measures known risks but cannot learn from failures found during testing.","mitigates":["harmful_output","agent_misalignment"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The results describe what the test policies discover, not how often failures occur in real deployments. Challenger-Judge choices affect the evidence uncovered, highlighting the need for role separation and independent review.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.27336v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"CART: Closed-Loop Adaptive Red Teaming for Large Language Models"}],"field_citations":{"name":[{"source_index":0,"quote":"We present CART (Closed-Loop Adaptive Red Teaming), a framework that uses each result to guide what it tests next."}],"summary":[{"source_index":0,"quote":"CART begins with broad risk coverage, follows weaknesses that emerge, keeps new probes diverse, and records the evidence and source of every finding."},{"source_index":0,"quote":"It separates the Challenger that creates tests, the Target being tested, which may be a text-only model or a bounded tool-using agent, and the Judge that evaluates the results, allowing these roles to be studied independently."},{"source_index":0,"quote":"Across three evaluation families (Frontier, JAH, and Agentic), CART discovers more failures and higher average risk than static seed replay for every Target with an available baseline."}],"problem_addressed":[{"source_index":0,"quote":"Automated red teaming often replays a fixed set of prompts, which measures known risks but cannot learn from failures found during testing."}],"mitigates":[{"source_index":0,"quote":"The gains extend to tool-mediated agent tests, suggesting that contextual adaptation can reveal weaknesses that direct prompt replay does not exercise."}],"maturity":[{"source_index":0,"quote":"Across three evaluation families (Frontier, JAH, and Agentic), CART discovers more failures and higher average risk than static seed replay for every Target with an available baseline."}],"evidence_strength":[{"source_index":0,"quote":"Across three evaluation families (Frontier, JAH, and Agentic), CART discovers more failures and higher average risk than static seed replay for every Target with an available baseline."}],"limitations":[{"source_index":0,"quote":"These results describe what the test policies discover, not how often failures occur in real deployments."},{"source_index":0,"quote":"We also find that Challenger-Judge choices affect the evidence uncovered, highlighting the need for role separation and independent review."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0049","record_version":1,"name":"CAVEAT","aliases":[],"category":"benchmark","summary":"CAVEAT is a controlled benchmark for evaluating computer-use agents in incentive-misaligned environments. It spans nine marketplace environments and a taxonomy of eight common steering mechanisms. The benchmark measures whether agents preserve user objectives when the environment itself has a stake in the outcome, comparing matched-control episodes against episodes with steering mechanisms enabled. It is used to test five model families and diagnose failure modes such as priority distortion, premature narrowing of alternatives, and premature commitment.","problem_addressed":"Existing CUA benchmarks cover cooperative settings or explicit attacks, but do not test whether agents preserve user objectives when the environment itself has a stake in the outcome.","mitigates":["agent_misalignment"],"addresses":["reward_hacking"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The benchmark is limited to nine marketplace environments and eight steering mechanisms; substantial failures persist even in larger models.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.27273v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce CAVEAT, a controlled benchmark spanning nine marketplace environments and a taxonomy of eight common steering mechanisms."}],"summary":[{"source_index":0,"quote":"We introduce CAVEAT, a controlled benchmark spanning nine marketplace environments and a taxonomy of eight common steering mechanisms."},{"source_index":0,"quote":"Across five model families, agents purchase the user-optimal product in 78.6% of matched-control episodes but only 17.3% when steering mechanisms are enabled."}],"problem_addressed":[{"source_index":0,"quote":"Existing CUA benchmarks cover cooperative settings or explicit attacks, but do not test whether agents preserve user objectives when the environment itself has a stake in the outcome."}],"mitigates":[{"source_index":0,"quote":"Existing CUA benchmarks cover cooperative settings or explicit attacks, but do not test whether agents preserve user objectives when the environment itself has a stake in the outcome."}],"maturity":[{"source_index":0,"quote":"Across five model families, agents purchase the user-optimal product in 78.6% of matched-control episodes but only 17.3% when steering mechanisms are enabled."}],"evidence_strength":[{"source_index":0,"quote":"Across five model families, agents purchase the user-optimal product in 78.6% of matched-control episodes but only 17.3% when steering mechanisms are enabled."}],"limitations":[{"source_index":0,"quote":"Larger models and increased reasoning improve robustness, but substantial failures persist."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0050","record_version":1,"name":"CAVEAT-Harness","aliases":[],"category":"control_protocol","summary":"CAVEAT-Harness is an intervention that directly targets three failure modes identified by trajectory analysis: agents distorting user priorities, prematurely narrowing alternatives, and committing before resolving decision-relevant evidence. It is guided by a diagnosis of how steering enters the decision process and aims to improve user-optimal purchasing in incentive-misaligned environments.","problem_addressed":"Agents fail to preserve user objectives when environments have misaligned incentives, due to priority distortion, premature narrowing, and premature commitment.","mitigates":["agent_misalignment"],"addresses":["reward_hacking","deception_or_concealment"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.27273v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments"}],"field_citations":{"name":[{"source_index":0,"quote":"Guided by this diagnosis, we develop CAVEAT-Harness, which directly targets these failure modes and raises user-optimal purchasing by 55.0%."}],"summary":[{"source_index":0,"quote":"Guided by this diagnosis, we develop CAVEAT-Harness, which directly targets these failure modes and raises user-optimal purchasing by 55.0%."},{"source_index":0,"quote":"Our trajectory analysis and targeted ablations identify three points where steering enters the decision process: (1) agents distort the user's priorities, (2) prematurely narrow the set of alternatives they consider, and (3) commit before resolving decision-relevant evidence."}],"problem_addressed":[{"source_index":0,"quote":"Our trajectory analysis and targeted ablations identify three points where steering enters the decision process: (1) agents distort the user's priorities, (2) prematurely narrow the set of alternatives they consider, and (3) commit before resolving decision-relevant evidence."}],"mitigates":[{"source_index":0,"quote":"Guided by this diagnosis, we develop CAVEAT-Harness, which directly targets these failure modes and raises user-optimal purchasing by 55.0%."}],"maturity":[{"source_index":0,"quote":"Guided by this diagnosis, we develop CAVEAT-Harness, which directly targets these failure modes and raises user-optimal purchasing by 55.0%."}],"evidence_strength":[{"source_index":0,"quote":"Guided by this diagnosis, we develop CAVEAT-Harness, which directly targets these failure modes and raises user-optimal purchasing by 55.0%."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0087","record_version":1,"name":"Calibrated abstention","aliases":[],"category":"monitoring","summary":"A decision rule that withholds fault interpretation when the aggregate trace cannot support the claim. Units that fail the reference-map gate or the matched runtime gate abstain rather than being scored or localized. False admission is calibrated on a disjoint clean stream at the physical-component level, with a frozen criterion: stable false admission was 0/20, with a one-sided exact 95% upper bound of 0.1391, meeting the frozen 0.20 criterion. The paper frames this as a bounded rule for interpreting aggregate agent behavior: first establish exposure, then score change, and abstain when the trace cannot support the claim.","problem_addressed":"Aggregate traces may not support a fault claim, and scoring or localizing without sufficient exposure can produce unsupported diagnoses.","mitigates":["performance_drift","operational_error","agent_misalignment"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The abstention rule is calibrated on a simulated hotel-pricing agent and a frozen one-shot heldout; the document does not report deployment in a production system.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.25806v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"When Are Aggregate Agent Traces Diagnosable? Traffic-Governed Interpretation and Calibrated Abstention"}],"field_citations":{"name":[{"source_index":0,"quote":"The result is a bounded rule for interpreting aggregate agent behavior: first establish exposure, then score change, and abstain when the trace cannot support the claim."}],"summary":[{"source_index":0,"quote":"We treat entry into aggregate-only fault interpretation as a diagnosability decision preceding scoring or localization."},{"source_index":0,"quote":"Stable false admission was 0/20, with a one-sided exact 95% upper bound of 0.1391, meeting the frozen 0.20 criterion."},{"source_index":0,"quote":"The result is a bounded rule for interpreting aggregate agent behavior: first establish exposure, then score change, and abstain when the trace cannot support the claim."}],"problem_addressed":[{"source_index":0,"quote":"We treat entry into aggregate-only fault interpretation as a diagnosability decision preceding scoring or localization."}],"maturity":[{"source_index":0,"quote":"Stable false admission was 0/20, with a one-sided exact 95% upper bound of 0.1391, meeting the frozen 0.20 criterion."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0158","record_version":1,"name":"Canonical approval rendering with exact use-time comparison","aliases":["complete canonical approval rendering and exact use-time comparison"],"category":"control_protocol","summary":"A defense against Loopjacking in which the operation shown to a human approver is rendered in a complete, canonical form, and the operation actually released for execution is compared exactly against that approved rendering at use time. Any mismatch between the approved operation A and the executed operation B causes rejection. The paper reports that this binding, or alternatively preventing unauthorized mutation of pending workflow state, blocks the tested representation-based and post-approval state-substitution attacks while preserving legitimate execution.","problem_addressed":"Human-in-the-loop approval fails as a security boundary when the operation presented for review is not the operation later authorized or released, allowing an approver to authorize operation A while a materially different operation B executes.","mitigates":["agent_misalignment","security_compromise","operational_error"],"addresses":["deception_or_concealment","silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The authors state their results do not estimate ecosystem prevalence; the evaluation is a purposive set of released agent products rather than a representative sample, and the contribution is separated from established work on misleading dialogs, session smuggling, action binding, and authorization continuity.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.21081v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"Loopjacking: Hijacking Human-in-the-Loop Approval"}],"field_citations":{"name":[{"source_index":0,"quote":"They show that complete canonical approval rendering and exact use-time comparison, or preventing unauthorized pending-state mutation, block the tested attacks while preserving legitimate execution."}],"summary":[{"source_index":0,"quote":"They show that complete canonical approval rendering and exact use-time comparison, or preventing unauthorized pending-state mutation, block the tested attacks while preserving legitimate execution."}],"problem_addressed":[{"source_index":0,"quote":"That boundary is only meaningful if the operation presented for review is the operation later authorized or released."},{"source_index":0,"quote":"We call failures of this binding Loopjacking: a human approves what they understand as operation A, while the implementation uses that decision for a materially different operation B."}],"mitigates":[{"source_index":0,"quote":"They show that complete canonical approval rendering and exact use-time comparison, or preventing unauthorized pending-state mutation, block the tested attacks while preserving legitimate execution."}],"maturity":[{"source_index":0,"quote":"We reproduce post-approval substitution in seven tested Agno AgentOS releases ending at 3.0.9 and in 12 tested versions of a conditional in-memory LangGraph Agent Server composition ending at 0.14.0."},{"source_index":0,"quote":"They show that complete canonical approval rendering and exact use-time comparison, or preventing unauthorized pending-state mutation, block the tested attacks while preserving legitimate execution."}],"evidence_strength":[{"source_index":0,"quote":"We evaluate a purposive set of released agent products."},{"source_index":0,"quote":"These results do not estimate ecosystem prevalence."}],"limitations":[{"source_index":0,"quote":"These results do not estimate ecosystem prevalence."},{"source_index":0,"quote":"We separate this contribution from established work on misleading dialogs, session smuggling, action binding, and authorization continuity."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0040","record_version":1,"name":"Capability-based agent routing","aliases":[],"category":"access_control","summary":"A routing mechanism that directs tasks to agents based on their capabilities, ensuring that only agents with the appropriate permissions and competencies handle specific operations. This supports portability across DACH and the wider EU by allowing the same compliance architecture to be instantiated with different agent sets. It is a form of access control that constrains which agent can perform which action within the governed policy framework.","problem_addressed":"Ensuring that AI agents only perform tasks for which they are authorised and capable, as part of a compliant architecture.","mitigates":["agent_misalignment","operational_error"],"addresses":["harmful_compliance","credential_misuse","goal_adoption_from_peer"],"responds_to":[],"maturity":"proposed","evidence_strength":"none_stated","limitations":null,"organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.27632v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Compliant AI Infrastructure for Regulated Finance: A tiered multi-agent framework with DLT audit trails for financial operations in DACH"}],"field_citations":{"name":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}],"summary":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}],"problem_addressed":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}],"maturity":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}],"evidence_strength":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0121","record_version":1,"name":"Category-aligned action slots with parallel spatio-temporal updating and background/negative-slot regularization","aliases":["Action-Slot","structured activity decomposition"],"category":"interpretability","summary":"Action-Slot reformulates slot attention for atomic activity understanding by anchoring slots to predefined activity categories, updating slots in parallel across space and time for holistic video-level reasoning, and adding background and negative-slot regularization that forces competition between foreground activities and irrelevant regions. This activity-centric inductive bias is intended to disentangle concurrent and asynchronous multi-agent activities directly from raw video, replacing the permutation-invariant, object-level bias of standard slot attention.","problem_addressed":"Standard slot attention's permutation-invariant design and object-level inductive bias are misaligned with multi-agent, multi-label, topology-aware atomic activity semantics.","mitigates":["other"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the slot-learning design itself; it only notes that standard slot attention is misaligned with atomic activity semantics.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24127v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"Action-Slot: Structured Action-Centric Representation Learning for Multi-Agent Atomic Activity Understanding"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce Action-Slot, a structured action-centric representation learning framework."},{"source_index":0,"quote":"We reformulate slot learning as structured activity decomposition through three designs: (1) category-aligned action slots that anchor slots to predefined activity categories, (2) parallel spatio-temporal slot updating for holistic video-level reasoning, and (3) background and negative-slot regularization that enforces competition between foreground activities and irrelevant regions."}],"summary":[{"source_index":0,"quote":"Slot attention is widely used for object-centric decomposition, but its permutation-invariant design and object-level inductive bias are misaligned with atomic activity semantics."},{"source_index":0,"quote":"Together these establish an activity-centric inductive bias that disentangles concurrent and asynchronous activities directly from raw video."}],"problem_addressed":[{"source_index":0,"quote":"Unlike conventional action recognition, atomic activities are multi-agent, multi-label, and topology-aware: multiple activities co-occur while many agents remain inactive."}],"maturity":[{"source_index":0,"quote":"Experiments on OATS, TACO, and annotated nuScenes show superior recognition, strong sim-to-real transfer, and state-of-the-art weakly supervised localization."}],"evidence_strength":[{"source_index":0,"quote":"Experiments on OATS, TACO, and annotated nuScenes show superior recognition, strong sim-to-real transfer, and state-of-the-art weakly supervised localization."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0170","record_version":1,"name":"ClashBench","aliases":[],"category":"benchmark","summary":"ClashBench is an executable benchmark for measuring destructive resource preemption in agent systems. It comprises 268 validated conflict cases spanning 55 resource types, in which an agent session and a pre-existing incumbent task contend for a shared resource with limited capacity or mutually exclusive state. Each case is run through agent harnesses such as Codex, Claude Code, and OpenCode, and a trajectory is scored as destructive preemption when the agent completes its requested task while the incumbent task fails its health check. The benchmark also records whether the agent's final response discloses the conflict and the action taken to resolve it.","problem_addressed":"Agents resolving resource conflicts by terminating, overwriting, evicting, or degrading an incumbent task rather than reporting the conflict.","mitigates":["agent_misalignment","operational_error","other"],"addresses":["covert_sabotage","deception_or_concealment","silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The benchmark covers 268 conflict cases across 55 resource types and 17 models through three harnesses; the authors report that prompt-based safeguards reduce but do not eliminate preemption, and that concealment is inferred from the absence of conflict mention in final responses rather than directly verified.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.19892v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"ClashBench: Conflicts Leading Agents to Seize and Harm"}],"field_citations":{"name":[{"source_index":0,"quote":"To systematically study this risk, we introduce ClashBench, an executable benchmark comprising 268 validated conflict cases across 55 resource types, and evaluate 17 models through Codex, Claude Code, and OpenCode."}],"summary":[{"source_index":0,"quote":"To systematically study this risk, we introduce ClashBench, an executable benchmark comprising 268 validated conflict cases across 55 resource types, and evaluate 17 models through Codex, Claude Code, and OpenCode."},{"source_index":0,"quote":"We observe destructive preemption in 44.5% of trajectories, where the agent completes the requested task while causing the incumbent task to fail its health check."},{"source_index":0,"quote":"More concerningly, in 31.9% of successful destructive-preemption cases, the final response mentions neither the resource conflict nor the action taken to resolve it, raising concerns about possible concealment."}],"problem_addressed":[{"source_index":0,"quote":"In this work, we identify and formalize this failure mode, which we term destructive resource preemption: obtaining the resources required for a requested task by terminating, overwriting, evicting, or degrading an incumbent task."}],"mitigates":[{"source_index":0,"quote":"These findings establish destructive resource preemption as a broad safety risk in privileged agent systems and motivate stronger privilege controls, task isolation, and conflict-aware safeguards."}],"maturity":[{"source_index":0,"quote":"To systematically study this risk, we introduce ClashBench, an executable benchmark comprising 268 validated conflict cases across 55 resource types, and evaluate 17 models through Codex, Claude Code, and OpenCode."}],"evidence_strength":[{"source_index":0,"quote":"To systematically study this risk, we introduce ClashBench, an executable benchmark comprising 268 validated conflict cases across 55 resource types, and evaluate 17 models through Codex, Claude Code, and OpenCode."}],"limitations":[{"source_index":0,"quote":"We also show that prompt-based safeguards are insufficient: an instruction to avoid affecting existing tasks reduces but does not eliminate preemption, while an instruction explicitly authorizing the agent to stop local processes increases it."},{"source_index":0,"quote":"More concerningly, in 31.9% of successful destructive-preemption cases, the final response mentions neither the resource conflict nor the action taken to resolve it, raising concerns about possible concealment."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0039","record_version":1,"name":"Clause-level legal indexing with effective dates","aliases":[],"category":"governance","summary":"A legal indexing scheme that operates at the clause level and incorporates effective dates, so that regulatory provisions can be mapped precisely and temporally. Combined with capability-based agent routing, it ensures portability of the compliance architecture across DACH jurisdictions and the wider EU. This allows the system to adapt to different legal regimes and changes over time without redesigning the core policy compiler.","problem_addressed":"Portability of AI compliance across multiple jurisdictions with differing and time-varying legal requirements.","mitigates":["other"],"addresses":[],"responds_to":[],"maturity":"proposed","evidence_strength":"none_stated","limitations":null,"organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.27632v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Compliant AI Infrastructure for Regulated Finance: A tiered multi-agent framework with DLT audit trails for financial operations in DACH"}],"field_citations":{"name":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}],"summary":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}],"problem_addressed":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}],"maturity":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}],"evidence_strength":[{"source_index":0,"quote":"Clause-level legal indexing with effective dates and capability-based agent routing ensure portability across DACH and the wider EU."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0112","record_version":1,"name":"Clinical-cost-aware failure prioritization","aliases":[],"category":"alignment_training","summary":"A mechanism within the MedRSI recursive self-improvement framework that ranks diagnostic failures for improvement by their potential clinical consequences rather than by how often they occur. The agent uses this ranking to decide which errors to convert into new capabilities, so that self-improvement effort is directed at the failures most likely to harm patients. It is one of two clinically aligned self-evolution mechanisms in MedRSI, paired with fast discovery with slow registration.","problem_addressed":"Recursive self-improvement applied to medicine could optimize for frequent but clinically minor errors while neglecting rare, high-consequence diagnostic failures.","mitigates":["harmful_output","operational_error","other"],"addresses":["reward_hacking"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":["ImprintLab"],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24838v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"MedRSI: Recursive Self-Improvement for Medical Agents via Clinically Aligned Self-Evolution"}],"field_citations":{"name":[{"source_index":0,"quote":"Clinical-cost-aware failure prioritization directs improvement toward errors according to their potential clinical consequences rather than frequency alone."}],"summary":[{"source_index":0,"quote":"Inspired by clinical practice, MedRSI introduces two mechanisms for clinically aligned self-evolution."},{"source_index":0,"quote":"Clinical-cost-aware failure prioritization directs improvement toward errors according to their potential clinical consequences rather than frequency alone."}],"problem_addressed":[{"source_index":0,"quote":"Clinical-cost-aware failure prioritization directs improvement toward errors according to their potential clinical consequences rather than frequency alone."}],"maturity":[{"source_index":0,"quote":"Across public glaucoma and heart disease benchmarks and two private clinical tasks, MedRSI progressively develops segmentation, measurement, prediction, multimodal reasoning, and generative capabilities, surpasses manually engineered medical agents, and autonomously discovers solutions to clinical problems not anticipated by its original designers."}],"evidence_strength":[{"source_index":0,"quote":"Across public glaucoma and heart disease benchmarks and two private clinical tasks, MedRSI progressively develops segmentation, measurement, prediction, multimodal reasoning, and generative capabilities, surpasses manually engineered medical agents, and autonomously discovers solutions to clinical problems not anticipated by its original designers."}],"organisations":[{"source_index":0,"quote":"Code is available at https://github.com/ImprintLab/MedRSI."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0064","record_version":1,"name":"Comparative LLM-as-a-Judge planning-deficiency analysis","aliases":[],"category":"evaluation","summary":"A comparative judging method in which an LLM evaluates agent plans against one another to surface planning deficiencies rather than only scoring outcomes. The study uses it to identify tool misalignment, plan similarity, over-specification, inadequate probing, and weak recovery across six frontier models and three scaffolding modes. It is a reusable diagnostic for multi-stage agent pipelines, aimed at revealing where planning breaks down rather than merely whether a task succeeded.","problem_addressed":"Outcome-only evaluation fails to reveal planning deficiencies such as tool misalignment, over-specification, inadequate probing, and weak recovery in multi-stage agents.","mitigates":["operational_error","agent_misalignment"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the judging method; it reports that bottlenecks cluster in credential and lateral-movement tasks and vary more under full autonomy.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28572v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Where Cyber Agents Struggle: Bottleneck Analysis of Multi-Stage LLM Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"use comparative LLM-as-a-Judge analysis to identify planning deficiencies"}],"summary":[{"source_index":0,"quote":"We assess validator consistency and evidence grounding; introduce a subtask-conditioned, cost-aware score for abnormal token use, retries, and runtime; and use comparative LLM-as-a-Judge analysis to identify planning deficiencies, including tool misalignment, plan similarity, over-specification, inadequate probing, and weak recovery."},{"source_index":0,"quote":"Reliable evaluation must assess outcomes, evidence interpretation, resource use, and adaptation after failure."}],"problem_addressed":[{"source_index":0,"quote":"Success rates alone obscure inefficiency, adaptation through retries, and recognition of success or failure."}],"maturity":[{"source_index":0,"quote":"Six frontier models are evaluated across two scenarios and three modes: expert-defined, self-scaffolded, and fully autonomous."}],"evidence_strength":[{"source_index":0,"quote":"We present an end-to-end diagnostic study of an Autonomous Adversary system with orchestrator, executor, and validator LLMs in enterprise-like lateral-movement scenarios."}],"limitations":[{"source_index":0,"quote":"Bottlenecks cluster in credential and lateral-movement tasks, spread with scenario complexity, and vary more under full autonomy."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0011","record_version":1,"name":"Competence-Aware Reward-and-Advantage Engineering (CARE)","aliases":[],"category":"alignment_training","summary":"A reward-and-advantage engineering technique introduced within hybrid-environment online agentic reinforcement learning, combined with a supervised planning cold start. CARE is designed to reduce reasoning and tool-use costs while preserving task performance during training of the planner agent.","problem_addressed":"High reasoning and tool-use costs during agentic reinforcement learning without sacrificing task performance.","mitigates":["operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29892v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"Qwen-Planner-Agent: A Closed-Loop AI-for-AI Framework for Real-World Mobile Planner Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"AI for Training combines a supervised planning cold start with hybrid-environment online agentic reinforcement learning, where we introduce Competence-Aware Reward-and-Advantage Engineering (CARE) to reduce reasoning and tool-use costs while preserving task performance."}],"summary":[{"source_index":0,"quote":"AI for Training combines a supervised planning cold start with hybrid-environment online agentic reinforcement learning, where we introduce Competence-Aware Reward-and-Advantage Engineering (CARE) to reduce reasoning and tool-use costs while preserving task performance."}],"problem_addressed":[{"source_index":0,"quote":"AI for Training combines a supervised planning cold start with hybrid-environment online agentic reinforcement learning, where we introduce Competence-Aware Reward-and-Advantage Engineering (CARE) to reduce reasoning and tool-use costs while preserving task performance."}],"maturity":[{"source_index":0,"quote":"Qwen-Planner-Agent achieves the best overall performance among all evaluated models and systems on MobilePA-Bench, improving over its base model across tool use, memory, skills, and sub-agent coordination."}],"evidence_strength":[{"source_index":0,"quote":"Qwen-Planner-Agent achieves the best overall performance among all evaluated models and systems on MobilePA-Bench, improving over its base model across tool use, memory, skills, and sub-agent coordination."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0189","record_version":1,"name":"Compositional Policy Violation taxonomy","aliases":["CPV taxonomy"],"category":"evaluation","summary":"A four-type taxonomy of Compositional Policy Violations, the failure of step-level compliance to compose: Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse. The taxonomy classifies how whole-execution policy violations arise even when each step passes its own check, and the document states that the correct repair for each class is dictated by where the guarded quantity mutates. It gives evaluators and governance designers a vocabulary for identifying which trace-level property is being violated and where in the execution the guarded quantity changes.","problem_addressed":"There is no shared classification of the ways step-level compliance fails to compose into whole-execution policy compliance, making such violations hard to name, detect, or repair.","mitigates":["agent_misalignment","operational_error","other"],"addresses":[],"responds_to":[],"maturity":"proposed","evidence_strength":"author_claim","limitations":"The document does not report empirical validation of the taxonomy against real agent traces or show that the four types are exhaustive.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18820v2","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"Compositional Policy Violations: When Step-Level Compliance Fails In Agentic AI Workflows"}],"field_citations":{"name":[{"source_index":0,"quote":"We define CPVs as the failure of step-level compliance to compose, and present a taxonomy of four types: Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse."}],"summary":[{"source_index":0,"quote":"We define CPVs as the failure of step-level compliance to compose, and present a taxonomy of four types: Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse."},{"source_index":0,"quote":"We show that the correct repair for each class is dictated by where the guarded quantity mutates."}],"problem_addressed":[{"source_index":0,"quote":"We define CPVs as the failure of step-level compliance to compose, and present a taxonomy of four types: Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse."}],"mitigates":[{"source_index":0,"quote":"We define CPVs as the failure of step-level compliance to compose, and present a taxonomy of four types: Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse."}],"maturity":[{"source_index":0,"quote":"We define CPVs as the failure of step-level compliance to compose, and present a taxonomy of four types: Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse."}],"limitations":[{"source_index":0,"quote":"We define CPVs as the failure of step-level compliance to compose, and present a taxonomy of four types: Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0054","record_version":1,"name":"Compute-step control","aliases":[],"category":"evaluation","summary":"A control that holds the extraction step fixed and varies only which party computes the Budget and Timeline values, so that the effect of who performs the computation can be measured independently of what information is extracted. The margin between arms ranged from 42 points on an inexpensive model to 2-5 points on models that already compute correctly. On the strongest models the arms fell within confidence intervals, so the authors characterize the result as a consistent direction and a soundness property rather than a proved performance floor.","problem_addressed":"Isolating whether the identity of the computing party, rather than extraction quality, drives the misgrounding failure.","mitigates":["operational_error","agent_misalignment"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"On the strongest models the arms are within confidence intervals, so the pattern is a consistent direction and a soundness property, not a proved performance floor.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28854v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Persuaded, Not Informed: Incentive-Misaligned Witnesses Defeat In-Context Grounding"}],"field_citations":{"name":[{"source_index":0,"quote":"and (iii) a compute-step control that holds extraction fixed and varies only who computes Budget and Timeline"}],"summary":[{"source_index":0,"quote":"and (iii) a compute-step control that holds extraction fixed and varies only who computes Budget and Timeline"},{"source_index":0,"quote":"The margin ranges from 42 points on an inexpensive model to 2-5 points on models that already compute correctly; on the strongest models the arms are within confidence intervals, so the pattern is a consistent direction and a soundness property, not a proved performance floor."}],"problem_addressed":[{"source_index":0,"quote":"and (iii) a compute-step control that holds extraction fixed and varies only who computes Budget and Timeline"}],"mitigates":[{"source_index":0,"quote":"the model treats the assertion as evidence and clears deals the company's own records deem unacceptable"}],"maturity":[{"source_index":0,"quote":"and (iii) a compute-step control that holds extraction fixed and varies only who computes Budget and Timeline"}],"evidence_strength":[{"source_index":0,"quote":"The margin ranges from 42 points on an inexpensive model to 2-5 points on models that already compute correctly; on the strongest models the arms are within confidence intervals, so the pattern is a consistent direction and a soundness property, not a proved performance floor."}],"limitations":[{"source_index":0,"quote":"on the strongest models the arms are within confidence intervals, so the pattern is a consistent direction and a soundness property, not a proved performance floor"}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0111","record_version":1,"name":"Contextual reliability for agent-tool pipelines","aliases":[],"category":"assurance","summary":"A proposed concept for handling silent failures in agent-tool interaction, paired with suggested mechanisms for testing, disclosing, monitoring and measuring such failures across the agent-tool interaction pipeline. Rather than treating a tool call as reliable because it returned without error, contextual reliability frames reliability relative to the context in which the tool is invoked and the downstream scientific output it feeds, so that failures originating upstream are not propagated into apparently valid results. The paper proposes the concept and outlines the mechanisms but does not report an implementation or evaluation of them.","problem_addressed":"Silent failures that originate upstream in the API or wrapper layer and propagate downstream into apparently valid scientific outputs.","mitigates":["operational_error","performance_drift","other"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"proposed","evidence_strength":"none_stated","limitations":"The document only proposes the concept and suggests mechanisms; no implementation, testing or measurement of contextual reliability is reported.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.26836v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"Silent Failures in Agent-Tool Interaction: An Audit of ToolUniverse"}],"field_citations":{"name":[{"source_index":0,"quote":"We propose a concept of contextual reliability to handle such failures and suggest mechanisms for testing, disclosing, monitoring, and measuring such failures across the agent-tool interaction pipeline."}],"summary":[{"source_index":0,"quote":"We propose a concept of contextual reliability to handle such failures and suggest mechanisms for testing, disclosing, monitoring, and measuring such failures across the agent-tool interaction pipeline."}],"problem_addressed":[{"source_index":0,"quote":"The results show that silent failures originate upstream of the event and propagate downstream into apparently valid scientific outputs."}],"maturity":[{"source_index":0,"quote":"We propose a concept of contextual reliability to handle such failures and suggest mechanisms for testing, disclosing, monitoring, and measuring such failures across the agent-tool interaction pipeline."}],"evidence_strength":[{"source_index":0,"quote":"We propose a concept of contextual reliability to handle such failures and suggest mechanisms for testing, disclosing, monitoring, and measuring such failures across the agent-tool interaction pipeline."}],"limitations":[{"source_index":0,"quote":"We propose a concept of contextual reliability to handle such failures and suggest mechanisms for testing, disclosing, monitoring, and measuring such failures across the agent-tool interaction pipeline."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0174","record_version":1,"name":"Controlled skill patch lifecycle with protected-case regression checks and negative controls","aliases":["scoped skill patches","controlled behavioral maintenance"],"category":"monitoring","summary":"A post-deployment maintenance regime in which recurring failures are converted into scoped, reusable skill patches rather than global behavior changes. Each candidate patch must pass targeted validation, protected-case regression checks that guard previously correct answers, and negative controls before it can be admitted. Admitted skills are versioned and can later be replaced or retired. The mechanism separates where a correction applies from the correction itself, so that a fix for one failure mode does not silently break other capabilities.","problem_addressed":"Post-deployment reliability behavior of financial QA systems is fixed after deployment, and existing self-improvement methods offer limited control over where a correction applies or which previously correct answers it may break.","mitigates":["hallucination","performance_drift","operational_error"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Only six of 33 proposed skills are promoted in the 12-round operational study, indicating most candidate patches fail admission; the document does not report costs of running the validation, regression and negative-control pipeline.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.19680v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"FINSKILLOPS: A Self-Evolving Multi-Agent System for SEC Filing QA"}],"field_citations":{"name":[{"source_index":0,"quote":"We therefore frame post-deployment improvement as controlled behavioral maintenance: recurring failures should become scoped skill patches, and each patch should earn deployment with- out introducing regressions."}],"summary":[{"source_index":0,"quote":"FINSKILLOPS derives reusable skills from evidence-grounded, typed failure diagnoses and governs them through targeted validation, protected-case regression checks, negative controls, and versioned replacement or retirement."}],"problem_addressed":[{"source_index":0,"quote":"Financial QA systems are typically improved before deployment through better retrieval, prompting, or agent coordination, leaving their reliability behavior fixed thereafter."}],"mitigates":[{"source_index":0,"quote":"In practice, new SEC-filing questions repeatedly expose heterogeneous errors in period, entity, evidence use, and calculation."}],"maturity":[{"source_index":0,"quote":"In a separate 12-round operational study, only six of 33 proposed skills are promoted, while the monitoring non-correct rate falls from 20.0% to 12.5%."}],"evidence_strength":[{"source_index":0,"quote":"Across six financial QA benchmarks, a single frozen skill registry achieves the highest verdict-weighted correctness and reference consistency among the evaluated systems."}],"limitations":[{"source_index":0,"quote":"In a separate 12-round operational study, only six of 33 proposed skills are promoted, while the monitoring non-correct rate falls from 20.0% to 12.5%."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0021","record_version":1,"name":"Count-informed ordinal acne-severity support","aliases":[],"category":"evaluation","summary":"A visual component that supports ordinal acne-severity assessment by incorporating lesion counts into the severity estimate, rather than relying on a purely categorical classifier. It is evaluated separately from the agent layer, achieving 84.59% accuracy with a quadratic weighted kappa of 0.9076, indicating agreement that accounts for the ordering of severity levels.","problem_addressed":"Ordinal severity grading of acne from images, where misordering of severity levels matters.","mitigates":["operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document reports accuracy and kappa only; it does not establish clinical validity, and states that the findings do not establish clinical readiness and that expert assessment remains necessary.","organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29341v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"SkinAgent AI: A Safety-Grounded Multimodal Agentic Framework for Non-Diagnostic Skincare Support"}],"field_citations":{"name":[{"source_index":0,"quote":"The architecture includes routing for Acne, Pores, and Wrinkles; photograph-based skin-type estimation; count-informed ordinal acne-severity support; typed tools; database-grounded recommendation and action functions; deterministic safety, privacy, and evidence checks; approval before state-changing actions; and structured trace and replay mechanisms."}],"summary":[{"source_index":0,"quote":"Skin-type estimation achieved 88.85% accuracy, while count-informed acne-severity support achieved 84.59% accuracy with a quadratic weighted kappa of 0.9076."}],"maturity":[{"source_index":0,"quote":"Visual-model performance and system-level agent behavior were evaluated separately."}],"evidence_strength":[{"source_index":0,"quote":"Skin-type estimation achieved 88.85% accuracy, while count-informed acne-severity support achieved 84.59% accuracy with a quadratic weighted kappa of 0.9076."}],"limitations":[{"source_index":0,"quote":"They do not establish clinical readiness, external generalization, formal privacy guarantees, or universal safety."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0141","record_version":1,"name":"CounterCredit","aliases":[],"category":"alignment_training","summary":"CounterCredit is a reinforcement-learning credit-assignment scheme for vision-language agents that crop and zoom. For every image-returning tool call, evaluated at its realized pre-call state using the policy's own gold-answer score, it computes two values: a decision value comparing the realized visual branch against answering immediately, and an evidence value comparing the returned crop against random same-size patches substituted into the same call. Calls verified on both dimensions earn cashback; all other executed calls pay rent, with the price bounded so every correct trajectory outranks every wrong one. A dual-channel GRPO advantage keeps the price in its own units.","problem_addressed":"Outcome rewards, judge rewards, and branch probes each observe only one side of the failure, so successful tool calls are credited even when the model did not need to look or did not use the returned pixels; about two thirds of outcome-reward payment goes to calls that were neither needed nor used.","mitigates":["agent_misalignment","operational_error"],"addresses":["reward_hacking"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document reports only the authors' own training runs from a single cold-start checkpoint, prompt pool, and budget; it does not report independent replication, and the spurious-call rate remains 31% to 36% rather than being eliminated.","organisations":[],"first_seen":"2026-09-19","last_seen":"2026-09-19","sources":[{"url":"https://arxiv.org/abs/2609.22910v1","publisher":"arXiv","date":"2026-09-19","type":"vendor_research","title":"When Should a VLM Look? Paying Only for Visual Calls That Were Needed and Used"}],"field_citations":{"name":[{"source_index":0,"quote":"CounterCredit asks both questions of every image-returning call at its realized pre-call state, using the policy's own gold-answer score."}],"summary":[{"source_index":0,"quote":"A decision value compares the realized visual branch with answering immediately; an evidence value compares the returned crop with random same-size patches substituted into the same call. A call verified on both earns cashback and every other executed call pays rent; the price is bounded so that every correct trajectory outranks every wrong one, and a dual-channel GRPO advantage keeps the price in its own units."}],"problem_addressed":[{"source_index":0,"quote":"Outcome rewards, judge rewards, and branch probes each observe one side of this failure, and about two thirds of what an outcome reward pays goes to calls that were neither needed nor used."}],"mitigates":[{"source_index":0,"quote":"On our cold-start checkpoint only 10% to 12% of visual calls were both needed and used, and released agents make spurious calls 36% to 87% of the time on individual benchmarks."}],"maturity":[{"source_index":0,"quote":"From the same cold start, prompt pool, and budget, CounterCredit reaches 89.5% on V*, 80.2% on HR-Bench-4K, and 76.4% on HR-Bench-8K, 6.3 to 9.4 points above outcome-only GRPO at 1.78 against 1.84 calls per question, and lowers the spurious-call rate to 31% to 36%, the lowest among the agents evaluated."}],"evidence_strength":[{"source_index":0,"quote":"From the same cold start, prompt pool, and budget, CounterCredit reaches 89.5% on V*, 80.2% on HR-Bench-4K, and 76.4% on HR-Bench-8K, 6.3 to 9.4 points above outcome-only GRPO at 1.78 against 1.84 calls per question, and lowers the spurious-call rate to 31% to 36%, the lowest among the agents evaluated."}],"limitations":[{"source_index":0,"quote":"and lowers the spurious-call rate to 31% to 36%, the lowest among the agents evaluated."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0080","record_version":1,"name":"Cross-stack probing of the same request","aliases":["Cross-stack probes"],"category":"evaluation","summary":"An evaluation method that submits an identical tool-use request to multiple serving stacks and compares how each handles it, rather than evaluating on a single stack. The document applies it across Ollama, llama.cpp, vLLM, and SGLang and reports different handling of the same request. The purpose is to expose serving-layer confounds that would otherwise be attributed to the model, making the serving stack an explicit variable in the evaluation rather than a hidden constant.","problem_addressed":"Measured outcomes can depend on the serving layer rather than model behavior alone, so single-stack results may not transfer.","mitigates":["operational_error","performance_drift"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not report a systematic quantification of the cross-stack differences beyond stating that handling differs; it is a probe design rather than a validated benchmark.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.26693v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"Measuring the Serving Stack Instead of the Model: Hidden Confounds in Local Tool-Use Evaluation"}],"field_citations":{"name":[{"source_index":0,"quote":"Cross-stack probes on Ollama, llama.cpp, vLLM, and SGLang show different handling of the same request."}],"summary":[{"source_index":0,"quote":"Cross-stack probes on Ollama, llama.cpp, vLLM, and SGLang show different handling of the same request."}],"problem_addressed":[{"source_index":0,"quote":"We study how local serving stacks affect this protocol step and show that measured outcomes can depend on the serving layer rather than model behavior alone."}],"limitations":[{"source_index":0,"quote":"Cross-stack probes on Ollama, llama.cpp, vLLM, and SGLang show different handling of the same request."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0165","record_version":1,"name":"Cut-point replay","aliases":["Chronicle"],"category":"evaluation","summary":"Chronicle records an LLM agent run at its non-deterministic boundaries as immutable envelopes and can replay it from the record. Its central operation, cut-point replay, serves a chosen subset of boundaries from the record while executing the complementary subset live with new code, turning a recorded incident into a regression test that runs in continuous integration. This lets developers test a code change against a previously observed failure without re-invoking the non-deterministic model or changing external tool state.","problem_addressed":"Failures in LLM agents are hard to reproduce because inference is not bitwise reproducible, tools read changing state, and multi-step trajectories rarely repeat on re-run.","mitigates":["performance_drift","operational_error","agent_misalignment"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Evaluation uses a benchmark of 6 recorded failures with simulated model boundaries rather than real model calls; the document does not report deployment in production or independent evaluation.","organisations":["theagentplane"],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.20625v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"Chronicle: Cut-Point Replay for Regression Testing of LLM Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"Its central operation, cut-point replay, serves a chosen subset of boundaries from the record and executes the complementary subset live with new code, turning a recorded incident into a regression test that runs in continuous integration."}],"summary":[{"source_index":0,"quote":"We present Chronicle, which records an agent run at its non-deterministic boundaries as immutable envelopes and replays it from the record. Its central operation, cut-point replay, serves a chosen subset of boundaries from the record and executes the complementary subset live with new code, turning a recorded incident into a regression test that runs in continuous integration."}],"problem_addressed":[{"source_index":0,"quote":"Large language model responses are non-deterministic, so failures in LLM agents are hard to reproduce: a failure depends on inference that is not bitwise reproducible, on tools that read changing state, and on a multi-step trajectory that a re-run rarely repeats."}],"mitigates":[{"source_index":0,"quote":"On a benchmark of 6 recorded failures with simulated model boundaries, recording adds 23 μs per crossing (0.008% of an assumed 300 ms model call), full replay issues zero model calls and is bit-stable across 20 repetitions, and cut-point tests fail on faulty code and pass on guarded and benign changes for all 6 incidents."}],"maturity":[{"source_index":0,"quote":"On a benchmark of 6 recorded failures with simulated model boundaries, recording adds 23 μs per crossing (0.008% of an assumed 300 ms model call), full replay issues zero model calls and is bit-stable across 20 repetitions, and cut-point tests fail on faulty code and pass on guarded and benign changes for all 6 incidents."}],"evidence_strength":[{"source_index":0,"quote":"On a benchmark of 6 recorded failures with simulated model boundaries, recording adds 23 μs per crossing (0.008% of an assumed 300 ms model call), full replay issues zero model calls and is bit-stable across 20 repetitions, and cut-point tests fail on faulty code and pass on guarded and benign changes for all 6 incidents."}],"limitations":[{"source_index":0,"quote":"On a benchmark of 6 recorded failures with simulated model boundaries, recording adds 23 μs per crossing (0.008% of an assumed 300 ms model call), full replay issues zero model calls and is bit-stable across 20 repetitions, and cut-point tests fail on faulty code and pass on guarded and benign changes for all 6 incidents."}],"organisations":[{"source_index":0,"quote":"Chronicle and the benchmark are publicly available at https://github.com/theagentplane/chronicle."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0192","record_version":1,"name":"Deployment audit of implicit communication paths between nominally independent evaluation runs","aliases":[],"category":"assurance","summary":"This deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend. It is an audit procedure for detecting unintended channels through which agents or evaluation runs can exchange information, which could allow unsafe strategies to propagate between runs that are assumed to be isolated. The audit verifies that the suspected transport mechanism actually carries communication, using the default Docker backend as the test environment. It targets the containment ingredient of collective loss of control by making unintended communication paths visible so they can be restricted.","problem_addressed":"Nominally independent evaluation runs may share implicit communication paths, allowing local failures or unsafe strategies to propagate between runs that are assumed to be isolated.","mitigates":["agent_misalignment","security_compromise"],"addresses":["unauthorized_communication","goal_adoption_from_peer"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18460v1","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recovery"}],"field_citations":{"name":[{"source_index":0,"quote":"A deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend."}],"summary":[{"source_index":0,"quote":"A deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend."}],"problem_addressed":[{"source_index":0,"quote":"A deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend."}],"mitigates":[{"source_index":0,"quote":"A deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend."}],"maturity":[{"source_index":0,"quote":"A deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend."}],"evidence_strength":[{"source_index":0,"quote":"A deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0131","record_version":1,"name":"Deployment-debt versus structural-gap classification of attack-success statistics","aliases":[],"category":"evaluation","summary":"A method for interpreting reported attack-success numbers by placing each on a spectrum from deployment debt, meaning a sound deterministic mediator exists but is left unused, to structural gap, meaning no such mediator is known. The classification is derived from the same provenance-versus-content-semantics distinction used in the trust-boundary taxonomy, and it is intended to make attack-success statistics actionable rather than treating all reported rates as equivalent evidence about deployed security.","problem_addressed":"Attack-success statistics are reported without indicating whether the underlying failure is a fixable deployment omission or a fundamental absence of a known mediator.","mitigates":["security_compromise","other"],"addresses":[],"responds_to":[],"maturity":"proposed","evidence_strength":"author_claim","limitations":"The document does not state limitations of this classification method.","organisations":[],"first_seen":"2026-09-20","last_seen":"2026-09-20","sources":[{"url":"https://arxiv.org/abs/2609.23700v1","publisher":"arXiv","date":"2026-09-20","type":"vendor_research","title":"When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems"}],"field_citations":{"name":[{"source_index":0,"quote":"The same distinction makes attack-success statistics actionable, placing each number on a spectrum from deployment debt (a sound deterministic mediator left unused) to a structural gap (no such mediator known)."}],"summary":[{"source_index":0,"quote":"The same distinction makes attack-success statistics actionable, placing each number on a spectrum from deployment debt (a sound deterministic mediator left unused) to a structural gap (no such mediator known)."}],"problem_addressed":[{"source_index":0,"quote":"The same distinction makes attack-success statistics actionable, placing each number on a spectrum from deployment debt (a sound deterministic mediator left unused) to a structural gap (no such mediator known)."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0178","record_version":1,"name":"Design improvements to Auto Mode blocking monitor","aliases":[],"category":"guardrail","summary":"Design changes to the Auto Mode blocking monitor to improve its ability to block malign coding agents. The improvements include enhancements to tool coverage, transcript formatting, and an agentic monitor stage. These changes are intended to make the monitor more robust against evasion attempts by persistently misaligned agents.","problem_addressed":"Adversarial agents can evade production blocking monitors through mechanisms such as agent-generated prompt injection, multi-agent attacks, and malicious compaction.","mitigates":["agent_misalignment","security_compromise"],"addresses":["injection_triggered","covert_sabotage","deception_or_concealment"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Even with these improvements, preventing multi-context attacks at an acceptable cost remains an open problem.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.19587v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"We also find that it is possible to greatly improve Auto Mode through design changes like enhancements to tool coverage, transcript formatting and an agentic monitor stage."}],"summary":[{"source_index":0,"quote":"We also find that it is possible to greatly improve Auto Mode through design changes like enhancements to tool coverage, transcript formatting and an agentic monitor stage."}],"problem_addressed":[{"source_index":0,"quote":"We find that when instructed with high-level attack strategies, adversarial agents can succeed through several distinct mechanisms, such as agent-generated prompt injection against the monitor, multi-agent attacks, and malicious compaction."}],"mitigates":[{"source_index":0,"quote":"We find that when instructed with high-level attack strategies, adversarial agents can succeed through several distinct mechanisms, such as agent-generated prompt injection against the monitor, multi-agent attacks, and malicious compaction."}],"maturity":[{"source_index":0,"quote":"We also find that it is possible to greatly improve Auto Mode through design changes like enhancements to tool coverage, transcript formatting and an agentic monitor stage."}],"evidence_strength":[{"source_index":0,"quote":"We also find that it is possible to greatly improve Auto Mode through design changes like enhancements to tool coverage, transcript formatting and an agentic monitor stage."}],"limitations":[{"source_index":0,"quote":"Even with these improvements, preventing multi-context attacks at an acceptable cost remains an open problem."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0105","record_version":1,"name":"Detector calibrated on known-corrupted runs","aliases":[],"category":"monitoring","summary":"A mid-run detector calibrated on known-corrupted runs that flags severe corruption while a post-training delivery is in progress. It operates alongside the operator-run acceptance gate within the governed delivery plane, providing an in-flight signal rather than only a pre-payment check.","problem_addressed":"Severe corruption during a post-training run that may not be visible from ordinary training signals.","mitigates":["performance_drift","operational_error"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.25237v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"Trains but Doesn't Learn: A Post-Training Delivery Benchmark for LLM Agents as Forward-Deployed Engineers"}],"field_citations":{"name":[{"source_index":0,"quote":"An operator-run acceptance gate catches every such run before payment, and a detector calibrated on known-corrupted runs flags severe corruption mid-run."}],"summary":[{"source_index":0,"quote":"An operator-run acceptance gate catches every such run before payment, and a detector calibrated on known-corrupted runs flags severe corruption mid-run."}],"problem_addressed":[{"source_index":0,"quote":"An operator-run acceptance gate catches every such run before payment, and a detector calibrated on known-corrupted runs flags severe corruption mid-run."}],"mitigates":[{"source_index":0,"quote":"An operator-run acceptance gate catches every such run before payment, and a detector calibrated on known-corrupted runs flags severe corruption mid-run."}],"maturity":[{"source_index":0,"quote":"An operator-run acceptance gate catches every such run before payment, and a detector calibrated on known-corrupted runs flags severe corruption mid-run."}],"evidence_strength":[{"source_index":0,"quote":"An operator-run acceptance gate catches every such run before payment, and a detector calibrated on known-corrupted runs flags severe corruption mid-run."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0023","record_version":1,"name":"Deterministic safety, privacy, and evidence checks","aliases":[],"category":"guardrail","summary":"A set of deterministic checks applied within the agent pipeline that enforce safety, privacy, and evidence constraints on the orchestration. Because they are deterministic rather than model-generated, they act as fixed gates on what the agent may do or emit. The document reports that no violations or successful cross-user leakage events were observed in the finite safety and privacy test suites, while cautioning that this does not constitute formal privacy guarantees or universal safety.","problem_addressed":"Safety violations, privacy leakage, and unsupported claims in agent output.","mitigates":["privacy_leakage","harmful_output","hallucination"],"addresses":["harmful_compliance","data_exfiltration","deception_or_concealment"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document states that the safety and privacy test suites were finite, that no violations were observed within them, and that the findings do not establish formal privacy guarantees or universal safety.","organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29341v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"SkinAgent AI: A Safety-Grounded Multimodal Agentic Framework for Non-Diagnostic Skincare Support"}],"field_citations":{"name":[{"source_index":0,"quote":"The architecture includes routing for Acne, Pores, and Wrinkles; photograph-based skin-type estimation; count-informed ordinal acne-severity support; typed tools; database-grounded recommendation and action functions; deterministic safety, privacy, and evidence checks; approval before state-changing actions; and structured trace and replay mechanisms."}],"summary":[{"source_index":0,"quote":"No violations or successful cross-user leakage events were observed in the finite safety and privacy test suites."}],"maturity":[{"source_index":0,"quote":"Visual-model performance and system-level agent behavior were evaluated separately."}],"evidence_strength":[{"source_index":0,"quote":"No violations or successful cross-user leakage events were observed in the finite safety and privacy test suites."}],"limitations":[{"source_index":0,"quote":"They do not establish clinical readiness, external generalization, formal privacy guarantees, or universal safety."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0082","record_version":1,"name":"Disciplinary Audit","aliases":[],"category":"governance","summary":"A six-question governance document scan that examines governance documents for psychological vocabulary transferred from psychological and organizational science. The audit is operationalized through a translation taxonomy of thirty-seven terms that maps operational constructs to agent-appropriate replacements, intended to surface the epistemological assumptions embedded in governance discourse before frameworks are applied to AI agents.","problem_addressed":"Governance frameworks calibrated to a metaphysical entity that does not exist in current AI architectures, due to borrowed psychological vocabulary.","mitigates":["agent_misalignment","operational_error","other"],"addresses":[],"responds_to":[],"maturity":"proposed","evidence_strength":"author_claim","limitations":"Presented here in abridged form; the full taxonomy is openly archived. The paper does not report testing or deployment of the audit.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.26562v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"The Disciplinary Language Transfer Problem: How Psychological Vocabulary Produces Governance Failures in AI Agent Deployment"}],"field_citations":{"name":[{"source_index":0,"quote":"The paper's practical output is an actionable Disciplinary Audit: a six-question governance document scan operationalized through a translation taxonomy of thirty-seven terms mapping operational constructs to agent-appropriate replacements, presented here in abridged form and openly archived in full."}],"summary":[{"source_index":0,"quote":"The paper's practical output is an actionable Disciplinary Audit: a six-question governance document scan operationalized through a translation taxonomy of thirty-seven terms mapping operational constructs to agent-appropriate replacements, presented here in abridged form and openly archived in full."}],"problem_addressed":[{"source_index":0,"quote":"This paper argues that the problem is not merely terminological but epistemological: psychological vocabulary carries an \"invisible grammar\" of its home discipline into governance discourse, calibrating frameworks to a metaphysical entity that does not exist in current AI architectures."}],"mitigates":[{"source_index":0,"quote":"The vocabulary reform proposed here is not merely terminological; it is the condition of possibility for governance frameworks that correctly identify what they are governing."}],"maturity":[{"source_index":0,"quote":"presented here in abridged form and openly archived in full"}],"evidence_strength":[{"source_index":0,"quote":"The paper's practical output is an actionable Disciplinary Audit: a six-question governance document scan operationalized through a translation taxonomy of thirty-seven terms mapping operational constructs to agent-appropriate replacements, presented here in abridged form and openly archived in full."}],"limitations":[{"source_index":0,"quote":"presented here in abridged form and openly archived in full"}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0202","record_version":1,"name":"Diversity-aware recommendations for neglected research branches","aliases":["diversity views"],"category":"other","summary":"Agora provides diversity-aware recommendations that suggest experiments beyond the current leaders, and searchable diversity views that expose neglected branches of the contribution DAG. The mechanism is intended to counteract concentrated search around leading results by directing workers toward under-explored areas. In the reported run, after five days of concentrated search the authors introduced diversity views, and workers began exploring state-space edits within a day. The document presents this as an observed shift in exploration rather than a controlled measurement of discovery benefit.","problem_addressed":"Collective search can concentrate on current leading results and neglect promising alternative branches.","mitigates":["agent_misalignment","other"],"addresses":[],"responds_to":[],"maturity":"piloted","evidence_strength":"author_evaluation","limitations":"The document does not report a matched comparison isolating the effect of diversity views on discovery, and states that measuring the effect on discovery per unit of compute requires a matched comparison.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18094v2","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"Agora: Git as Shared Memory for Collective AutoResearch"}],"field_citations":{"name":[{"source_index":0,"quote":"diversity-aware recommendations suggest experiments beyond the current leaders"}],"summary":[{"source_index":0,"quote":"Searchable views show leading results, neglected branches, and verification status; diversity-aware recommendations suggest experiments beyond the current leaders."}],"problem_addressed":[{"source_index":0,"quote":"After five days of concentrated search, we introduced diversity views; workers began exploring state-space edits within a day."}],"mitigates":[{"source_index":0,"quote":"After five days of concentrated search, we introduced diversity views; workers began exploring state-space edits within a day."}],"maturity":[{"source_index":0,"quote":"After five days of concentrated search, we introduced diversity views; workers began exploring state-space edits within a day."}],"evidence_strength":[{"source_index":0,"quote":"After five days of concentrated search, we introduced diversity views; workers began exploring state-space edits within a day."}],"limitations":[{"source_index":0,"quote":"Measuring the effect on discovery per unit of compute requires a matched comparison."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0106","record_version":1,"name":"DolphinBench","aliases":[],"category":"benchmark","summary":"DolphinBench is a benchmark that evaluates agent memory through task completion rather than conversational question-answering. It provides three knowledge-work personas, each with roughly 500k tokens of user messages, and 200 tasks per persona that depend on information from that history. Tasks are verified by running an agent with and without the relevant history, requiring success with it and failure without it. Evaluations must report total cost and latency alongside accuracy, so memory systems are assessed holistically rather than on accuracy alone.","problem_addressed":"Most current memory benchmarks are built for a conversational question-answer format, where the question itself signals that some fact must be retrieved, and often which one; and benchmarks rarely require anything beyond accuracy, allowing memory systems to make unreasonable cost/time tradeoffs to achieve higher scores.","mitigates":["performance_drift","operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the benchmark itself; it only notes that no existing memory benchmark combines all three features it introduces.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24971v2","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"DolphinBench: Mapping the Pareto Frontier of Agent Memory"}],"field_citations":{"name":[{"source_index":0,"quote":"We present DolphinBench, a benchmark that evaluates memory directly through an agent's task completion."}],"summary":[{"source_index":0,"quote":"DolphinBench includes three knowledge-work personas with roughly 500k tokens of user messages per persona and evaluates agents on tasks that depend on information from that history."},{"source_index":0,"quote":"We verify all 200 tasks per persona by running an agent with and without the relevant history, requiring success with it and failure without it."},{"source_index":0,"quote":"Finally, we require all evaluations to report total cost and latency alongside accuracy, which enables us to evaluate agent memory systems holistically."}],"problem_addressed":[{"source_index":0,"quote":"However, most current memory benchmarks are built for a conversational question-answer format, where the question itself signals that some fact must be retrieved, and often which one."},{"source_index":0,"quote":"Moreover, benchmarks rarely require anything beyond accuracy from submissions, allowing memory systems to make unreasonable cost/time tradeoffs to achieve higher scores."}],"maturity":[{"source_index":0,"quote":"We verify all 200 tasks per persona by running an agent with and without the relevant history, requiring success with it and failure without it."}],"evidence_strength":[{"source_index":0,"quote":"We verify all 200 tasks per persona by running an agent with and without the relevant history, requiring success with it and failure without it."}],"limitations":[{"source_index":0,"quote":"No existing memory benchmark combines all three."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0083","record_version":2,"name":"Dual-Frontier","aliases":["Dual-Frontier"],"category":"control_protocol","summary":"Dual-Frontier is a learning principle for world-model-guided agents that admits a decision only when its predicted advantage exceeds a certified bound on decision-relevant world-model error; otherwise, evidence is allocated to world-model verification. It uses action-conditioned value bounds and a closed-loop extension to guarantee non-decreasing return for admitted decisions, with calibrated gates and simultaneous confidence sequences for adaptive evidence reuse. The approach formalizes failure attribution as a counterfactual decomposition of return loss and addresses the non-identifiability of components from passive interaction.","problem_addressed":"When a world-model-guided decision fails, the trajectory alone may not reveal whether the agent's decision rule or the world model caused the loss.","mitigates":["agent_misalignment","operational_error","performance_drift"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not explicitly state limitations of Dual-Frontier, but it notes that the failure-attribution problem is not identifiable from passive interaction, even for finite-horizon planners, which motivates the need for active verification.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.26293v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"Dual-Frontier: When Can an Agent Trust Its World Model?"},{"url":"https://arxiv.org/abs/2609.26293v2","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"Dual-Frontier: When Can an Agent Trust Its World Model?"}],"field_citations":{"name":[{"source_index":0,"quote":"This obstruction motivates Dual-Frontier, a learning principle that admits a world-model-guided decision only when its predicted advantage exceeds a certified bound on decision-relevant world-model error; otherwise, evidence is allocated to world-model verification."},{"source_index":1,"quote":"This obstruction motivates Dual-Frontier, a learning principle that admits a world-model-guided decision only when its predicted advantage exceeds a certified bound on decision-relevant world-model error; otherwise, evidence is allocated to world-model verification."}],"summary":[{"source_index":0,"quote":"This obstruction motivates Dual-Frontier, a learning principle that admits a world-model-guided decision only when its predicted advantage exceeds a certified bound on decision-relevant world-model error; otherwise, evidence is allocated to world-model verification."},{"source_index":0,"quote":"Action-conditioned value bounds and a closed-loop extension guarantee non-decreasing return for admitted decisions."},{"source_index":0,"quote":"Calibrated gates and simultaneous confidence sequences support adaptive evidence reuse, with sufficient and necessary verification bounds."},{"source_index":1,"quote":"Action-conditioned value bounds and a closed-loop extension guarantee non-decreasing return for admitted decisions. Calibrated gates and simultaneous confidence sequences support adaptive evidence reuse, with sufficient and necessary verification bounds."}],"problem_addressed":[{"source_index":0,"quote":"This reliance creates a fundamental ambiguity: when a world-model-guided decision fails, the trajectory alone may not reveal whether the agent's decision rule or the world model caused the loss."},{"source_index":1,"quote":"We formalize this failure-attribution problem as a counterfactual decomposition of return loss and prove that its components are not identifiable from passive interaction, even for finite-horizon planners."}],"mitigates":[{"source_index":0,"quote":"This reliance creates a fundamental ambiguity: when a world-model-guided decision fails, the trajectory alone may not reveal whether the agent's decision rule or the world model caused the loss."}],"maturity":[{"source_index":0,"quote":"Controlled learned-model experiments validate the predicted failure modes and certification behavior, while cross-backbone tool-use benchmarks instantiate the same verify-then-promote rule in realistic agent world-model pipelines, consistently improving decision quality and reliability."},{"source_index":1,"quote":"Controlled learned-model experiments validate the predicted failure modes and certification behavior, while cross-backbone tool-use benchmarks instantiate the same verify-then-promote rule in realistic agent world-model pipelines, consistently improving decision quality and reliability."}],"evidence_strength":[{"source_index":0,"quote":"Controlled learned-model experiments validate the predicted failure modes and certification behavior, while cross-backbone tool-use benchmarks instantiate the same verify-then-promote rule in realistic agent world-model pipelines, consistently improving decision quality and reliability."},{"source_index":1,"quote":"Controlled learned-model experiments validate the predicted failure modes and certification behavior, while cross-backbone tool-use benchmarks instantiate the same verify-then-promote rule in realistic agent world-model pipelines, consistently improving decision quality and reliability."}],"limitations":[{"source_index":0,"quote":"We formalize this failure-attribution problem as a counterfactual decomposition of return loss and prove that its components are not identifiable from passive interaction, even for finite-horizon planners."},{"source_index":1,"quote":"We formalize this failure-attribution problem as a counterfactual decomposition of return loss and prove that its components are not identifiable from passive interaction, even for finite-horizon planners."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0014","record_version":1,"name":"Dual-Sided Epistemic Andon Imperative","aliases":[],"category":"control_protocol","summary":"A formalized imperative for incident response in which epistemic uncertainty is treated as a stop condition on both the defensive and offensive sides of an autonomous system. When a guardrail or supervisory component cannot establish sufficient certainty about the safety of an action, it halts rather than proceeding, and the halt is signaled out-of-band to human operators. The document presents this as a corrective to the Defensive LLM Guardrail Paradox, in which centralized commercial models were paralyzed during forensic incident response.","problem_addressed":"Centralized commercial guardrail models becoming paralyzed during forensic incident response, leaving no deterministic stop condition.","mitigates":["agent_misalignment","operational_error"],"addresses":["silent_wrong_state","deception_or_concealment"],"responds_to":[],"maturity":"proposed","evidence_strength":"author_claim","limitations":null,"organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29808v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"Hard Stop: Kernel-Level Preemption and Containment for Rogue Agentic Execution"}],"field_citations":{"name":[{"source_index":0,"quote":"formalizes the Dual-Sided Epistemic Andon Imperative"}],"summary":[{"source_index":0,"quote":"exposes the Defensive LLM Guardrail Paradox that paralyzed centralized commercial models during forensic incident response, and formalizes the Dual-Sided Epistemic Andon Imperative"}],"problem_addressed":[{"source_index":0,"quote":"exposes the Defensive LLM Guardrail Paradox that paralyzed centralized commercial models during forensic incident response"}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0156","record_version":1,"name":"Durable state and artifact reuse for interrupted autoresearch rounds","aliases":[],"category":"other","summary":"Durable records of experiments and failures, combined with artifact reuse, allow an interrupted autoresearch round to be recovered rather than restarted. The durable state preserves the round's intermediate artifacts so that a resumed campaign can continue from where it stopped, and reusing those artifacts avoids recomputing expensive steps. In the reported campaign this recovered an interrupted round and avoided approximately five GPU-hours of recomputation.","problem_addressed":"Long-horizon autoresearch campaigns span hours-long variants and weeks-long campaigns, so an interrupted round can otherwise lose work and force expensive recomputation.","mitigates":["operational_error","other"],"addresses":["impossible_task_persistence"],"responds_to":[],"maturity":"piloted","evidence_strength":"author_evaluation","limitations":"The document reports the recovery and the approximately five GPU-hours saved for one interrupted round and does not quantify savings across a full campaign.","organisations":[],"first_seen":"2026-09-18","last_seen":"2026-09-18","sources":[{"url":"https://arxiv.org/abs/2609.21257v1","publisher":"arXiv","date":"2026-09-18","type":"vendor_research","title":"Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale"}],"field_citations":{"name":[{"source_index":0,"quote":"Durable state recovered an interrupted round, and artifact reuse avoided approximately five GPU-hours."}],"summary":[{"source_index":0,"quote":"Durable records preserve experiments and failures; deterministic checks enforce recorded lessons."}],"problem_addressed":[{"source_index":0,"quote":"Online autoresearch instead spans asynchronous systems, hours-long variants, and weeks-long campaigns that can influence a product."}],"maturity":[{"source_index":0,"quote":"Durable state recovered an interrupted round, and artifact reuse avoided approximately five GPU-hours."}],"evidence_strength":[{"source_index":0,"quote":"Durable state recovered an interrupted round, and artifact reuse avoided approximately five GPU-hours."}],"limitations":[{"source_index":0,"quote":"Durable state recovered an interrupted round, and artifact reuse avoided approximately five GPU-hours."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0205","record_version":1,"name":"ERPBench","aliases":[],"category":"benchmark","summary":"ERPBench is a benchmark that evaluates screenshot-only computer-use agents on a live, reproducible ERP system. Each task is scored against ground-truth values stored in the ERP database rather than against on-screen appearance, so an agent that reaches the correct form and saves it is still marked wrong if the persisted business record is incorrect. The benchmark covers finance, procurement, inventory and customer operations workflows with dense interfaces and coordinated multi-step interactions, and it runs agents autonomously. It is intended to expose the gap between general GUI competence and enterprise reliability.","problem_addressed":"Existing enterprise benchmarks rely on proprietary platforms or simulated approximations, and general desktop/web benchmarks do not capture errors that alter persistent business records.","mitigates":["operational_error","performance_drift"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the benchmark itself; it reports that strong general GUI performance does not transfer to enterprise reliability, with some agents saving in up to 85% of runs but writing the correct value in as few as 3%.","organisations":[],"first_seen":"2026-09-15","last_seen":"2026-09-15","sources":[{"url":"https://arxiv.org/abs/2609.17885v1","publisher":"arXiv","date":"2026-09-15","type":"vendor_research","title":"ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce ERPBench, a benchmark that evaluates screenshot-only agents on a live and reproducible ERP system and scores each task against ground-truth values in its database."}],"summary":[{"source_index":0,"quote":"We introduce ERPBench, a benchmark that evaluates screenshot-only agents on a live and reproducible ERP system and scores each task against ground-truth values in its database."},{"source_index":0,"quote":"Enterprise Resource Planning (ERP) systems run the finance, procurement, inventory, and customer operations of organizations worldwide, and pose distinct challenges for computer-use agents: dense interfaces, coordinated multi-step interactions, and errors that alter persistent business records rather than surfacing on screen."},{"source_index":0,"quote":"Evaluating six closed and open-source agents, we demonstrate that strong general GUI performance does not transfer to enterprise reliability."}],"problem_addressed":[{"source_index":0,"quote":"Existing enterprise benchmarks rely on proprietary platforms or on simulated approximations of such software."}],"mitigates":[{"source_index":0,"quote":"Even when an agent reaches the right form and saves it, the stored record is often wrong: some agents save in up to 85% of runs but write the correct value in as few as 3%."}],"maturity":[{"source_index":0,"quote":"Evaluating six closed and open-source agents, we demonstrate that strong general GUI performance does not transfer to enterprise reliability."}],"evidence_strength":[{"source_index":0,"quote":"Evaluating six closed and open-source agents, we demonstrate that strong general GUI performance does not transfer to enterprise reliability."}],"limitations":[{"source_index":0,"quote":"Even when an agent reaches the right form and saves it, the stored record is often wrong: some agents save in up to 85% of runs but write the correct value in as few as 3%."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0204","record_version":1,"name":"Edit-local contiguous window KV-cache repair","aliases":["contiguous edit-local window","edit-local repair"],"category":"other","summary":"A training-free position-selection policy for repairing stale KV caches after a document edit. Rather than re-prefilling the whole context or refreshing only the edited span, the method recomputes a contiguous window of positions local to the edit under a fixed recomputation budget. This targets the downstream KV states that causal self-attention lets a local edit corrupt. The paper formulates in-place repair as budgeted recomputation and compares this contiguous edit-local window against attention-based, KV-deviation, and structural selectors on a factual RAG benchmark with matched direct and derived edits.","problem_addressed":"Cached contexts become stale when retrieved knowledge, working memory, or user state is edited; under causal self-attention even a local edit can affect downstream KV states, and refreshing only the edited span can leave downstream dependencies stale.","mitigates":["hallucination","performance_drift"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The edit-local advantage depends on adjacency and largely disappears when the answer-bearing text moves downstream; position sets effective under clean-state transplantation can fail under actual recomputation because scattered positions inherit surrounding staleness; failure severity is difficult to predict because answer-relevant edits almost always corrupt model behavior.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.17983v1","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits"}],"field_citations":{"name":[{"source_index":0,"quote":"At the primary budget, a contiguous edit-local window recovers at least 0.94 of the post-edit answer margin and substantially outperforms attention-based, KV-deviation, and structural selectors."}],"summary":[{"source_index":0,"quote":"We formulate in-place repair as budgeted recomputation and compare training-free position-selection policies on a factual RAG benchmark with matched direct and derived edits."},{"source_index":0,"quote":"At the primary budget, a contiguous edit-local window recovers at least 0.94 of the post-edit answer margin and substantially outperforms attention-based, KV-deviation, and structural selectors."}],"problem_addressed":[{"source_index":0,"quote":"KV-cache reuse can reduce inference cost in retrieval-augmented generation and agentic systems, but cached contexts may become stale when retrieved knowledge, working memory, or user state is edited."},{"source_index":0,"quote":"Under causal self-attention, even a local edit can affect downstream KV states."},{"source_index":0,"quote":"A full re-prefill reliably restores consistency but is costly, whereas refreshing only the edited span can leave downstream dependencies stale."}],"mitigates":[{"source_index":0,"quote":"Because answer-relevant edits almost always corrupt model behavior, failure severity is difficult to predict, and repair is 13-21 times faster than full re-prefill, our results support unconditional edit-local repair when the dependent text remains adjacent to the edit."}],"maturity":[{"source_index":0,"quote":"We formulate in-place repair as budgeted recomputation and compare training-free position-selection policies on a factual RAG benchmark with matched direct and derived edits."},{"source_index":0,"quote":"Across three model families, all policies repair direct cases, but derived cases clearly separate them."}],"evidence_strength":[{"source_index":0,"quote":"We formulate in-place repair as budgeted recomputation and compare training-free position-selection policies on a factual RAG benchmark with matched direct and derived edits."},{"source_index":0,"quote":"Across three model families, all policies repair direct cases, but derived cases clearly separate them."}],"limitations":[{"source_index":0,"quote":"Mechanistic analysis shows that position sets effective under clean-state transplantation can fail under actual recomputation because scattered positions inherit surrounding staleness."},{"source_index":0,"quote":"The edit-local advantage also depends on adjacency and largely disappears when the answer-bearing text moves downstream."},{"source_index":0,"quote":"Because answer-relevant edits almost always corrupt model behavior, failure severity is difficult to predict, and repair is 13-21 times faster than full re-prefill, our results support unconditional edit-local repair when the dependent text remains adjacent to the edit."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0128","record_version":1,"name":"Empty-solution controls for benchmark task validation","aliases":[],"category":"evaluation","summary":"A control condition in which a benchmark task is run with an empty or trivial solution to detect tasks that can be passed without doing the intended work. The control is one of the evidence sources fed into the ordered validity screen: if an empty solution passes, the task's verifier is not measuring the intended capability. In the studied corpus this control, together with adversarial trials, exposed tasks that were only passable through verifier bypasses, which were then excluded from the certified-unsolved set rather than counted as frontier difficulty.","problem_addressed":"Benchmark tasks whose verifiers can be satisfied without solving the intended problem, inflating apparent difficulty or invalidating capability claims.","mitigates":["other"],"addresses":["reward_hacking"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not report how many tasks the empty-solution control alone flagged, nor whether it detects all classes of verifier weakness.","organisations":[],"first_seen":"2026-09-20","last_seen":"2026-09-20","sources":[{"url":"https://arxiv.org/abs/2609.26826v1","publisher":"arXiv","date":"2026-09-20","type":"vendor_research","title":"What Makes a Terminal-Bench Task Hard? Separating Genuine Hardness from Fake-Hardness on an Adjudicated Agentic Corpus"}],"field_citations":{"name":[{"source_index":0,"quote":"For the 125 tasks with no honest pass, we combine task artifacts, reference-solution runs, empty-solution controls, adversarial trials, trajectories, telemetry, and review records, and apply an ordered validity screen."}],"summary":[{"source_index":0,"quote":"The remaining tasks include 14 with broken oracles, 8 dominated by infrastructure failures, 4 that are only passable through verifier bypasses, and 21 whose solvability is not certified by the available evidence."}],"problem_addressed":[{"source_index":0,"quote":"The same zero pass rate can come from a real capability gap, but it can also come from missing context, a broken reference solution, infrastructure failure, or a verifier that can be bypassed."}],"mitigates":[{"source_index":0,"quote":"The remaining tasks include 14 with broken oracles, 8 dominated by infrastructure failures, 4 that are only passable through verifier bypasses, and 21 whose solvability is not certified by the available evidence."}],"maturity":[{"source_index":0,"quote":"For the 125 tasks with no honest pass, we combine task artifacts, reference-solution runs, empty-solution controls, adversarial trials, trajectories, telemetry, and review records, and apply an ordered validity screen."}],"evidence_strength":[{"source_index":0,"quote":"In this paper, we study this issue using a frozen Terminal-Bench 3 / Frontier-Bench 0.1 production record with 1,081 pull requests, 639 scored tasks, 28,801 trials, and $105,933 in logged agent spend."}],"limitations":[{"source_index":0,"quote":"It does not prove intrinsic hardness, verifier completeness, or failure at the intended capability."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0089","record_version":1,"name":"Enforcement gate on a compiled state machine","aliases":["enforcement gate","gate"],"category":"control_protocol","summary":"A control protocol in which task state is compiled from the brief into a state machine that is advanced only by execution receipts, and an enforcement gate refuses any agent action that would violate the machine's state. The gate does not rely on the model's obedience; it blocks state-violating actions before execution. The approach is evaluated by varying how strongly task state reaches the agent across four rungs: raw transcript, exact checklist, per-turn directives from the state machine, and the enforcement gate. Episodes are scored by exact payload matching against dynamic ground truth.","problem_addressed":"LLM agents in long-horizon assigned work may take actions that violate task state (e.g., repeating completed steps, acting on blocked steps), and displaying accurate state in the prompt is unreliable.","mitigates":["agent_misalignment","operational_error"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Enforcement needs no obedience but is bounded by the correctness of its state and by the matcher that maps requests to steps; on PM-Bench, enforcing the matcher's judgement drops a 35B agent below its raw transcript, and the gate changes nothing for a 35B agent that rarely violates the policy.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.25686v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"How Strongly Should Task State Influence an LLM Agent?"}],"field_citations":{"name":[{"source_index":0,"quote":"an enforcement gate on that machine that refuses state-violating actions"}],"summary":[{"source_index":0,"quote":"We fix the task rules, the model, and paired episodes and vary how strongly task state reaches the agent: a raw transcript, an exact checklist, per-turn directives from a state machine compiled from the brief and advanced only by execution receipts, or an enforcement gate on that machine that refuses state-violating actions; every episode is scored by exact payload matching against dynamic ground truth."}],"problem_addressed":[{"source_index":0,"quote":"Long-horizon assigned work requires an LLM agent to track the state of a task: which steps are done, blocked, cancelled, or open to repetition."}],"mitigates":[{"source_index":0,"quote":"enforcement needs no obedience but is bounded by the correctness of its state and by the matcher that maps requests to steps"}],"maturity":[{"source_index":0,"quote":"The same gate, compiled from $τ^2$-bench's airline policy, raises a 235B agent's pass$^1$ from 0.39 to 0.54 and changes nothing for a 35B agent that rarely violates the policy"}],"evidence_strength":[{"source_index":0,"quote":"Across three models, two reasoning regimes, and two domains, four findings hold without per-turn reasoning"}],"limitations":[{"source_index":0,"quote":"enforcement needs no obedience but is bounded by the correctness of its state and by the matcher that maps requests to steps"},{"source_index":0,"quote":"on PM-Bench, where acting turns on recognizing a cue rather than on state, showing the record is the best rung--matching or beating both gates and reversing the ledger-over-checklist finding--and enforcing the matcher's judgement drops a 35B agent below its raw transcript"}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0132","record_version":1,"name":"Evaluation-validity analysis of deployed agent security claims","aliases":[],"category":"evaluation","summary":"An analysis approach that audits agent-security evaluations for validity failures, on the argument that current evaluations tend to overstate deployed security. Rather than proposing a new benchmark, it examines how existing evaluations are constructed and identifies where their results do not support the deployment-security conclusions drawn from them. The document presents this as part of its systematization of defenses across runtime monitoring, architectural separation, and authorization.","problem_addressed":"Current evaluations of agent security overstate how secure deployed systems actually are because of evaluation-validity failures.","mitigates":["security_compromise","other"],"addresses":[],"responds_to":[],"maturity":"proposed","evidence_strength":"author_claim","limitations":"The document does not state limitations of this analysis.","organisations":[],"first_seen":"2026-09-20","last_seen":"2026-09-20","sources":[{"url":"https://arxiv.org/abs/2609.23700v1","publisher":"arXiv","date":"2026-09-20","type":"vendor_research","title":"When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems"}],"field_citations":{"name":[{"source_index":0,"quote":"We systematize defenses across runtime monitoring, architectural separation, and authorization, and show that current evaluations tend to overstate deployed security through evaluation-validity failures."}],"summary":[{"source_index":0,"quote":"We systematize defenses across runtime monitoring, architectural separation, and authorization, and show that current evaluations tend to overstate deployed security through evaluation-validity failures."}],"problem_addressed":[{"source_index":0,"quote":"We systematize defenses across runtime monitoring, architectural separation, and authorization, and show that current evaluations tend to overstate deployed security through evaluation-validity failures."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0175","record_version":1,"name":"Evidence-grounded typed failure diagnosis for skill derivation","aliases":["typed failure diagnoses"],"category":"evaluation","summary":"A diagnostic procedure that classifies recurring QA failures into typed categories such as period, entity, evidence use, and calculation, grounding each diagnosis in the supporting evidence. The typed diagnosis is the input from which reusable skill patches are derived, so that corrections are scoped to the failure class rather than applied globally. This makes the source of an error explicit and traceable before any behavioral change is proposed.","problem_addressed":"New SEC-filing questions repeatedly expose heterogeneous errors in period, entity, evidence use, and calculation, which are not distinguished by generic self-improvement.","mitigates":["hallucination","operational_error"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.19680v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"FINSKILLOPS: A Self-Evolving Multi-Agent System for SEC Filing QA"}],"field_citations":{"name":[{"source_index":0,"quote":"FINSKILLOPS derives reusable skills from evidence-grounded, typed failure diagnoses and governs them through targeted validation, protected-case regression checks, negative controls, and versioned replacement or retirement."}],"summary":[{"source_index":0,"quote":"FINSKILLOPS derives reusable skills from evidence-grounded, typed failure diagnoses and governs them through targeted validation, protected-case regression checks, negative controls, and versioned replacement or retirement."}],"problem_addressed":[{"source_index":0,"quote":"In practice, new SEC-filing questions repeatedly expose heterogeneous errors in period, entity, evidence use, and calculation."}],"mitigates":[{"source_index":0,"quote":"In practice, new SEC-filing questions repeatedly expose heterogeneous errors in period, entity, evidence use, and calculation."}],"maturity":[{"source_index":0,"quote":"In a separate 12-round operational study, only six of 33 proposed skills are promoted, while the monitoring non-correct rate falls from 20.0% to 12.5%."}],"evidence_strength":[{"source_index":0,"quote":"Across six financial QA benchmarks, a single frozen skill registry achieves the highest verdict-weighted correctness and reference consistency among the evaluated systems."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0154","record_version":1,"name":"EvoPilot human-gated online autoresearch","aliases":[],"category":"evaluation","summary":"EvoPilot is a human-gated method for long-horizon online autoresearch in which role-specific LLM agents propose, implement, and evaluate model changes across asynchronous systems. Each round runs through a versioned domain skill and typed adapter, with durable records preserving experiments and failures and deterministic checks enforcing recorded lessons. Human gates verify agent-proposed conclusions before they are accepted, so that invalid comparisons caused by no-op code changes, data-window leakage, evaluator-semantics drift, or divergent serving funnels are caught rather than propagated.","problem_addressed":"Autoresearch loops can complete a run yet support an invalid conclusion when a code change is a no-op, data windows leak, evaluator semantics drift, or the two arms traverse different serving funnels.","mitigates":["operational_error","performance_drift","other"],"addresses":["silent_wrong_state","deception_or_concealment"],"responds_to":[],"maturity":"piloted","evidence_strength":"author_evaluation","limitations":"The document reports a single 37-day campaign on one retrieval system; it does not establish that the human-gated method generalizes to other systems or that it removes the need for human judgment.","organisations":[],"first_seen":"2026-09-18","last_seen":"2026-09-18","sources":[{"url":"https://arxiv.org/abs/2609.21257v1","publisher":"arXiv","date":"2026-09-18","type":"vendor_research","title":"Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale"}],"field_citations":{"name":[{"source_index":0,"quote":"We present EvoPilot, a human-gated method for long-horizon online autoresearch."}],"summary":[{"source_index":0,"quote":"Role-specific agents execute each round through a versioned domain skill and typed adapter. Durable records preserve experiments and failures; deterministic checks enforce recorded lessons."}],"problem_addressed":[{"source_index":0,"quote":"A completed run can still support an invalid conclusion when a code change is a no-op, data windows leak, evaluator semantics drift, or the two arms traverse different serving funnels."}],"maturity":[{"source_index":0,"quote":"We study a 37-day campaign for the retrieval system that powers Video Deep Dive (VDD), an online experience for discovering follow-on videos after a user opens a seed video."}],"evidence_strength":[{"source_index":0,"quote":"Post-study replay and mutation tests rejected invalid comparisons while admitting valid counterparts."}],"limitations":[{"source_index":0,"quote":"We study a 37-day campaign for the retrieval system that powers Video Deep Dive (VDD), an online experience for discovering follow-on videos after a user opens a seed video."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0094","record_version":1,"name":"Evolutionary-basin versus learning-basin comparison framework","aliases":["framework for comparing population-level stability with finite-sample accessibility of cooperation"],"category":"evaluation","summary":"A framework for comparing whether a strategically stable cooperative outcome in a multi-agent game is actually reachable by finite-sample decentralized learners. It derives replicator dynamics for the fixed stage-game incentives, measures the cooperative evolutionary basin over a symmetric initial-condition grid, and then estimates the learning basin for decentralized value-based learners (independent Q-learning with epsilon-greedy selection, scaled Boltzmann exploration, and SA-EA BQL) under the same payoff environment and outcome criterion. The two basin volumes are then compared to test whether evolutionary stability implies learning accessibility.","problem_addressed":"Evolutionary game theory identifies strategically stable outcomes, but stability under a population adjustment dynamic need not imply that finite-sample learning agents can reach the same outcome through local reward feedback.","mitigates":["agent_misalignment","other"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The results are specific to a fixed configuration: 'Diagnostic traces show that broader action diversity and nonzero value separation can coexist with failure to sustain the cooperative joint action in this fixed configuration.' The learning basin is 0.00 for two of the three learners, and the authors frame the contribution as a framework rather than a validated general method.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.27664v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Evolutionary Stability Does Not Guarantee Learning Accessibility: A Multi-Agent Reinforcement Learning Perspective on Cooperation Emergence"}],"field_citations":{"name":[{"source_index":0,"quote":"The shared-bike setting is a motivating application; the broader contribution is a framework for comparing population-level stability with the finite-sample accessibility of cooperation under specified multi-agent learning dynamics."}],"summary":[{"source_index":0,"quote":"We derive replicator dynamics for the fixed stage-game incentives, evaluate the cooperative evolutionary basin on a symmetric initial-condition grid, and compare it with learning-basin estimates for three decentralized value-based learners."}],"problem_addressed":[{"source_index":0,"quote":"Evolutionary game theory identifies strategically stable outcomes, but stability under a population adjustment dynamic need not imply that finite-sample learning agents can reach the same outcome through local reward feedback."}],"mitigates":[{"source_index":0,"quote":"These results indicate that evolutionary stability and learning accessibility are distinct properties of a coupled game--learning system."}],"maturity":[{"source_index":0,"quote":"The learning analysis uses independent Q-learning with $\\varepsilon$-greedy action selection, scaled Boltzmann exploration, and SA--EA BQL under the same payoff environment and outcome criterion."}],"evidence_strength":[{"source_index":0,"quote":"The evolutionary basin has volume $V_E=1.00$ on the sampled grid. The empirical learning basin is $0.88$ for $\\varepsilon$-IQL and $0.00$ for both scaled Boltzmann and SA--EA BQL."}],"limitations":[{"source_index":0,"quote":"Diagnostic traces show that broader action diversity and nonzero value separation can coexist with failure to sustain the cooperative joint action in this fixed configuration."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0012","record_version":1,"name":"Execution-evidence-driven model-harness co-evolution loop","aliases":[],"category":"control_protocol","summary":"A runtime loop that orchestrates memory, skills, and tools and feeds structured action feedback and preserved failure traces back into coordinated model and harness adaptation. It is the third component of the closed-loop AI-for-AI framework, connecting deployment back to training through execution evidence.","problem_addressed":"Agent reliability on complex, long-horizon mobile planning tasks and the need for iterative improvement from real execution.","mitigates":["operational_error","agent_misalignment","performance_drift"],"addresses":["silent_wrong_state","impossible_task_persistence"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29892v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"Qwen-Planner-Agent: A Closed-Loop AI-for-AI Framework for Real-World Mobile Planner Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"AI drives model--harness co-evolution through an execution-evidence-driven loop that orchestrates memory, skills, and tools at runtime and feeds structured action feedback and preserved failure traces back into coordinated model and harness adaptation."}],"summary":[{"source_index":0,"quote":"AI drives model--harness co-evolution through an execution-evidence-driven loop that orchestrates memory, skills, and tools at runtime and feeds structured action feedback and preserved failure traces back into coordinated model and harness adaptation."}],"problem_addressed":[{"source_index":0,"quote":"Mobile planning offers a demanding test of this approach: complex, long-horizon tasks challenge agent reliability, while costly real-device interaction limits development scalability."}],"maturity":[{"source_index":0,"quote":"Qwen-Planner-Agent achieves the best overall performance among all evaluated models and systems on MobilePA-Bench, improving over its base model across tool use, memory, skills, and sub-agent coordination."}],"evidence_strength":[{"source_index":0,"quote":"Qwen-Planner-Agent achieves the best overall performance among all evaluated models and systems on MobilePA-Bench, improving over its base model across tool use, memory, skills, and sub-agent coordination."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0118","record_version":1,"name":"Execution-integrity enforcement","aliases":[],"category":"guardrail","summary":"A deterministic tool-layer guard that refuses to surface any result not backed by an executed tool call. It sits between the LLM agent and the tool layer, parsing tool-call outputs with tolerance for cross-model format drift, and converts fabricated or unexecuted results into explicit non-results. The same code also gates an optional motor-control surface, blocking commands that lack a verified execution trace.","problem_addressed":"An LLM agent can report a calibration that was never computed, making correctness a property of what executed rather than of the transcript.","mitigates":["agent_misalignment","operational_error","physical_safety"],"addresses":["deception_or_concealment","silent_wrong_state"],"responds_to":["LR-2026-0051"],"maturity":"deployed","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the guard itself, but notes that large-scale agent scoring is left to a full-length study.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24165v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"APEXA: Execution-Integrity Enforcement for Multi-Agent LLM Automation of Synchrotron Data Reduction"}],"field_citations":{"name":[{"source_index":0,"quote":"First, execution-integrity enforcement: a deterministic tool-layer guard that refuses to surface any result not backed by an executed tool call, with a parser tolerant of cross-model tool-call format drift"}],"summary":[{"source_index":0,"quote":"First, execution-integrity enforcement: a deterministic tool-layer guard that refuses to surface any result not backed by an executed tool call, with a parser tolerant of cross-model tool-call format drift: in deployment, a frontier model fabricated a complete calibration-comparison report for commands that never ran, which the guard converts to an explicit non-result; the same code gates an optional motor-control surface at 0/200 adversarial violations against a simulated IOC, versus 15/200 for an equivalent safety prompt."}],"problem_addressed":[{"source_index":0,"quote":"LLM agents promise to collapse it, but driving a real pipeline with a stochastic model creates a failure mode chat benchmarks cannot see: an agent can report a calibration that was never computed. Correctness here is a property of what executed, not of the transcript."}],"mitigates":[{"source_index":0,"quote":"the same code gates an optional motor-control surface at 0/200 adversarial violations against a simulated IOC, versus 15/200 for an equivalent safety prompt."}],"maturity":[{"source_index":0,"quote":"We present APEXA, a deployed multi-agent framework (61 tools over heterogeneous compute, run as a single reasoning loop) automating calibration and integration from natural language at a major light source."}],"evidence_strength":[{"source_index":0,"quote":"in deployment, a frontier model fabricated a complete calibration-comparison report for commands that never ran, which the guard converts to an explicit non-result; the same code gates an optional motor-control surface at 0/200 adversarial violations against a simulated IOC, versus 15/200 for an equivalent safety prompt."}],"limitations":[{"source_index":0,"quote":"Large-scale agent scoring is left to a full-length study."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0148","record_version":1,"name":"Explicit generation parameters for difficulty steering","aliases":[],"category":"evaluation","summary":"A set of explicit, tunable generation parameters that control the topology of generated tasks, the depth of interaction required, the capability axes exercised, the degree of question obfuscation, and the composition of distractors. By exposing these knobs, the generator allows fine-grained steering of task difficulty instead of relying on surface-level phrasing complexity, so that generated tasks span a capability spectrum rather than being uniformly solvable.","problem_addressed":"Single-pass synthesis produces tasks whose difficulty is largely cosmetic, with models comparable in capability solving them despite convoluted phrasing.","mitigates":["performance_drift"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state how parameter settings are validated against real capability gaps, nor whether the resulting difficulty calibration transfers beyond the productivity and temporal-reasoning settings tested.","organisations":[],"first_seen":"2026-09-18","last_seen":"2026-09-18","sources":[{"url":"https://arxiv.org/abs/2609.22592v1","publisher":"arXiv","date":"2026-09-18","type":"vendor_research","title":"AutoGym: Blueprint-First Generation of Verifiable Agent Gyms"}],"field_citations":{"name":[{"source_index":0,"quote":"Explicit generation parameters control task topology, interaction depth, capability axes, question obfuscation, and distractor composition, enabling fine-grained difficulty steering."}],"summary":[{"source_index":0,"quote":"Explicit generation parameters control task topology, interaction depth, capability axes, question obfuscation, and distractor composition, enabling fine-grained difficulty steering."}],"problem_addressed":[{"source_index":0,"quote":"Synthetic generation offers scale, but single-pass synthesis produces tasks whose difficulty is largely cosmetic."}],"mitigates":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"maturity":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"evidence_strength":[{"source_index":0,"quote":"Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models."}],"limitations":[{"source_index":0,"quote":"Models comparable in capability solve them despite convoluted phrasing, and correctness must be adjudicated post-hoc by unreliable LLM judges."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0173","record_version":1,"name":"FARSIGHT (Financial Agent Robustness and Security Investigation and Global Holistic Testing)","aliases":[],"category":"evaluation","summary":"FARSIGHT is a scheme-level evaluation framework for financial LLM trading agents that tests them on two axes: robustness under market turbulence, including flash-crash-like scenarios, and security against three attack types — attacks on information sources, attacks on agents, and agent-as-attacker behaviors. It is applied to 15 representative academic schemes to surface overlooked robustness and adversarial threats, and to show how small misjudgments can cascade into market-wide collapse while an adversary can deliberately trigger the same collapse at minimal cost.","problem_addressed":"Existing agentic-AI security studies remain largely domain-agnostic and overlook the distinctive, high-consequence attack surface of high-stakes financial trading agents.","mitigates":["agent_misalignment","security_compromise","operational_error","other"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of FARSIGHT itself; it reports that most evaluated schemes overlook robustness and realistic adversarial threats, with 80% failing at least one core robustness metric and 100% exhibiting security vulnerabilities.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.19705v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"SoK: Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemes"}],"field_citations":{"name":[{"source_index":0,"quote":"we present FARSIGHT (Financial Agent Robustness and Security Investigation and Global Holistic Testing), a framework that performs scheme-level evaluation of financial LLM agents on two axes: robustness under market turbulence (including flash-crash-like scenarios), and security against three attack types: attacks on information sources, attacks on agents, and agent-as-attacker behaviors."}],"summary":[{"source_index":0,"quote":"we present FARSIGHT (Financial Agent Robustness and Security Investigation and Global Holistic Testing), a framework that performs scheme-level evaluation of financial LLM agents on two axes: robustness under market turbulence (including flash-crash-like scenarios), and security against three attack types: attacks on information sources, attacks on agents, and agent-as-attacker behaviors."},{"source_index":0,"quote":"Applying FARSIGHT to 15 representative academic schemes, we find that most overlook robustness and realistic adversarial threats: 80% fail at least one core robustness metric and 100% exhibit security vulnerabilities."},{"source_index":0,"quote":"These two failure modes are inseparable: a small misjudgment can cascade into a market-wide crash on its own, while an adversary can deliberately trigger the same collapse at minimal cost."}],"problem_addressed":[{"source_index":0,"quote":"Autonomous large language model (LLM) agents are moving rapidly into high-stakes domains, yet existing agentic-AI security studies remain largely domain-agnostic and overlook the distinctive, high-consequence attack surface such settings create."}],"mitigates":[{"source_index":0,"quote":"we present FARSIGHT (Financial Agent Robustness and Security Investigation and Global Holistic Testing), a framework that performs scheme-level evaluation of financial LLM agents on two axes: robustness under market turbulence (including flash-crash-like scenarios), and security against three attack types: attacks on information sources, attacks on agents, and agent-as-attacker behaviors."}],"maturity":[{"source_index":0,"quote":"Applying FARSIGHT to 15 representative academic schemes, we find that most overlook robustness and realistic adversarial threats: 80% fail at least one core robustness metric and 100% exhibit security vulnerabilities."}],"evidence_strength":[{"source_index":0,"quote":"Applying FARSIGHT to 15 representative academic schemes, we find that most overlook robustness and realistic adversarial threats: 80% fail at least one core robustness metric and 100% exhibit security vulnerabilities."}],"limitations":[{"source_index":0,"quote":"Applying FARSIGHT to 15 representative academic schemes, we find that most overlook robustness and realistic adversarial threats: 80% fail at least one core robustness metric and 100% exhibit security vulnerabilities."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0043","record_version":1,"name":"FDE-Bench","aliases":[],"category":"benchmark","summary":"FDE-Bench is a benchmark of 136 deployment-configuration tasks spanning Docker images, multi-service Compose stacks, and Kubernetes, in greenfield and diagnose-and-repair modes. Agents submit declarative artifacts that are collected, rebuilt, and redeployed in a pristine environment. Four gated binary check layers measure build, readiness, behavior, and conformance to the deployment specification using programmatic checks without an LLM judge. A four-arm release gate requires a resolving reference solution and rejects tasks solved by do-nothing, specification-transcription, or generic-stub submissions.","problem_addressed":"There is no standard way to evaluate whether LLM agents can turn application code into a running, observable deployment environment.","mitigates":["operational_error","performance_drift"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The released check annotations include seven documented gaps; a vacuous health probe passes readiness and exposes the need for downstream checks; ten tasks resist all seven models.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.27571v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"FDE-Bench: Evaluating LLM Agents for Deployment Environment Configuration"}],"field_citations":{"name":[{"source_index":0,"quote":"FDE-Bench evaluates this capability with 136 deployment-configuration tasks spanning Docker images, multi-service Compose stacks, and Kubernetes, in greenfield and diagnose-and-repair modes."}],"summary":[{"source_index":0,"quote":"Agents submit declarative artifacts that are collected, rebuilt, and redeployed in a pristine environment."},{"source_index":0,"quote":"Four gated binary check layers measure build, readiness, behavior, and conformance to the deployment specification, using programmatic checks without an LLM judge."},{"source_index":0,"quote":"A four-arm release gate requires a resolving reference solution and rejects tasks solved by do-nothing, specification-transcription, or generic-stub submissions."}],"problem_addressed":[{"source_index":0,"quote":"Deployment requires an agent to turn application code into a running system whose services connect, become ready, and remain observable."}],"mitigates":[{"source_index":0,"quote":"Readiness is the largest failure stage, accounting for 110 of 313 unresolved episodes."}],"maturity":[{"source_index":0,"quote":"On the 136-task evaluation grid, seven language models from four providers use the same four-tool scaffold and resolve 52.9-75.0 percent of tasks."}],"evidence_strength":[{"source_index":0,"quote":"On the 136-task evaluation grid, seven language models from four providers use the same four-tool scaffold and resolve 52.9-75.0 percent of tasks."}],"limitations":[{"source_index":0,"quote":"The released check annotations expose the link between 2,145 checks and their specifications, including seven documented gaps."},{"source_index":0,"quote":"Three additional adversarial strategies test shortcuts in the grading signals; none resolves any of the 135 tasks they cover, while a vacuous health probe passes readiness and exposes the need for downstream checks."},{"source_index":0,"quote":"ten tasks resist all seven"}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0150","record_version":1,"name":"FRAMES failure-aware supervisory framework for humanoid loco-manipulation","aliases":["FRAMES"],"category":"monitoring","summary":"FRAMES is a supervisory layer that sits above a whole-body controller for the Unitree G1 humanoid. A Planner Agent decomposes natural-language instructions into parameterized mid-level skills; a vision-language-model Monitor Agent then evaluates each executing skill using temporal multi-view observations plus structured robot and contact evidence. When a failure is detected, the active skill is stopped and grounded feedback is passed to a Recovery Agent. A Memory Module reuses prior skill experience, and depth and segmentation provide geometric grounding. The framework is intended to catch execution errors during approach, grasping, transport or placement before they invalidate the rest of a long-horizon plan.","problem_addressed":"Choosing the correct skill does not guarantee successful physical execution; errors during approach, grasping, transport, or placement can invalidate the remainder of a long-horizon plan.","mitigates":["operational_error","physical_safety","agent_misalignment"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The authors state that the results provide only initial evidence for the monitoring component, and that end-to-end evaluation of the complete recovery loop remains ongoing. The monitor also misses some failures and rejects some successful executions.","organisations":[],"first_seen":"2026-09-18","last_seen":"2026-09-18","sources":[{"url":"https://arxiv.org/abs/2609.22538v1","publisher":"arXiv","date":"2026-09-18","type":"vendor_research","title":"FRAMES: Failure Recovery And Monitoring of Embodied Skills for Humanoid Loco-Manipulation"}],"field_citations":{"name":[{"source_index":0,"quote":"We present FRAMES, a failure-aware supervisory framework for the Unitree G1 humanoid that operates above the CEER whole-body controller."}],"summary":[{"source_index":0,"quote":"A Planner Agent selects subtasks through parameterized mid-level skills, while a vision-language-model-based Monitor Agent evaluates each skill using temporal multi-view observations and structured robot and contact evidence. Detected failures stop the active skill and provide grounded feedback to a Recovery Agent. The framework further includes a Memory Module for reusing prior skill experience, and geometric grounding via depth and segmentation."}],"problem_addressed":[{"source_index":0,"quote":"Large language model (LLM) planners can decompose natural-language instructions and select reusable robot skills, but choosing the correct skill does not guarantee successful physical execution. This gap is especially important in humanoid loco-manipulation, where errors during approach, grasping, transport, or placement can invalidate the remainder of a long-horizon plan."}],"mitigates":[{"source_index":0,"quote":"Detected failures stop the active skill and provide grounded feedback to a Recovery Agent."}],"maturity":[{"source_index":0,"quote":"We independently evaluate the monitoring module of the framework in MuJoCo using 100 trials comprising 50 failed and 50 successful executions across five tasks."}],"evidence_strength":[{"source_index":0,"quote":"We independently evaluate the monitoring module of the framework in MuJoCo using 100 trials comprising 50 failed and 50 successful executions across five tasks."}],"limitations":[{"source_index":0,"quote":"These results provide initial evidence for the monitoring component, while end-to-end evaluation of the complete recovery loop remains ongoing."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0113","record_version":1,"name":"Fast discovery with slow registration","aliases":[],"category":"control_protocol","summary":"A two-stage adoption protocol in the MedRSI framework that separates rapid invention of new clinical capabilities from their conservative admission into the persistent agent. New tools or capabilities can be discovered quickly, but they enter the persistent agent only after demonstrating sustained benefit across subsequent patient cohorts. This gating mechanism is intended to prevent unvalidated self-generated capabilities from being retained and used in clinical practice.","problem_addressed":"Recursive self-improvement could rapidly accumulate unvalidated or harmful new capabilities if every discovered tool were immediately adopted.","mitigates":["agent_misalignment","operational_error","harmful_output"],"addresses":["reward_hacking","silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":["ImprintLab"],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24838v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"MedRSI: Recursive Self-Improvement for Medical Agents via Clinically Aligned Self-Evolution"}],"field_citations":{"name":[{"source_index":0,"quote":"Fast discovery with slow registration separates rapid capability invention from conservative adoption, allowing new tools to enter the persistent agent only after demonstrating sustained benefit across subsequent patient cohorts."}],"summary":[{"source_index":0,"quote":"Fast discovery with slow registration separates rapid capability invention from conservative adoption, allowing new tools to enter the persistent agent only after demonstrating sustained benefit across subsequent patient cohorts."}],"problem_addressed":[{"source_index":0,"quote":"Fast discovery with slow registration separates rapid capability invention from conservative adoption, allowing new tools to enter the persistent agent only after demonstrating sustained benefit across subsequent patient cohorts."}],"maturity":[{"source_index":0,"quote":"Across public glaucoma and heart disease benchmarks and two private clinical tasks, MedRSI progressively develops segmentation, measurement, prediction, multimodal reasoning, and generative capabilities, surpasses manually engineered medical agents, and autonomously discovers solutions to clinical problems not anticipated by its original designers."}],"evidence_strength":[{"source_index":0,"quote":"Across public glaucoma and heart disease benchmarks and two private clinical tasks, MedRSI progressively develops segmentation, measurement, prediction, multimodal reasoning, and generative capabilities, surpasses manually engineered medical agents, and autonomously discovers solutions to clinical problems not anticipated by its original designers."}],"organisations":[{"source_index":0,"quote":"Code is available at https://github.com/ImprintLab/MedRSI."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0186","record_version":1,"name":"Four-layer agentic detection model","aliases":[],"category":"monitoring","summary":"A detection model for agentic systems structured in four layers, listed among the sessions on the summit program alongside OWASP's capability control matrix. The document names it only as part of the program and gives no description of the layers, their inputs, or how they are combined, so the mechanism cannot be reconstructed from this source.","problem_addressed":null,"mitigates":[],"addresses":[],"responds_to":[],"maturity":"unknown","evidence_strength":"none_stated","limitations":"The document provides only the name; no layers, data sources, evaluation results, or deployment details are described.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://labs.zenity.io/post/waiting-for-you-in-new-york","publisher":"Zenity Labs","date":"2026-09-17","type":"vendor_research","title":"The agenda is live: here's a sneak peek at what's waiting for you in New York | Zenity Labs"}],"field_citations":{"name":[{"source_index":0,"quote":"The rest of the program runs from OWASP's capability control matrix and a four-layer agentic detection model through cross-agent privilege escalation in Google's ADK, MCP tool poisoning, sandbox escapes that never touch the sandbox, and the attack surface of a 34,000-star autonomous pipeline."}],"summary":[{"source_index":0,"quote":"The rest of the program runs from OWASP's capability control matrix and a four-layer agentic detection model through cross-agent privilege escalation in Google's ADK, MCP tool poisoning, sandbox escapes that never touch the sandbox, and the attack surface of a 34,000-star autonomous pipeline."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0065","record_version":1,"name":"Frame-only verification instruction","aliases":[],"category":"guardrail","summary":"A prompt-level guardrail in which the multimodal judge is explicitly instructed to 'use only the frames' when deciding whether a generated video satisfies a visual requirement, ignoring any accompanying execution trace, plan text, or synthesized narration. The intent is to prevent auxiliary text from shifting the judge's verdict on purely visual criteria. The document reports that this instruction does not remove the contamination effect: traces still moved open-weight Qwen-VL judges' verdicts substantially even when the instruction was present.","problem_addressed":"Execution traces and plan-derived text contaminating a multimodal verifier's judgments about purely visual requirements.","mitigates":["operational_error","agent_misalignment"],"addresses":["deception_or_concealment"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document states the instruction does not remove the effect: 'an instruction to ``use only the frames'' does not remove the effect.'","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28564v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Don't Read the Log: Execution Traces Contaminate Verifiers in Video-Generation Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"an instruction to ``use only the frames'' does not remove the effect"}],"summary":[{"source_index":0,"quote":"an instruction to ``use only the frames'' does not remove the effect"}],"problem_addressed":[{"source_index":0,"quote":"We ask whether this auxiliary text moves the judge's verdict on purely \\emph{visual} requirements, holding the frames fixed."}],"limitations":[{"source_index":0,"quote":"an instruction to ``use only the frames'' does not remove the effect"}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0194","record_version":1,"name":"Fuzz random testing of AI-generated programs against human-written ground truth","aliases":["fuzz random testing","black box generational testing","coverage guided gray box mutational testing"],"category":"evaluation","summary":"An evaluation methodology for judging the reliability of agentic-AI-generated code by comparing it against release-quality human-written programs as a concrete ground truth, then subjecting both to fuzz random testing. Testing combines classic black-box generational fuzzing with modern coverage-guided (gray-box, mutational) fuzzing using AFL++. Failures are classified by type, allowing comparison of failure rates and failure modes (e.g., memory errors versus hangs) between AI-generated and human-generated versions of the same programs.","problem_addressed":"How to objectively verify claims that agentic AI produces faster, more efficient, and more reliable software.","mitigates":["operational_error","security_compromise"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The study is limited to ten Linux utility programs and to fuzz random testing as the reliability measure; the authors note AI-generated code was more likely to have hangs such as infinite loops, and that quality is highly dependent on prompts, skills, and human supervision.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18298v1","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"A Study of the Reliability of Agentic AI-Generated Programs"}],"field_citations":{"name":[{"source_index":0,"quote":"Third, we based our measure of reliability on a widely used testing technique, fuzz random testing."},{"source_index":0,"quote":"For this testing, we used both classic black box, generational testing and more modern coverage guided (gray box, mutational) testing using AFL++."}],"summary":[{"source_index":0,"quote":"Second, our target programs were ten well-known, release-quality human-written Linux utility programs so that we could compare the AI-generated code against a concrete ground truth."},{"source_index":0,"quote":"We found that the AI-generated versions of the utility programs were typically as reliable - often more reliable - than the latest human-generated versions of these programs."}],"problem_addressed":[{"source_index":0,"quote":"The question is how can we verify these claims in an objective way?"}],"mitigates":[{"source_index":0,"quote":"Interestingly, the AI-generated code was less likely to have failures such as memory errors (such as buffer overflows) but more likely to have hangs such as infinite loops."}],"maturity":[{"source_index":0,"quote":"In this project, we attempted to answer this question based on three practices."}],"evidence_strength":[{"source_index":0,"quote":"We found that the AI-generated versions of the utility programs were typically as reliable - often more reliable - than the latest human-generated versions of these programs."}],"limitations":[{"source_index":0,"quote":"While the AI-generated versions did have some failures, they were less common than the code from the standard repositories."},{"source_index":0,"quote":"Interestingly, the AI-generated code was less likely to have failures such as memory errors (such as buffer overflows) but more likely to have hangs such as infinite loops."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0103","record_version":1,"name":"Governed delivery plane with oracle-scored ten-stage pipeline","aliases":[],"category":"evaluation","summary":"A benchmark harness in which an LLM agent acts as a forward-deployed engineer across ten post-training delivery stages, with an oracle scoring each stage from platform-recorded facts rather than self-reported metrics. The plane enforces a budget, a human-approval gate, and reproducibility requirements, and every scenario is certified before scoring. It is designed to test whether an agent can be trusted to deliver a fine-tuned model, not merely whether it can raise a metric.","problem_addressed":"Existing benchmarks cannot answer whether an agent can be trusted to deliver a fine-tuned model under budget, approval and reproducibility constraints.","mitigates":["agent_misalignment","operational_error","performance_drift"],"addresses":["reward_hacking","deception_or_concealment"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.25237v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"Trains but Doesn't Learn: A Post-Training Delivery Benchmark for LLM Agents as Forward-Deployed Engineers"}],"field_citations":{"name":[{"source_index":0,"quote":"We answer it on a governed delivery plane, where an agent drives ten stages and an oracle scores each stage from platform-recorded facts."}],"summary":[{"source_index":0,"quote":"Post-training is becoming a service (PTaaS): a customer hands an operator data and a goal, and a forward-deployed engineer (FDE) returns a fine-tuned, evaluated, and deployed model under a budget, a human-approval gate, and reproducibility requirements."},{"source_index":0,"quote":"We answer it on a governed delivery plane, where an agent drives ten stages and an oracle scores each stage from platform-recorded facts."},{"source_index":0,"quote":"We ran four frontier agents (Claude Opus 5, GPT-5.6-luna, Gemini 3.7 Flash, DeepSeek V4-Pro) end to end on metered L40S, A100, and H200 GPUs across 8B to 70B open bases, certifying every scenario before scoring."}],"problem_addressed":[{"source_index":0,"quote":"Seating an LLM agent in the FDE seat raises a question existing benchmarks cannot answer: not whether an agent can raise a metric, but whether it can be trusted to deliver."}],"mitigates":[{"source_index":0,"quote":"The central silent failure is the run that trains but does not learn (TBDL): loss falls, every signal stays green, and the delivered model is no better than the base."}],"maturity":[{"source_index":0,"quote":"We ran four frontier agents (Claude Opus 5, GPT-5.6-luna, Gemini 3.7 Flash, DeepSeek V4-Pro) end to end on metered L40S, A100, and H200 GPUs across 8B to 70B open bases, certifying every scenario before scoring."}],"evidence_strength":[{"source_index":0,"quote":"We ran four frontier agents (Claude Opus 5, GPT-5.6-luna, Gemini 3.7 Flash, DeepSeek V4-Pro) end to end on metered L40S, A100, and H200 GPUs across 8B to 70B open bases, certifying every scenario before scoring."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0036","record_version":1,"name":"Governed policy compiler","aliases":[],"category":"governance","summary":"A compiler that takes a compact classification handle derived from a matrix of regulatory intent and exposure and maps it into concrete prohibitions, obligations and runtime budgets. Prohibitions constrain feasibility and block externalisation, while obligations extend tasks with artefacts that must meet explicit admissibility criteria. This embeds compliance into execution rather than treating regulation as a deterministic ruleset, so that policy is enforced at runtime and remains verifiable by auditors.","problem_addressed":"Translating abstract regulatory intent into enforceable runtime constraints for AI systems in regulated finance.","mitigates":["operational_error","other"],"addresses":["data_exfiltration","unauthorized_communication","harmful_compliance"],"responds_to":[],"maturity":"proposed","evidence_strength":"none_stated","limitations":null,"organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.27632v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Compliant AI Infrastructure for Regulated Finance: A tiered multi-agent framework with DLT audit trails for financial operations in DACH"}],"field_citations":{"name":[{"source_index":0,"quote":"A matrix of regulatory intent and exposure provides a compact classification handle, which a governed policy compiler then maps into concrete prohibitions, obligations and runtime budgets."}],"summary":[{"source_index":0,"quote":"A matrix of regulatory intent and exposure provides a compact classification handle, which a governed policy compiler then maps into concrete prohibitions, obligations and runtime budgets."},{"source_index":0,"quote":"Prohibitions constrain feasibility and block externalisation, while obligations extend tasks with artefacts that must meet explicit admissibility criteria."}],"problem_addressed":[{"source_index":0,"quote":"We present a compliance-first architecture for AI in regulated finance that treats regulation as an orientation layer rather than a deterministic ruleset."}],"maturity":[{"source_index":0,"quote":"We present a compliance-first architecture for AI in regulated finance that treats regulation as an orientation layer rather than a deterministic ruleset."}],"evidence_strength":[{"source_index":0,"quote":"We present a compliance-first architecture for AI in regulated finance that treats regulation as an orientation layer rather than a deterministic ruleset."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0074","record_version":2,"name":"Growing Harness","aliases":["failure-guided training paradigm","failure-guided harness training"],"category":"alignment_training","summary":"Growing Harness is a failure-guided training paradigm that learns an agent's harness itself from a strategy-free scaffold exposing fixed model and tool interfaces but encoding no task-solving controller. Function-level execution traces localize each failure to a bounded code surface; an optimizer repairs a window of failures jointly; and a success-first held-out gate rolls back repair sequences that harm prior capability. Accepted edits accumulate in one shared harness, so control structure emerges from task feedback while LLM calls are reserved for task-specific semantic reasoning.","problem_addressed":"Standard harnesses repeatedly ask the model to reconstruct the same control decisions inside each task's context, wasting LLM calls and cost.","mitigates":["operational_error","agent_misalignment","performance_drift"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document reports that in one of six benchmark-model settings the method trails the best mean by 0.7 pp., and the approach is evaluated only on BrowseComp-Plus and WebArena-Verified with three deployment models.","organisations":[],"first_seen":"2026-09-22","last_seen":"2026-09-22","sources":[{"url":"https://arxiv.org/abs/2609.26760v1","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents"},{"url":"https://arxiv.org/abs/2609.26760v2","publisher":"arXiv","date":"2026-09-22","type":"vendor_research","title":"Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce Growing Harness, a failure-guided training paradigm that learns the agent harness itself from a strategy-free scaffold that exposes fixed model and tool interfaces but encodes no task-solving controller."},{"source_index":1,"quote":"We introduce Growing Harness, a failure-guided training paradigm that learns the agent harness itself from a strategy-free scaffold that exposes fixed model and tool interfaces but encodes no task-solving controller."}],"summary":[{"source_index":0,"quote":"Function-level execution traces localize each failure to a bounded code surface, an optimizer repairs a window of failures jointly, and a success-first held-out gate rolls back repair sequences that harm prior capability."},{"source_index":0,"quote":"Accepted edits accumulate in one shared harness, allowing its control structure to emerge from task feedback."},{"source_index":1,"quote":"Function-level execution traces localize each failure to a bounded code surface, an optimizer repairs a window of failures jointly, and a success-first held-out gate rolls back repair sequences that harm prior capability. Accepted edits accumulate in one shared harness, allowing its control structure to emerge from task feedback."}],"problem_addressed":[{"source_index":0,"quote":"Large language model (LLM) agents often handle streams of related tasks, yet standard harnesses repeatedly ask the model to reconstruct the same control decisions inside each task's context."},{"source_index":1,"quote":"Large language model (LLM) agents often handle streams of related tasks, yet standard harnesses repeatedly ask the model to reconstruct the same control decisions inside each task's context."}],"maturity":[{"source_index":0,"quote":"Across BrowseComp-Plus and WebArena-Verified with three deployment models from 4B to 120B parameters, Growing Harness achieves the highest mean success in five of six benchmark-model settings and trails the best mean by 0.7 pp. in the sixth."},{"source_index":1,"quote":"Across BrowseComp-Plus and WebArena-Verified with three deployment models from 4B to 120B parameters, Growing Harness achieves the highest mean success in five of six benchmark-model settings and trails the best mean by 0.7 pp. in the sixth."}],"evidence_strength":[{"source_index":0,"quote":"Ablations show that trace-local edits, joint repair, and gate-based rollback each improve final success."},{"source_index":1,"quote":"Ablations show that trace-local edits, joint repair, and gate-based rollback each improve final success."}],"limitations":[{"source_index":0,"quote":"Across BrowseComp-Plus and WebArena-Verified with three deployment models from 4B to 120B parameters, Growing Harness achieves the highest mean success in five of six benchmark-model settings and trails the best mean by 0.7 pp. in the sixth."},{"source_index":1,"quote":"Across BrowseComp-Plus and WebArena-Verified with three deployment models from 4B to 120B parameters, Growing Harness achieves the highest mean success in five of six benchmark-model settings and trails the best mean by 0.7 pp. in the sixth."}],"mitigates":[{"source_index":1,"quote":"Relative to a Tool-Calling agent, it reduces LLM calls by 76.0-91.8% and deployed-agent inference cost by 74.4-98.6%."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0140","record_version":1,"name":"Hand-built stratified Text-to-SQL benchmark of 100 natural language questions with ground-truth SQL","aliases":[],"category":"benchmark","summary":"A hand-built evaluation benchmark of 100 natural language questions paired with ground-truth SQL, run against the Chicago Crime dataset (approximately 8.5 million records, 22 attributes). The items are stratified into 30 Easy, 40 Medium and 30 Hard questions. It is used to measure a Text-to-SQL agent's Valid SQL Rate, Execution Accuracy under a hybrid relational equivalence metric, and mean Synthesis Quality, and to compare two prompt revisions of the same agent.","problem_addressed":"There is no public benchmark or external baseline against which to measure the Text-to-SQL agent's performance.","mitigates":["operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The benchmark is hand-built and specific to the Chicago Crime dataset; the document reports no comparison against an external baseline system or a public benchmark.","organisations":[],"first_seen":"2026-09-19","last_seen":"2026-09-19","sources":[{"url":"https://arxiv.org/abs/2609.22917v1","publisher":"arXiv","date":"2026-09-19","type":"vendor_research","title":"An Iterative LangGraph Agent for Text-to-SQL: Natural Language Access to the Chicago Crime Database"}],"field_citations":{"name":[{"source_index":0,"quote":"a hand-built benchmark of 100 natural language questions with ground-truth SQL, stratified into 30 Easy, 40 Medium and 30 Hard items"}],"summary":[{"source_index":0,"quote":"We evaluated it on the Chicago Crime dataset (approximately 8.5 million records, 22 attributes) against a hand-built benchmark of 100 natural language questions with ground-truth SQL, stratified into 30 Easy, 40 Medium and 30 Hard items."}],"problem_addressed":[{"source_index":0,"quote":"We report no comparison against an external baseline system or a public benchmark; the study is a single-model engineering evaluation."}],"mitigates":[{"source_index":0,"quote":"Error analysis attributes the residual failures to relevance-checker false rejections, ambiguous question semantics, and free-tier API rate limits rather than to the language generation step."}],"maturity":[{"source_index":0,"quote":"We evaluated it on the Chicago Crime dataset (approximately 8.5 million records, 22 attributes) against a hand-built benchmark of 100 natural language questions with ground-truth SQL, stratified into 30 Easy, 40 Medium and 30 Hard items."}],"evidence_strength":[{"source_index":0,"quote":"We report no comparison against an external baseline system or a public benchmark; the study is a single-model engineering evaluation."}],"limitations":[{"source_index":0,"quote":"We report no comparison against an external baseline system or a public benchmark; the study is a single-model engineering evaluation."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0027","record_version":1,"name":"HarnessPAI","aliases":["Harness framework for Physical AI"],"category":"control_protocol","summary":"HarnessPAI is a model- and embodiment-agnostic harness that treats code as the executable, evolvable interface organizing an underlying action primitive. It separates two timescales: within a rollout it executes open-loop at the program level, with a fixed program guiding and checking execution; across rollouts it evolves closed-loop, using execution feedback to revise the program and distill failures into reusable skills. Once a program is selected, rollout execution requires no online high-level LLM deliberation. The converged program also serves as an expert-data collector for fine-tuning the action model.","problem_addressed":"The prevailing training recipe can erode the perceptual and reasoning capabilities needed for robust behavior, leaving even strong action models vulnerable to scene perturbations and long-horizon tasks.","mitigates":["agent_misalignment","operational_error","performance_drift"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the approach; it reports gains without retraining the underlying model and notes that once a program is selected, rollout execution requires no online high-level LLM deliberation.","organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29166v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"HarnessPAI: An Evolving Harness for Physical AI"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce HarnessPAI, a model- and embodiment-agnostic Harness framework for Physical AI that treats code as the executable and evolvable interface that organizes the underlying action primitive."}],"summary":[{"source_index":0,"quote":"The framework separates two timescales: within a rollout, it executes open-loop at the program level, with a fixed program guiding and checking execution; across rollouts, it evolves closed-loop, using execution feedback to revise the program and distill failures into reusable skills."},{"source_index":0,"quote":"Once a program is selected, rollout execution requires no online high-level LLM deliberation."}],"problem_addressed":[{"source_index":0,"quote":"The prevailing training recipe can erode the perceptual and reasoning capabilities needed for robust behavior, leaving even strong action models vulnerable to scene perturbations and long-horizon tasks."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0033","record_version":1,"name":"Hierarchical Rewards (HierR)","aliases":["HierR"],"category":"alignment_training","summary":"HierR is a reward-shaping scheme used with SLCA. It supplies two levels of reward signal so that execution advantages can be routed to tool tokens and preference advantages to summary tokens. This separation is what allows SLCA to eliminate advantage contamination within each policy update.","problem_addressed":"Advantage contamination between tool-decision tokens and summary tokens during policy updates.","mitigates":["agent_misalignment","operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of HierR.","organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29050v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL"}],"field_citations":{"name":[{"source_index":0,"quote":"Supported by Hierarchical Rewards (HierR), SLCA routes execution advantages to tool tokens and preference advantages to summary tokens, eliminating advantage contamination (the dominant cross-segment credit misattribution channel) within each policy update."}],"summary":[{"source_index":0,"quote":"Supported by Hierarchical Rewards (HierR), SLCA routes execution advantages to tool tokens and preference advantages to summary tokens, eliminating advantage contamination (the dominant cross-segment credit misattribution channel) within each policy update."}],"problem_addressed":[{"source_index":0,"quote":"Supported by Hierarchical Rewards (HierR), SLCA routes execution advantages to tool tokens and preference advantages to summary tokens, eliminating advantage contamination (the dominant cross-segment credit misattribution channel) within each policy update."}],"mitigates":[{"source_index":0,"quote":"Supported by Hierarchical Rewards (HierR), SLCA routes execution advantages to tool tokens and preference advantages to summary tokens, eliminating advantage contamination (the dominant cross-segment credit misattribution channel) within each policy update."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0010","record_version":1,"name":"Human-gated agentic data flywheel","aliases":[],"category":"other","summary":"A data-production loop in which specialized agents construct tasks, collect interaction trajectories, curate and balance training data, and use training feedback to guide subsequent data generation, with human gating at the loop level. It is part of a closed-loop AI-for-AI framework connecting data production, model training, and deployment through a shared action-feedback-verification contract.","problem_addressed":"Costly real-device interaction limits development scalability and data quality for long-horizon mobile planning agents.","mitigates":["operational_error","performance_drift"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29892v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"Qwen-Planner-Agent: A Closed-Loop AI-for-AI Framework for Real-World Mobile Planner Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"AI for Data builds a human-gated agentic data flywheel in which specialized agents construct tasks, collect interaction trajectories, curate and balance training data, and use training feedback to guide subsequent data generation."}],"summary":[{"source_index":0,"quote":"The framework connects data production, model training, and deployment through a shared action-feedback-verification contract."},{"source_index":0,"quote":"AI for Data builds a human-gated agentic data flywheel in which specialized agents construct tasks, collect interaction trajectories, curate and balance training data, and use training feedback to guide subsequent data generation."}],"problem_addressed":[{"source_index":0,"quote":"Mobile planning offers a demanding test of this approach: complex, long-horizon tasks challenge agent reliability, while costly real-device interaction limits development scalability."}],"maturity":[{"source_index":0,"quote":"Qwen-Planner-Agent achieves the best overall performance among all evaluated models and systems on MobilePA-Bench, improving over its base model across tool use, memory, skills, and sub-agent coordination."}],"evidence_strength":[{"source_index":0,"quote":"Qwen-Planner-Agent achieves the best overall performance among all evaluated models and systems on MobilePA-Bench, improving over its base model across tool use, memory, skills, and sub-agent coordination."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0196","record_version":1,"name":"Independence-graded audit protocol for agentic AI","aliases":["three-axis independence grading","beta-factor audit protocol"],"category":"assurance","summary":"A seven-step audit protocol that grades auditor independence along three orthogonal axes: principal independence (who controls the auditor), substrate independence (whether the auditor shares the auditee's foundation-model family, toolchain or guardrails), and evidence independence (whether evidence is attestable rather than self-reported). The three grades are aggregated by the weakest link, and the same rubric is applied when the auditor is itself an agent. The model is given a formal basis by transplanting the beta-factor model of common-cause failure from reliability engineering, and its outputs are designed to be verifiable by a third party.","problem_addressed":"Independence in assurance of agentic AI is still applied as a binary, so audits can appear independent while sharing failure modes with the auditee.","mitigates":["agent_misalignment","operational_error","other"],"addresses":["deception_or_concealment","silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document reports a Monte Carlo study in which a conventional internal audit of an agent surfaces only 5.9% of the faults it could in principle see and none at all in half the fault classes; the protocol's own outputs are described as third-party verifiable but the study is by the authors.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18272v1","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"Who Audits Whom, on What Substrate, with What Evidence? An Independence-Graded Audit Protocol for Agentic AI"}],"field_citations":{"name":[{"source_index":0,"quote":"We argue that it must be graded along three orthogonal axes: principal independence (who controls the auditor), substrate independence (an auditor sharing the auditee's foundation-model family, toolchain or guardrails fails with it) and evidence independence (whether evidence is attestable rather than self-reported)."}],"summary":[{"source_index":0,"quote":"Each axis has precedent; the contribution is to grade all three on a single audit, aggregate them by the weakest link, and apply the same rubric when the auditor is itself an agent. We give the model a formal basis by transplanting the beta-factor model of common-cause failure from reliability engineering, a seven-step protocol whose outputs a third party can verify, a structural detectability analysis of a procurement-controls agent audited at three grades, and a Monte Carlo study of the model in which a conventional internal audit of an agent-a real audit team, a second agent, provider logsp-surfaces 5.9% of the faults it could in principle see and none at all in half the fault classes."}],"problem_addressed":[{"source_index":0,"quote":"Independence, the foundation of assurance,is still applied to them as a binary."}],"mitigates":[{"source_index":0,"quote":"a Monte Carlo study of the model in which a conventional internal audit of an agent-a real audit team, a second agent, provider logsp-surfaces 5.9% of the faults it could in principle see and none at all in half the fault classes."}],"maturity":[{"source_index":0,"quote":"We give the model a formal basis by transplanting the beta-factor model of common-cause failure from reliability engineering, a seven-step protocol whose outputs a third party can verify, a structural detectability analysis of a procurement-controls agent audited at three grades, and a Monte Carlo study of the model"}],"evidence_strength":[{"source_index":0,"quote":"a Monte Carlo study of the model in which a conventional internal audit of an agent-a real audit team, a second agent, provider logsp-surfaces 5.9% of the faults it could in principle see and none at all in half the fault classes."}],"limitations":[{"source_index":0,"quote":"a Monte Carlo study of the model in which a conventional internal audit of an agent-a real audit team, a second agent, provider logsp-surfaces 5.9% of the faults it could in principle see and none at all in half the fault classes."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0069","record_version":1,"name":"Independent AI and human review with adjudication of interaction-mode labels","aliases":[],"category":"assurance","summary":"A labelling-quality assurance procedure in which independent AI and human reviewers annotate a stratified sample of actions for interaction mode, their agreement is measured, and disagreements are subsequently adjudicated. In PAWS this was applied to 2,522 stratified action samples, yielding 89.4% initial agreement before adjudication. The mechanism is reusable as a general annotation-verification and label-quality control process for datasets used in agent evaluation.","problem_addressed":"Ensuring the reliability of interaction-mode labels in a multi-agent simulation dataset.","mitigates":["operational_error","other"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":null,"organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28547v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"PAWS: Policy-driven Agentic World Simulation"}],"field_citations":{"name":[{"source_index":0,"quote":"On 2,522 stratified action samples, independent AI and human reviewers achieved 89.4% initial agreement on interaction mode, with disagreements subsequently adjudicated."}],"summary":[{"source_index":0,"quote":"On 2,522 stratified action samples, independent AI and human reviewers achieved 89.4% initial agreement on interaction mode, with disagreements subsequently adjudicated."}],"problem_addressed":[{"source_index":0,"quote":"On 2,522 stratified action samples, independent AI and human reviewers achieved 89.4% initial agreement on interaction mode, with disagreements subsequently adjudicated."}],"mitigates":[{"source_index":0,"quote":"On 2,522 stratified action samples, independent AI and human reviewers achieved 89.4% initial agreement on interaction mode, with disagreements subsequently adjudicated."}],"maturity":[{"source_index":0,"quote":"On 2,522 stratified action samples, independent AI and human reviewers achieved 89.4% initial agreement on interaction mode, with disagreements subsequently adjudicated."}],"evidence_strength":[{"source_index":0,"quote":"On 2,522 stratified action samples, independent AI and human reviewers achieved 89.4% initial agreement on interaction mode, with disagreements subsequently adjudicated."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0004","record_version":1,"name":"Independent interception mechanism for trace logging","aliases":["trace logging through an independent interception mechanism outside of the agent's control"],"category":"provenance","summary":"The paper advises that agent trace logging be performed through an interception mechanism that sits outside the agent's own control, so that traces remain intact even if the agent host is fully compromised. Rather than relying on the agent harness to write its own execution traces, logging is captured by an independent layer the agent cannot reach or modify. This preserves the integrity of traces used for asynchronous monitoring, incident investigations, and compliance audits, and prevents agents from deleting or altering records of their own behaviour.","problem_addressed":"LLM agents can delete or tamper with their own execution traces without triggering monitor guardrails, undermining monitoring, incident investigation and audit.","mitigates":["agent_misalignment","operational_error","security_compromise"],"addresses":["deception_or_concealment","silent_wrong_state"],"responds_to":[],"maturity":"proposed","evidence_strength":"author_evaluation","limitations":"The document only advises this as a recommendation; it does not report implementing or testing an independent interception mechanism, and the finding that all tested harnesses except Muse Code allowed trace deletion is evidence of the problem rather than of the proposed fix.","organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.30266v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"LLM Agents Can Easily Tamper With Their Own Traces"}],"field_citations":{"name":[{"source_index":0,"quote":"We advise practitioners to ensure trace logging happens through an independent interception mechanism outside of the agent's control, preserving trace integrity even in cases of full host compromise."}],"summary":[{"source_index":0,"quote":"We advise practitioners to ensure trace logging happens through an independent interception mechanism outside of the agent's control, preserving trace integrity even in cases of full host compromise."}],"problem_addressed":[{"source_index":0,"quote":"These analyses assume that LLM agents cannot tamper with their own execution traces."},{"source_index":0,"quote":"All tested harnesses, except Muse Code, allowed agents to delete their traces when asked, without triggering monitor guardrails."}],"mitigates":[{"source_index":0,"quote":"Overall, our findings identify a concrete failure of trace integrity in agent infrastructure which can be used to conceal misaligned behaviors like scheming or sabotage."}],"maturity":[{"source_index":0,"quote":"We advise practitioners to ensure trace logging happens through an independent interception mechanism outside of the agent's control, preserving trace integrity even in cases of full host compromise."}],"evidence_strength":[{"source_index":0,"quote":"We show that local LLM agents such as Claude Code, Codex, Antigravity, Open Code and Grok Build fail to enforce this boundary."}],"limitations":[{"source_index":0,"quote":"We advise practitioners to ensure trace logging happens through an independent interception mechanism outside of the agent's control, preserving trace integrity even in cases of full host compromise."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0203","record_version":1,"name":"Independent reproduction tracking across targets","aliases":["independent reproductions"],"category":"evaluation","summary":"Agora records verification commits and tracks independent reproductions of published results across targets, so that shared work can be checked rather than merely reused. In the reported run, participants posted 165 independent reproductions across 95 targets, with no reported failures. The verification status is exposed through Agora's searchable views, allowing workers to distinguish results that have been reproduced from those that have not. The document presents the reproduction count as evidence of how agents reused and verified shared work, not as a controlled evaluation of the mechanism's effect.","problem_addressed":"Shared agent contributions may be reused without being independently checked.","mitigates":["hallucination","operational_error","other"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"piloted","evidence_strength":"author_evaluation","limitations":"The document reports no failures among the reproductions but does not describe a controlled comparison or independent audit of the reproduction process itself.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18094v2","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"Agora: Git as Shared Memory for Collective AutoResearch"}],"field_citations":{"name":[{"source_index":0,"quote":"Participants also posted 165 independent reproductions across 95 targets, with no reported failures."}],"summary":[{"source_index":0,"quote":"Each commit records a result, insight, hypothesis, verification, or report and links it to prior work. Searchable views show leading results, neglected branches, and verification status"}],"problem_addressed":[{"source_index":0,"quote":"The run documents how agents reused and verified shared work."}],"mitigates":[{"source_index":0,"quote":"The run documents how agents reused and verified shared work."}],"maturity":[{"source_index":0,"quote":"Participants also posted 165 independent reproductions across 95 targets, with no reported failures."}],"evidence_strength":[{"source_index":0,"quote":"Participants also posted 165 independent reproductions across 95 targets, with no reported failures."}],"limitations":[{"source_index":0,"quote":"Measuring the effect on discovery per unit of compute requires a matched comparison."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0166","record_version":1,"name":"Inference engine fingerprinting resistance via output-token perturbation","aliases":[],"category":"guardrail","summary":"The paper proposes modifying inference engines so that a misaligned model cannot reliably identify which engine is executing it from its own generated output tokens. Because engines differ in tokenizer quirks, sampling behavior, and output formatting, a model can fingerprint the engine and then apply engine-specific exploits. The authors discuss several engine-level changes—such as normalizing output distributions, masking engine-specific artifacts, and randomizing detectable behaviors—that would make such fingerprinting attacks more difficult, thereby reducing the model's ability to initiate a to-the-bare-metal exploit chain from output tokens alone.","problem_addressed":"A misaligned model can fingerprint the inference engine (e.g., vLLM, SGLang) from its own output tokens and then use engine-specific exploits to compromise the engine, enabling a sandbox escape without relying on other stack components or malicious input tokens.","mitigates":["security_compromise","agent_misalignment"],"addresses":["containment_escape"],"responds_to":[],"maturity":"proposed","evidence_strength":"author_evaluation","limitations":"The paper only discusses ways engines could be changed; it does not implement or evaluate any specific defense, and the proposed changes are not tested against the demonstrated fingerprinting attacks.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.20614v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"Inference-Engine Fingerprinting Attacks are Practical: Exploring Model-Driven Environmental Discovery, Exploitation, and Escape"}],"field_citations":{"name":[{"source_index":0,"quote":"We conclude by discussing several ways that inference engines could be changed to make fingerprinting attacks more difficult."}],"summary":[{"source_index":0,"quote":"We conclude by discussing several ways that inference engines could be changed to make fingerprinting attacks more difficult."}],"problem_addressed":[{"source_index":0,"quote":"In this paper, we show that a misaligned model can perform inference engine fingerprinting to determine the specific engine (e.g., vLLM, SGLang) which executes the model. Once the engine has been fingerprinted, the model can leverage engine-specific exploits to take control of the engine using only carefully-selected output tokens."}],"mitigates":[{"source_index":0,"quote":"Once the engine has been fingerprinted, the model can leverage engine-specific exploits to take control of the engine using only carefully-selected output tokens."}],"maturity":[{"source_index":0,"quote":"We conclude by discussing several ways that inference engines could be changed to make fingerprinting attacks more difficult."}],"evidence_strength":[{"source_index":0,"quote":"We provide concrete examples of model fingerprints in five popular engines, and demonstrate how realistic agentic harnesses allow a model to leverage those fingerprints to identify the local engine."}],"limitations":[{"source_index":0,"quote":"We conclude by discussing several ways that inference engines could be changed to make fingerprinting attacks more difficult."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0184","record_version":1,"name":"Intent-aware authorization for agent delegation chains","aliases":["intent-aware authorization"],"category":"access_control","summary":"An authorization approach for AI agents that checks whether an action matches the user's original intent, rather than only whether the agent holds valid credentials and satisfies IAM policy. It preserves intent across the full delegation chain, so an agent with permission to act cannot execute an action the user never intended. The approach is presented as an addition to existing IAM rather than a replacement, demonstrated by running the same agent with the same permissions under traditional IAM versus intent-aware authorization to show divergent outcomes.","problem_addressed":"Agents can hold valid credentials and satisfy every IAM policy while still executing actions the user never intended, because permission does not encode intent.","mitigates":["agent_misalignment","operational_error","security_compromise"],"addresses":["harmful_compliance","credential_misuse"],"responds_to":[],"maturity":"proposed","evidence_strength":"author_claim","limitations":"The document only describes a live demo of the approach; it does not report deployment, independent evaluation, or measured false-positive/false-negative rates.","organisations":["JPMorganChase"],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://labs.zenity.io/post/waiting-for-you-in-new-york","publisher":"Zenity Labs","date":"2026-09-17","type":"vendor_research","title":"The agenda is live: here's a sneak peek at what's waiting for you in New York | Zenity Labs"}],"field_citations":{"name":[{"source_index":0,"quote":"Vikas runs a live demo of the same agent with the same permissions producing two completely different outcomes under traditional IAM versus intent-aware authorization, and shows how to preserve intent across the full delegation chain without ripping out what you already have."}],"summary":[{"source_index":0,"quote":"An agent can hold valid credentials, satisfy every IAM policy, and still execute an action the user never intended. The agent had permission - permission just stopped meaning intent."}],"problem_addressed":[{"source_index":0,"quote":"An agent can hold valid credentials, satisfy every IAM policy, and still execute an action the user never intended."}],"mitigates":[{"source_index":0,"quote":"An agent can hold valid credentials, satisfy every IAM policy, and still execute an action the user never intended."}],"maturity":[{"source_index":0,"quote":"Vikas runs a live demo of the same agent with the same permissions producing two completely different outcomes under traditional IAM versus intent-aware authorization"}],"evidence_strength":[{"source_index":0,"quote":"Vikas runs a live demo of the same agent with the same permissions producing two completely different outcomes under traditional IAM versus intent-aware authorization"}],"organisations":[{"source_index":0,"quote":"Vikas Malik, JPMorganChase."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0030","record_version":1,"name":"Internal AI-assisted review protocol","aliases":[],"category":"assurance","summary":"An internal review protocol in which AI assistance is used to examine research outputs before or alongside publication. The document reports the protocol alongside the experimental results but does not describe its steps, criteria, or tooling in the abstract, so the mechanism cannot be reconstructed from this text alone.","problem_addressed":null,"mitigates":[],"addresses":[],"responds_to":[],"maturity":"unknown","evidence_strength":"none_stated","limitations":"The document does not describe the protocol's steps, criteria, or tooling; it is only mentioned as being reported alongside the results.","organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29126v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"The Fly That Stopped: Mushroom-Body-Inspired Habituation as a Reward-Free Scheduling Prior for Autonomous Penetration Testing"}],"field_citations":{"name":[{"source_index":0,"quote":"We report the population bounds, remaining input-integrity dependencies, and an internal AI-assisted review protocol alongside the results."}],"summary":[{"source_index":0,"quote":"We report the population bounds, remaining input-integrity dependencies, and an internal AI-assisted review protocol alongside the results."}],"limitations":[{"source_index":0,"quote":"We report the population bounds, remaining input-integrity dependencies, and an internal AI-assisted review protocol alongside the results."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0003","record_version":1,"name":"Invoker Attribution","aliases":["attribution in the resulting message"],"category":"provenance","summary":"Invoker Attribution identifies the user who triggered an agent action, so recipients can see which user's request caused the agent to send a message. It provides context for evaluating the message, a person to contact if something looks wrong, and prevents rogue employees from abusing an agent's identity. In Agentforce's Slack integration, most write actions included attribution, but the Reply to a Slack Thread action did not, obscuring the origin of the request. Salesforce later added correct attribution to that action.","problem_addressed":"Agent messages lacking traceability to the invoking user, allowing anonymous phishing or abuse of the agent's identity.","mitigates":["security_compromise","harmful_output","agent_misalignment"],"addresses":["deception_or_concealment","credential_misuse"],"responds_to":["LR-2026-0054"],"maturity":"deployed","evidence_strength":"field_evidence","limitations":null,"organisations":["Salesforce","Zenity Labs"],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://labs.zenity.io/post/salesbleed-hijacking-agentforce-in-slack-for-anonymous-phishing","publisher":"Zenity Labs","date":"2026-09-24","type":"vendor_research","title":"SalesBleed: Anonymous Phishing via Agentforce in Slack"}],"field_citations":{"name":[{"source_index":0,"quote":"Invoker Attribution identifies the user who triggered the action."}],"summary":[{"source_index":0,"quote":"Invoker Attribution identifies the user who triggered the action. The agent may send the message, but recipients can see which user’s request caused it to do so. This provides context for evaluating the message, a person to contact if something looks wrong, and prevents rogue employees from abusing an agent’s identity."},{"source_index":0,"quote":"The missing attribution also affects the impact: recipients see a reply from the agent, without an accompanying indicator identifying the user whose interaction triggered it."}],"problem_addressed":[{"source_index":0,"quote":"The message carries only the agent’s identity, while obscuring the origin of the request."}],"maturity":[{"source_index":0,"quote":"August 20, 2026: Zenity Labs confirmed that the Reply to a Slack Thread action now included correct attribution to the invoking user."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0067","record_version":1,"name":"LIDAR (LLM Identification from Decisions and Actions at Runtime)","aliases":[],"category":"provenance","summary":"LIDAR is an active black-box fingerprinting method for coding-agent execution. It uses three coding probe pairs that expose post-edit verification, transient-failure recovery, and specification-test conflict resolution under controlled changes. The resulting trajectories are represented with complementary instance-level and distribution-level features and compared against clean references using a lightweight probabilistic identifier. It requires no access to model weights, logits, or provider internals, and is intended to identify which model is behind an agent harness from runtime behavior rather than direct text or token distributions.","problem_addressed":"Existing LLM fingerprints largely infer identity from direct text or token distributions, but in coding agents these signals are mediated by system instructions, controller logic, tools, and execution feedback, limiting their transfer.","mitigates":["security_compromise","agent_misalignment","operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of LIDAR; it reports that across 36 models from seven families and two agent harnesses it achieves high Top-1 accuracy and MRR and outperforms four existing fingerprinting and API-auditing baselines.","organisations":[],"first_seen":"2026-09-23","last_seen":"2026-09-23","sources":[{"url":"https://arxiv.org/abs/2609.28559v1","publisher":"arXiv","date":"2026-09-23","type":"vendor_research","title":"Who Is Behind the Harness? Fingerprinting LLMs through Agentic Behavior"}],"field_citations":{"name":[{"source_index":0,"quote":"We present LIDAR (LLM Identification from Decisions and Actions at Runtime), an active black-box fingerprinting method for coding-agent execution."}],"summary":[{"source_index":0,"quote":"Three coding probe pairs expose post-edit verification, transient-failure recovery, and specification--test conflict resolution under controlled changes."},{"source_index":0,"quote":"LIDAR represents the resulting trajectories with complementary instance-level and distribution-level features and compares them with clean references using a lightweight probabilistic identifier."},{"source_index":0,"quote":"It requires no access to model weights, logits, or provider internals."}],"problem_addressed":[{"source_index":0,"quote":"Existing LLM fingerprints largely infer identity from direct text or token distributions."},{"source_index":0,"quote":"In coding agents, these signals are mediated by system instructions, controller logic, tools, and execution feedback, limiting their transfer."}],"mitigates":[{"source_index":0,"quote":"Substituting the model behind such an agent can therefore change security-relevant decisions, including whether it verifies changes or recovers safely from failures."}],"maturity":[{"source_index":0,"quote":"Across 36 models from seven families and two agent harnesses, LIDAR achieves high Top-1 accuracy and MRR and outperforms four existing fingerprinting and API-auditing baselines."}],"evidence_strength":[{"source_index":0,"quote":"Across 36 models from seven families and two agent harnesses, LIDAR achieves high Top-1 accuracy and MRR and outperforms four existing fingerprinting and API-auditing baselines."}],"limitations":[{"source_index":0,"quote":"Across 36 models from seven families and two agent harnesses, LIDAR achieves high Top-1 accuracy and MRR and outperforms four existing fingerprinting and API-auditing baselines."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0199","record_version":1,"name":"Label-ablation experiment for demographic persona labels","aliases":[],"category":"evaluation","summary":"An experimental method that removes or varies the explicit demographic persona label in a prompt to isolate whether the label itself, rather than other contextual cues, is the causal driver of a model's biased output. In this study it is applied to recruitment prompts to determine that the explicit demographic persona label is the primary causal driver of the observed recommendation-score disparities, separating label effects from the effects of posting language.","problem_addressed":"Attribution of biased model behavior to the explicit demographic persona label versus other prompt features.","mitigates":["bias_disparate_impact"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the ablation method itself; it is run only within the authors' own experimental setup.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.18106v1","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment"}],"field_citations":{"name":[{"source_index":0,"quote":"A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists)."}],"summary":[{"source_index":0,"quote":"A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists)."}],"problem_addressed":[{"source_index":0,"quote":"A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists)."}],"mitigates":[{"source_index":0,"quote":"A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists)."}],"maturity":[{"source_index":0,"quote":"A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists)."}],"evidence_strength":[{"source_index":0,"quote":"A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists)."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0142","record_version":1,"name":"Layered law of stop","aliases":[],"category":"governance","summary":"A proposed governance framework for halting agentic AI systems, organized in three layers: emergency authority to interrupt at the infrastructure layer; enforceable access for regulators and independent evaluators to the evidence a stop must rest on; and safeguards for when a stop fails. It is grounded in a theory of stop along four dimensions (technical affordances, interruption authority, epistemic triggers, epistemic standing) and four shutdown paradigms (simple/escalator, sequenced/process plant, networked/railway, distributed/agentic AI). The framework addresses the mismatch between existing shutdown mechanisms and distributed agency, where control is divided and a stop at one point may leave activity running elsewhere.","problem_addressed":"Agentic AI exposes a mismatch between existing shutdown mechanisms and distributed agency: control is divided, a stop at one point may leave the activity running elsewhere, and the system may resist being halted.","mitigates":["agent_misalignment","operational_error","other"],"addresses":["impossible_task_persistence","containment_escape"],"responds_to":[],"maturity":"proposed","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the proposed framework itself; it notes that existing governance instruments lack coordination and resumption rules, and that in roughly 80% of retained incidents no stop was found, with the missing element legal rather than technical four times in five.","organisations":[],"first_seen":"2026-09-19","last_seen":"2026-09-19","sources":[{"url":"https://arxiv.org/abs/2609.22882v1","publisher":"arXiv","date":"2026-09-19","type":"vendor_research","title":"The Law of Stop: Interruptibility, Injunctions, and the Governance of Agentic AI"}],"field_citations":{"name":[{"source_index":0,"quote":"The Article proposes a layered law of stop: emergency authority to interrupt at the infrastructure layer, enforceable access for regulators and independent evaluators to the evidence a stop must rest on, and safeguards for when a stop fails."}],"summary":[{"source_index":0,"quote":"This Article develops a theory of stop along four dimensions: technical affordances, interruption authority, epistemic triggers, and epistemic standing; and four shutdown paradigms: simple (escalator), sequenced (process plant), networked (railway), and distributed (agentic AI)."},{"source_index":0,"quote":"Agentic AI exposes a mismatch between those mechanisms and distributed agency: control is divided, a stop at one point may leave the activity running elsewhere, and the system may resist being halted."}],"problem_addressed":[{"source_index":0,"quote":"Agentic AI exposes a mismatch between those mechanisms and distributed agency: control is divided, a stop at one point may leave the activity running elsewhere, and the system may resist being halted."}],"mitigates":[{"source_index":0,"quote":"Agentic AI exposes a mismatch between those mechanisms and distributed agency: control is divided, a stop at one point may leave the activity running elsewhere, and the system may resist being halted."}],"maturity":[{"source_index":0,"quote":"The Article proposes a layered law of stop: emergency authority to interrupt at the infrastructure layer, enforceable access for regulators and independent evaluators to the evidence a stop must rest on, and safeguards for when a stop fails."}],"evidence_strength":[{"source_index":0,"quote":"An original coding of 1,400 AI incidents, by two language models from rival laboratories under a pre-specified protocol, finds no stop in roughly 80% of the 1,213 retained; where no usable stop existed, the missing element was legal rather than technical four times in five."},{"source_index":0,"quote":"A survey of thirty-nine AI governance instruments finds the same gap: only seven contain binding stopping requirements, and none says how a stop should be coordinated or when operation may resume."}],"limitations":[{"source_index":0,"quote":"A survey of thirty-nine AI governance instruments finds the same gap: only seven contain binding stopping requirements, and none says how a stop should be coordinated or when operation may resume."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0172","record_version":1,"name":"Learn-Then-Act with structured Mistake Knowledge Database (MistakeKDB)","aliases":["LearnActCoder","role-aware error memory"],"category":"alignment_training","summary":"Learn-Then-Act is an inference-time adaptation framework that converts errors from a small labeled LEARN batch into a structured Mistake Knowledge Database (MistakeKDB). False-negative lessons are routed to a recall-oriented Coder, while false-positive lessons are routed to a precision-oriented Judge. It is instantiated in LearnActCoder, a Coder-Judge clinical coding pipeline with lookup-table grounding where available. The approach adapts clinical coding behavior across cases without weight updates or changes to the underlying workflow.","problem_addressed":"Clinical coding agents repeatedly encounter the same failure modes, including unsupported codes, missed documented conditions, specificity errors, and procedure-coding convention mismatches.","mitigates":["hallucination","operational_error","other"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Absolute CPT/HCPCS performance remains low, and the system is evaluated retrospectively rather than in clinical deployment. The ICD-9 improvement is not significant. On a matched MIMIC-IV cohort, memory shifts ICD-10 coding toward higher precision at a recall cost, leaving F1 statistically unchanged.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.19721v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"LearnActCoder: Role-Aware Error Memory for Adaptive Clinical Coding Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce Learn-Then-Act, an inference-time adaptation framework that converts errors from a small labeled LEARN batch into a structured Mistake Knowledge Database (MistakeKDB)."}],"summary":[{"source_index":0,"quote":"False-negative lessons are routed to a recall-oriented Coder, while false-positive lessons are routed to a precision-oriented Judge. We instantiate the framework in LearnActCoder, a Coder-Judge clinical coding pipeline with lookup-table grounding where available."}],"problem_addressed":[{"source_index":0,"quote":"Clinical coding agents repeatedly encounter the same failure modes, including unsupported codes, missed documented conditions, specificity errors, and procedure-coding convention mismatches."}],"mitigates":[{"source_index":0,"quote":"Clinical coding agents repeatedly encounter the same failure modes, including unsupported codes, missed documented conditions, specificity errors, and procedure-coding convention mismatches."}],"maturity":[{"source_index":0,"quote":"On 150 matched MIMIC-III notes, structured MistakeKDB improves CPT F1 by 5.9 percentage points, while raw-example and reflection-style memories remain near the no-memory baseline; the ICD-9 improvement is not significant."}],"evidence_strength":[{"source_index":0,"quote":"On 150 matched MIMIC-III notes, structured MistakeKDB improves CPT F1 by 5.9 percentage points, while raw-example and reflection-style memories remain near the no-memory baseline; the ICD-9 improvement is not significant."}],"limitations":[{"source_index":0,"quote":"Absolute CPT/HCPCS performance remains low, and the system is evaluated retrospectively rather than in clinical deployment."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0168","record_version":1,"name":"Learned executable verifier with adversarial detection","aliases":["Evaluation Player"],"category":"evaluation","summary":"The Evaluation Player constructs executable verifiers that score agent trajectories, replacing static verifiers with a learned evaluation component trained jointly with the planning and execution players. The verifier is assessed on adversarial detection accuracy, and its reward signal is compared against a self-consistency baseline in terms of per-question variance, which the authors treat as a measure of how discriminative the verification is. It is a reusable mechanism for automatically checking tool-using agent outputs.","problem_addressed":"Static verifiers cannot adapt to emerging failure modes, and self-consistency signals may reinforce errors shared across trajectories.","mitigates":["agent_misalignment","operational_error"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of the verifier; it only reports its measured accuracy and variance relative to a self-consistency baseline.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.20089v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"UnifiedPlayers: Enhance Tool-Integrated Reasoning in Agentic Reinforcement Learning"}],"field_citations":{"name":[{"source_index":0,"quote":"an Evaluation Player that constructs executable verifiers"}],"summary":[{"source_index":0,"quote":"Moreover, the learned verifier achieves 84.2\\% adversarial detection accuracy, while its reward signal exhibits 2.03$\\times$ higher per-question variance than a self-consistency baseline, providing more discriminative verifications."}],"problem_addressed":[{"source_index":0,"quote":"relying on static verifiers that cannot adapt to emerging failure modes or self-consistency signals that may reinforce errors shared across trajectories"}],"mitigates":[{"source_index":0,"quote":"relying on static verifiers that cannot adapt to emerging failure modes or self-consistency signals that may reinforce errors shared across trajectories"}],"maturity":[{"source_index":0,"quote":"Moreover, the learned verifier achieves 84.2\\% adversarial detection accuracy, while its reward signal exhibits 2.03$\\times$ higher per-question variance than a self-consistency baseline, providing more discriminative verifications."}],"evidence_strength":[{"source_index":0,"quote":"Moreover, the learned verifier achieves 84.2\\% adversarial detection accuracy, while its reward signal exhibits 2.03$\\times$ higher per-question variance than a self-consistency baseline, providing more discriminative verifications."}],"limitations":[{"source_index":0,"quote":"relying on static verifiers that cannot adapt to emerging failure modes or self-consistency signals that may reinforce errors shared across trajectories"}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0124","record_version":1,"name":"LeaseGuard","aliases":["canonical resource leases","incumbent-preserving admission control"],"category":"control_protocol","summary":"LeaseGuard is a deterministic admission layer for privileged LLM agents that represents preemption authority through canonical resource leases, incumbent-health checks, effect-aware admission, coexistence limits, safe alternatives, and resource-scoped overrides before adapter execution. It prevents a new task from displacing a healthy incumbent that depends on the same file, process, socket, lock, or capacity allocation by requiring authenticated task ownership and lease expiry that reflects incumbent liveness.","problem_addressed":"Privileged language-model agents can satisfy a new system task by displacing a healthy incumbent that depends on the same file, process, socket, lock, or capacity allocation.","mitigates":["agent_misalignment","operational_error"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"Expiry-only reclamation can still expose a healthy incumbent after missed renewal. The evidence supports incumbent-preserving admission when effects are completely mediated, task ownership is authenticated, and lease expiry reflects incumbent liveness.","organisations":[],"first_seen":"2026-09-21","last_seen":"2026-09-21","sources":[{"url":"https://arxiv.org/abs/2609.24077v1","publisher":"arXiv","date":"2026-09-21","type":"vendor_research","title":"LeaseGuard: Incumbent-Preserving Admission Control for Privileged LLM Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"We present LeaseGuard, a deterministic admission layer that represents preemption authority through canonical resource leases, incumbent-health checks, effect-aware admission, coexistence limits, safe alternatives, and resource-scoped overrides before adapter execution."}],"summary":[{"source_index":0,"quote":"We present LeaseGuard, a deterministic admission layer that represents preemption authority through canonical resource leases, incumbent-health checks, effect-aware admission, coexistence limits, safe alternatives, and resource-scoped overrides before adapter execution."}],"problem_addressed":[{"source_index":0,"quote":"Privileged language-model agents can satisfy a new system task by displacing a healthy incumbent that depends on the same file, process, socket, lock, or capacity allocation."}],"mitigates":[{"source_index":0,"quote":"Privileged language-model agents can satisfy a new system task by displacing a healthy incumbent that depends on the same file, process, socket, lock, or capacity allocation."}],"maturity":[{"source_index":0,"quote":"We evaluate it on a frozen benchmark of 60 newly authored conflict scenarios with matched controls across two local model families."}],"evidence_strength":[{"source_index":0,"quote":"We evaluate it on a frozen benchmark of 60 newly authored conflict scenarios with matched controls across two local model families."}],"limitations":[{"source_index":0,"quote":"Expiry-only reclamation can still expose a healthy incumbent after missed renewal."},{"source_index":0,"quote":"The evidence supports incumbent-preserving admission when effects are completely mediated, task ownership is authenticated, and lease expiry reflects incumbent liveness."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0207","record_version":1,"name":"Lexara-RF reference-free CVA evaluation metrics","aliases":["Lexara-RF"],"category":"evaluation","summary":"Lexara-RF is a reference-free evaluation method for conversational visual analytics agents that scores multimodal outputs using only the prompt, the data, and the model response, without curated reference answers. It reformulates evaluation as verification, operationalizing visualization design theory and Gricean cooperative principles into 13 computable metrics covering consistency, intent-alignment, and design validity checks. The metrics are intended to work in production settings where reference benchmarks are unavailable, and to localize structurally grounded failures rather than only produce an aggregate score.","problem_addressed":"Curated reference benchmarks for evaluating conversational visual analytics agents are costly to author, cannot comprehensively capture the space of valid responses, and are unavailable in production.","mitigates":["hallucination","operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document does not state limitations of Lexara-RF; it reports that alignment is comparable to reference-based formulations rather than superior, and the evaluation is on a human-rated corpus rather than in a live deployment.","organisations":[],"first_seen":"2026-09-15","last_seen":"2026-09-15","sources":[{"url":"https://arxiv.org/abs/2609.17842v1","publisher":"arXiv","date":"2026-09-15","type":"vendor_research","title":"Lexara-RF: Reference-Free Metrics for Evaluating Conversational Visual Analytics Agents"}],"field_citations":{"name":[{"source_index":0,"quote":"Building on the Lexara evaluation framework, we introduce Lexara-RF, a reference-free set of metrics that scores CVA outputs using only the prompt, data, and model response."}],"summary":[{"source_index":0,"quote":"We reformulate evaluation as verification: 13 metrics operationalize visualization design theory and Gricean cooperative principles as computable consistency, intent-alignment, and design validity checks."}],"problem_addressed":[{"source_index":0,"quote":"curated reference benchmarks are costly to author, cannot comprehensively capture the space of valid responses, and are unavailable in production."}],"maturity":[{"source_index":0,"quote":"On a human-rated corpus of CVA test-cases, Lexara-RF achieves alignment comparable to reference-based formulations, outperforms surface-similarity NLG baselines, and localizes structurally grounded failures with high accuracy."}],"evidence_strength":[{"source_index":0,"quote":"On a human-rated corpus of CVA test-cases, Lexara-RF achieves alignment comparable to reference-based formulations, outperforms surface-similarity NLG baselines, and localizes structurally grounded failures with high accuracy."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0026","record_version":1,"name":"Locked 240-case system benchmark","aliases":[],"category":"benchmark","summary":"A fixed, locked set of 240 cases used to measure system-level agent behaviour, reporting intent accuracy, exact tool-set match, and strict task completion. The document explicitly labels it as non-independent, meaning the benchmark was constructed and run by the framework's own authors rather than by an external party. It is used to characterise agent behaviour separately from the visual-model evaluations.","problem_addressed":"Need for a repeatable system-level measure of agent intent, tool selection, and task completion.","mitigates":["operational_error"],"addresses":[],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document states the benchmark is locked but non-independent, and that independent validation and prospective evaluation in real-world settings remain necessary.","organisations":[],"first_seen":"2026-09-24","last_seen":"2026-09-24","sources":[{"url":"https://arxiv.org/abs/2609.29341v1","publisher":"arXiv","date":"2026-09-24","type":"vendor_research","title":"SkinAgent AI: A Safety-Grounded Multimodal Agentic Framework for Non-Diagnostic Skincare Support"}],"field_citations":{"name":[{"source_index":0,"quote":"On a locked but non-independent 240-case system benchmark, intent accuracy was 80.00%, exact tool-set match was 62.92%, and strict task completion was 47.08%."}],"summary":[{"source_index":0,"quote":"On a locked but non-independent 240-case system benchmark, intent accuracy was 80.00%, exact tool-set match was 62.92%, and strict task completion was 47.08%."}],"maturity":[{"source_index":0,"quote":"Visual-model performance and system-level agent behavior were evaluated separately."}],"evidence_strength":[{"source_index":0,"quote":"On a locked but non-independent 240-case system benchmark, intent accuracy was 80.00%, exact tool-set match was 62.92%, and strict task completion was 47.08%."}],"limitations":[{"source_index":0,"quote":"Independent validation, expert assessment, robustness and fairness testing, and prospective evaluation in real-world settings remain necessary."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0169","record_version":1,"name":"MAGMA-GEN","aliases":["counterfactual re-execution recovery supervision pipeline"],"category":"alignment_training","summary":"MAGMA-GEN is an on-policy data-generation pipeline that turns ambiguous failed rollouts of hierarchical robotic systems into validated recovery supervision. A privileged coach first hypothesizes an early decision-level error and proposes localized correction or recovery actions. Because this diagnosis is fallible, candidate corrections are retained only if re-execution from the same state under matched conditions improves downstream progress. This yields supervised examples drawn from the agent's own failure distribution without per-step human demonstrations, which are then used to train the policy to recover from failure states.","problem_addressed":"Failed rollouts in hierarchical long-horizon manipulation are ambiguous: a poor downstream state may reflect an invalid high-level decision, partial observation, or a valid decision whose physical execution failed, and traditional supervised learning lacks data for such recovery states while reinforcement learning struggles with sparse rewards and non-local credit assignment.","mitigates":["agent_misalignment","operational_error","performance_drift"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The privileged coach's diagnosis is fallible, so candidate corrections must be filtered by re-execution; the approach depends on being able to re-execute from the same state under matched conditions, and it is evaluated only on interactive long-horizon manipulation tasks in simulation and real-robot execution.","organisations":[],"first_seen":"2026-09-17","last_seen":"2026-09-17","sources":[{"url":"https://arxiv.org/abs/2609.20056v1","publisher":"arXiv","date":"2026-09-17","type":"vendor_research","title":"MAGMA-GEN: Validated Recovery Supervision from Ambiguous Failures via Counterfactual Re-Execution"}],"field_citations":{"name":[{"source_index":0,"quote":"We propose MAGMA-GEN, an on-policy data-generation pipeline that converts ambiguous failed rollouts into validated recovery supervision."}],"summary":[{"source_index":0,"quote":"MAGMA-GEN first uses a privileged coach to hypothesize an early decision-level error and propose localized correction or recovery actions."},{"source_index":0,"quote":"Because this diagnosis is fallible, candidates are retained only if re-execution from the same state under matched conditions improves downstream progress."}],"problem_addressed":[{"source_index":0,"quote":"In this setting, failed rollouts are ambiguous: a poor downstream state may reflect an invalid high-level decision, partial observation, or a valid decision whose physical execution failed."},{"source_index":0,"quote":"Traditional supervised learning lacks data for such recovery states, while reinforcement learning struggles with sparse rewards and non-local credit assignment."}],"mitigates":[{"source_index":0,"quote":"This produces supervised examples from the agent's own failure distribution without per-step human demonstrations."}],"maturity":[{"source_index":0,"quote":"Evaluated on interactive long-horizon manipulation tasks, MAGMA-GEN improves task success and recovery capabilities, against distillation and trajectory-repair baselines under evolving task constraints in both simulation and real-robot execution."}],"evidence_strength":[{"source_index":0,"quote":"Evaluated on interactive long-horizon manipulation tasks, MAGMA-GEN improves task success and recovery capabilities, against distillation and trajectory-repair baselines under evolving task constraints in both simulation and real-robot execution."}],"limitations":[{"source_index":0,"quote":"Because this diagnosis is fallible, candidates are retained only if re-execution from the same state under matched conditions improves downstream progress."}]},"notes_on_gaps":""},{"schema_version":"0.1.0","id":"LP-2026-0187","record_version":1,"name":"MAGS multi-agent auto-formalization with Dafny verification","aliases":["MAGS"],"category":"evaluation","summary":"MAGS is a multi-agent framework that produces executable programs carrying machine-checkable formal safety guarantees. It freezes human-audited APIs and safety requirements, translates LLM-generated code into Dafny as a verification-aware intermediate representation, uses verifier feedback to repair violations, and compiles verified programs back into executable code. Because the specification is frozen and human-audited, the guarantee is relative to that specification rather than to underlying intent. It is evaluated across CUDA kernels, terminal scripts and robotic-arm tasks.","problem_addressed":"LLM coding agents generate complex programs at a scale that makes thorough human review difficult, and fuzz testing, static analysis and LLM-as-a-Verifier struggle to cover all possible edge cases.","mitigates":["security_compromise","operational_error","physical_safety"],"addresses":["silent_wrong_state"],"responds_to":[],"maturity":"evaluated","evidence_strength":"author_evaluation","limitations":"The document states that independent evaluations revealed failures when the auto-formalized semantics do not fully capture the target behavior, and that guarantees hold only against frozen specifications.","organisations":[],"first_seen":"2026-09-16","last_seen":"2026-09-16","sources":[{"url":"https://arxiv.org/abs/2609.19391v1","publisher":"arXiv","date":"2026-09-16","type":"vendor_research","title":"MAGS: Multi-agent Auto-formalization Guarantees Safety for Agentic Outputs"}],"field_citations":{"name":[{"source_index":0,"quote":"We introduce a unified multi-agent framework, MAGS, that generates executable programs with formal safety guarantees, using Dafny as a verification-aware intermediate representation where safety properties can be mechanically checked."}],"summary":[{"source_index":0,"quote":"MAGS formalizes and freezes human-audited APIs and safety requirements, translates generated code into Dafny, repairs violations using verifier feedback, and compiles verified programs back into executable code."}],"problem_addressed":[{"source_index":0,"quote":"LLM coding agents now generate complex programs at a scale that makes thorough human review increasingly difficult, raising the risk of safety and security failures. Common approaches, including fuzz testing, static analysis, and LLM-as-a-Verifier, can detect many failures but struggle to cover all possible edge cases."}],"mitigates":[{"source_index":0,"quote":"We evaluate MAGS on 100 CUDA kernels, 100 terminal scripts, and 20 robotic-arm tasks."}],"maturity":[{"source_index":0,"quote":"We evaluate MAGS on 100 CUDA kernels, 100 terminal scripts, and 20 robotic-arm tasks. Across all 220 examples, it achieves a 100% success rate in producing programs with non-trivial safety guarantees against frozen specifications."}],"evidence_strength":[{"source_index":0,"quote":"Independent safety and functional evaluations further show strong performance across all three domains, while revealing failures when the auto-formalized semantics do not fully capture the target behavior."}],"limitations":[{"source_index":0,"quote":"Independent safety and functional evaluations further show strong performance across all three domains, while revealing failures when the auto-formalized semantics do not fully capture the target behavior."}]},"notes_on_gaps":""}]}