Cross-effects among three-zone agent interventions

Research Question

Under what conditions do the measured benefits of pool-level annotations, harness context management, and goal-attainment verification hold, shrink, or reverse as the consumer model, task family, content-pool properties, and verifier model family vary, and which properties predict those changes?

Findings

(Populated from §6 Synthesis above.)

Executive Summary

No study located varies more than one zone at a time, so the three gains cannot be certified as independent: the best-supported statement is that each gain is conditional on the consumer model, task family, pool noise, and verifier-harness dependence, and that independence should be assumed only where each condition is separately met. Pool-level source and time metadata changes consumer answers in a model-dependent way, and a credibility signal improved conflict resolution on one fact-checking dataset. The observation-masking advantage over summarisation is measured only on coding tasks, and structure-dependent pool gains have one low-confidence report of reversal on a noisy corpus. Verifier accuracy depends on at least three dependence channels with the harness, family self-preference, cross-provider error correlation, and shared context, and the evidence supports treating independence as a graded property. The primary uncertainty is that every cross-zone condition stated here is an inference from single-zone studies and needs a factorial experiment.

Key Findings

1. Consumer models differ in how retrieved-document metadata changes their answers, since swapping publication time changed the answer for most models tested while changing the source mostly did not and appearance strongly affected one model family.
2. Adding source-credibility background information at retrieval and generation stages significantly improved conflict resolution on a fact-checking dataset, which is direct evidence for a pool-level credibility annotation benefit in that single task family.
3. Models can favour generated over retrieved context because of similarity to the question and segmentation damage, so a provenance annotation is not guaranteed to override a consumer's similarity-driven preference.
4. No study located serialises Provenance Ontology, Resource Description Framework-star, or minimal-justification overlays into consumer prompts and measures conflict resolution, so the proposed overlay effect on the context-collision categories remains an untested hypothesis.
5. The measured masking advantage over summarisation comes from coding tasks, and the one non-coding result located compares context folding with summarisation, so the masking advantage on non-coding tasks is untested.
6. Context-rot curves are model-specific and recall degrades non-uniformly with input length, so a mask depth tuned on one consumer model is not expected to transfer to another.
7. An ontology-guided extraction gain of 3.17 percentage points on a clean benchmark is paired with a low-confidence report that a fixed-ontology pipeline lost 21.0 percentage points on a noisy corpus against 10.5 for a schema-free pipeline, and pool noise is a candidate predictor of reversal that remains a hypothesis.
8. Language-model judges favour their own outputs and recognise them with non-trivial accuracy, reported independently by two groups on single-family judging tasks, and extension to agent verifiers is untested.
9. Across more than 350 models, errors are substantially correlated, with 60 percent agreement when both err on one leaderboard, and larger models have correlated errors even across distinct providers, so a different model family reduces but does not remove shared-error exposure.
10. Shared context is a dependence channel separate from model family, because a same-class verifier inspects the same contaminated context and a co-located monitor can be compromised by the same injection, with the monitor-compromise rate unquantified.
11. Prior work on conflicting evidence reports that earlier studies treated ambiguity, misinformation, and noise one at a time, and no factorial study of the three zone interventions was located, so independence cannot be certified from existing evidence.

Assumptions

Metadata sensitivity measured on webpage and fact-checking tasks is assumed to be informative for agent content pools. The justification is that both involve a consumer model choosing between conflicting retrieved claims with metadata present.

The coding-task masking result is assumed to be a usable baseline for non-coding harnesses. The justification is that the context-folding study spans deep-research tasks and reports context-size effects of the same kind, although it tests a different comparison.

Analysis

The strongest design point is that the three zones share objects, and so an intervention in one zone alters the inputs of another. The table below states each gain's independence conditions as inferences so a future experiment can falsify each.

The noisy-corpus ontology reversal is a pool-property condition on a gain, and it does not refute the clean-benchmark result. Leaderboard rows where richer agents beat minimal ones confound harness context management with other differences, so they do not contradict the controlled masking result. Model-specific context-rot curves explain why a masking window may not transfer between consumers. The co-located monitor case is a shared-context condition on the verifier gain, and its size remains unquantified.

The three-zone convergence evidence and the leaderboard rows are selected cases, so they cannot establish how often each condition holds. A change in pool content is a rival cause for any apparent harness or verifier effect, because corpus edits alone change behaviour with weights and prompts fixed. Adversarial collaboration research states that correlated perspectives protect less than their count implies, which supports grading verifier independence rather than counting verifiers. Judge-based pipeline checkpoints carry documented bias modes, so verifier placement does not by itself restore independence.

Risks, Gaps, and Uncertainties

• The Alchourron-Gardenfors-Makinson (AGM) postulate-violation figures from the context-collision item rest on an unreviewed submission and are not used as a quantitative anchor here.
• The only noisy-corpus ontology comparison located is non-peer-reviewed, single-author, and small scale, so the reversal is low confidence.
• Searches for overlay-on-consumer tests, non-coding masking comparisons, and agent-verifier factorial studies returned nothing relevant, so the corresponding conclusions rest on absence of located evidence.
• The agent-memory item found no standard benchmark scoring provenance fidelity at read time, so annotation quality has no agreed metric.
• Resolver ownership between pool and consumer is deferred to issue #653 and affects which component applies annotations.

Open Questions

• Does a masked or summarised harness window preserve per-statement annotations, and at what depth do annotations drop out for each consumer?
• What is the monitor-compromise rate when verifier and harness share external data sources?
• Does a factorial experiment crossing annotation, masking versus summarisation, and verifier family show interaction terms?

sources

Starting points from issue #669 and its cited work, followed by external sources added during investigation.

• [ ] GitHub issue #669: Cross-effects among three-zone agent interventions: question, scope, evidence standard, and counter-cases.
• [ ] Semantic clarity and conflict resolvability in mediated content pools: proposed pool-level provenance and justification annotations.
• [ ] Context collision and relational blindness in flat-vector retrieval-augmented generation: conflict categories and reported conflict-label intervention.
• [ ] Closed-loop performance techniques for mediated execution harnesses: observation masking and summarisation comparison.
• [ ] Independent verification and durable evidence for goal attainment: reliability rubric and graded independence.
• [ ] Zone-specific interface constraints in three-zone architectures: interfaces, selection-artefact caveat, and resolver deferral.
• [ ] Evaluation frameworks for agentic memory quality, relevance, and retrieval accuracy: provenance-fidelity evaluation gap.
• [ ] Adversarial input propagation through multi-step tool-using systems: verifier and strategy-selection error propagation.
• [ ] Adversarial agents with shared goals: multi-perspective coverage: evidence on agent independence and shared perspectives.
• [ ] Large Language Model-as-judge pipeline validation checkpoints: verifier placement and pipeline validation.
• [ ] Policy ambiguity and feedback-loop homogenisation risk: model-family alignment risk.
• [ ] Failure mode taxonomy: empirical frequency and causal mechanisms: co-located monitor case identified as unquantified.
• [ ] Schema-led versus instance-emergent ontology approaches in graph-based retrieval: ontology gain and noisy-corpus reversal.
• [ ] Context engineering: first principles of steering large language model output: model-specific context-rot curves.
• [ ] Behavioral change after retrieval-augmented generation source-document edits: corpus edits changing behaviour with weights and prompts fixed.
• [ ] Minimal coding-agent benchmark results across model families: leaderboard counter-case to minimal-harness superiority.
• [ ] Nuance collapse in deterministic neuro-symbolic ontology pipelines: ambiguity lost through normalisation.
• [x] Chiang and Lee (2024) Do Metadata and Appearance of the Retrieved Webpages Affect LLM's Reasoning in Retrieval-Augmented Generation?: publication time, source, and appearance metadata change consumer answers by model.
• [x] Ge et al. (2025) Resolving Conflicting Evidence in Automated Fact-Checking: source-credibility information in retrieval and generation improves conflict resolution.
• [x] Tan et al. (2024) Blinded by Generated Contexts: consumer bias toward generated contexts when knowledge conflicts.
• [x] Wang et al. (2025) Retrieval-Augmented Generation with Conflicting Evidence: joint ambiguity, misinformation, and noise dataset; prior work studied factors in isolation.
• [x] Lindenbauer et al. (2025) The Complexity Trap: masking versus summarisation on coding tasks.
• [x] Sun et al. (2025) Scaling Long-Horizon LLM Agent via Context-Folding: context management on deep-research and software tasks.
• [x] Panickssery et al. (2024) LLM Evaluators Recognize and Favor Their Own Generations: self-recognition tracks self-preference.
• [x] Wataoka et al. (2024) Self-Preference Bias in LLM-as-a-Judge: self-preference metric and familiarity hypothesis.
• [x] Kim et al. (2025) Correlated Errors in Large Language Models: error correlation across more than 350 models.
• [x] Zheng et al. (2023) Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena: documented judge bias modes.
• [x] Chroma Research (2025) Context Rot: model-specific degradation with input length.
• [x] Terminal-Bench 2.0 leaderboard: leaderboard rows used as counter-case.
• [x] graphrag-comparison repository: noisy-corpus fixed-ontology comparison.
• [x] Alchourron, Gardenfors and Makinson (1985) On the Logic of Theory Change: primary source for the AGM postulates.
• [x] Lebo et al. (2013) PROV-O: The PROV Ontology: provenance vocabulary.
• [x] NIST SP 800-53 Revision 5: separation-of-duties control reference.
• [x] Goguen and Meseguer (1982) Security Policies and Security Models: non-interference and independence reference.
• [x] arXiv 2506.08500: consumer-side conflict handling reference.
• [x] arXiv 1009.3391: referenced background.
• [x] arXiv 2310.01798: referenced background on self-correction.
• [x] Socio-technical theory (Newcastle): behavioural lens reference.