Autonomous knowledge curation and truth maintenance for agentic ontologies

Research Question

What mechanisms exist, or are under active research, to enable Artificial Intelligence (AI) agents to autonomously curate which extracted knowledge is worth retaining in a long-term ontology, detect and resolve contradictions when new knowledge conflicts with existing ontological facts (truth maintenance), and manage ontology noise from imperfect or ambiguous sensory inputs, without requiring continuous human supervision?

Findings

(Populated from §6 Synthesis above.)

Executive Summary

No production system autonomously runs a formal, classical-style Truth Maintenance System (TMS) at the scale of a Large Language Model (LLM)-integrated Knowledge Graph (KG); instead, current autonomous curation is achieved through narrower, purpose-built pipeline stages, explicit conflict detection followed by LLM-assisted resolution, dual-memory routing for edited facts, that substitute for, rather than implement, dependency-directed justification tracking. Benchmarked directly against the Alchourron-Gardenfors-Makinson (AGM) rationality standard that classical TMS theory targets, current LLMs systematically violate minimal-change and stability postulates during iterated belief revision, producing measurable belief inertia and collateral retraction of unrelated facts. Of the five curation sub-problems in this item's scope, conflict detection and resolution have the most developed measured evidence base, since a detect-then-resolve architecture shows double-digit percentage gains in recall and F1-score over resolution-only baselines. Retention policy (what to keep) and noise robustness against adversarially manipulated input are the least mature sub-problems: no source consulted in this investigation describes a standalone, evaluated retention policy independent of the extraction step, and no dedicated adversarial-robustness benchmark for autonomous ontology curation was located. Provenance schemas exist and are being extended for agentic workflows, but this item finds no evidence that they have been evaluated end-to-end as the backbone of an autonomous retraction mechanism.

Key Findings

1. Classical Justification-based Truth Maintenance System (JTMS) and Assumption-based Truth Maintenance System (ATMS) theory defines minimal-retraction dependency tracking and multi-context belief labelling, but no source in this investigation describes a production LLM-KG agent implementing this mechanism directly. ([inference]; medium confidence; source: doi.org de Kleer (1986) An Assumption-Based TMS Chen et al. KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment
2. Frontier LLMs evaluated against the six AGM belief-revision postulates satisfy Success and Consistency but systematically violate Inclusion and Preservation, producing belief inertia and collateral damage under iterated revision.
3. The Belief-R evaluation separately found that LLMs fail to suppress conclusions that should have been retracted after new evidence and, in other cases, over-update when no revision was warranted.
4. A detect-then-resolve architecture that filters candidate conflicts before invoking an LLM for resolution improved recall by 56.4% and F1-score by 68.2% over resolution-only baselines on knowledge graph conflict-resolution benchmarks.
5. Knowledge conflicts in LLM-based systems fall into three distinct types, context-memory, inter-context, and intra-memory, each requiring a different detection and resolution approach rather than one unified conflict-handling routine.
6. A dual-memory routing architecture that shards edited facts into a separate "side memory" from original model parameters reduces interference between old and new knowledge across thousands of sequential edits, but the underlying reliability-generalisation-locality trade-off is not fully resolved.
7. Temporal facts introduce a distinct noise source beyond ordinary extraction error: models trained on static snapshots exhibit "averaging" and "forgetting" failure modes on time-sensitive facts, meaning a curation policy that treats every contradiction as a binary true/false conflict will mishandle facts that are simply superseded by time.
8. Multi-agent knowledge graph enrichment pipelines assign conflict resolution to a dedicated agent within a nine-agent sequence, but no accessible description of a standalone, independently evaluated retention policy (what to keep versus discard) was found separate from the extraction and conflict-resolution stages.
9. No dedicated benchmark for adversarial robustness of autonomous ontology curation, as distinct from ordinary ambiguity or extraction noise, was located in this investigation.
10. The World Wide Web Consortium (W3C) PROV Ontology (PROV-O) and its agent-specific extension, PROV-AGENT, supply a standard schema for tracking why a fact was committed to a knowledge graph, but neither source describes a system that uses this schema as the operational backbone of autonomous contradiction resolution.
11. Knowledge-editing surveys and empirical studies independently converge on the same structural limitation, that edits intended to be local to one fact frequently disrupt logically related facts or fail to propagate consistently, corroborating the reliability-generalisation-locality trade-off from three separate research angles.

Assumptions

KARMA's retention decisions are governed by confidence thresholds and schema-alignment success rather than a separately codified retention policy. This is justified because the accessible description of KARMA's nine-agent pipeline names discovery, extraction, alignment, and conflict-resolution stages without naming a distinct retention-policy module, though the full paper text beyond the abstract was not directly consulted, so the absence could reflect incomplete access rather than an actual design gap.

No dedicated benchmark evaluates adversarial robustness of autonomous ontology curation as distinct from ordinary extraction ambiguity. This is justified because the EMNLP 2024 knowledge-conflicts survey, which is the most comprehensive taxonomy source consulted, categorises conflicts by their origin (context-memory, inter-context, intra-memory) without a category for deliberately adversarial or poisoned input, suggesting the taxonomy as currently constructed does not treat adversarial robustness as a first-class dimension.

Analysis

The strongest, most corroborated finding in this investigation is the gap between formal belief-revision theory and measured LLM behaviour: AGM-Bench and Belief-R independently measure the same class of failure (inability to perform minimal, stable belief revision), and the WISE, Yao et al., Onoe et al., and Jang et al. sources independently describe the same structural trade-off from the model-editing side. A plausible rival explanation for the detect-then-resolve pattern's success is that it works around the LLM's poor native belief revision by never asking the LLM to revise a belief unassisted; the detection stage narrows the input to cases the LLM's prompt-based resolution step can handle reliably, rather than solving the underlying minimal-change problem the AGM postulates describe. This reframes the field's apparent progress: measured gains in conflict-resolution accuracy do not indicate that LLM-KG agents have solved truth maintenance in the classical sense, only that engineered pipelines can compensate for the LLM's documented belief-revision weaknesses in the narrower cases those pipelines are designed to catch. Retention policy and adversarial robustness remain comparatively unaddressed because the reviewed literature is overwhelmingly organised around conflict detection and resolution once a candidate fact is already proposed, leaving the earlier decision of whether to admit a candidate fact at all, and the security-adjacent question of whether that candidate was adversarially crafted, without dedicated evaluation frameworks in the sources consulted.

Risks, Gaps, and Uncertainties

• The KARMA retention-policy claim rests on an abstract-level description rather than the full paper text, so the actual presence or absence of a codified retention module inside KARMA remains unconfirmed.
• A comprehensive search for adversarial-robustness benchmarks specific to autonomous ontology curation was not exhaustive within this item's time budget; a targeted follow-up search restricted to security and robustness venues (rather than natural language processing and knowledge graph venues) could surface relevant work not found here.
• Two seed sources in this item's original ## Sources list resolved to unrelated papers when their arXiv identifiers were checked (Dhingra et al. 2022 and Hase et al. 2023). Both have been corrected to verified identifiers in this item's Sources section, and this session's own verification failure rate on seed sources supports treating any inherited source list as unverified until each URL is independently checked.
• Several candidate sources found during search (TruthKeeper, NeuSymMS, SymAgent published only on academia.edu or Zenodo preprint servers) were excluded from Findings because they are self-published and not peer-reviewed. [assumption; justification: peer review status is used as the primary credibility filter for inclusion, consistent with this item's citation-discipline preference for primary and peer-reviewed sources; source: www.mdpi.com The excluded sources describe a "living, dependency-aware Truth Maintenance System (TMS) for Large Language Model (LLM) agents" concept that may exist in some form but is not independently corroborated by any peer-reviewed or primary source this investigation could verify.
• Doyle (1979) is paywalled at its Digital Object Identifier (DOI) landing page; the claims attributed to it in this item were verified against a publicly archived MIT AI Memo copy of the same paper rather than the ScienceDirect version.

Open Questions

• Has any peer-reviewed system evaluated a codified, standalone retention policy for autonomous ontology curation, independent of the extraction and conflict-resolution steps that precede it?
• Would applying PROV-O/PROV-AGENT-style provenance tracking as the operational backbone of a curation agent measurably reduce belief inertia or collateral damage compared to the detect-then-resolve pattern observed in CRDL?
• What adversarial-robustness benchmark, if any, is most appropriate for autonomous ontology curation, and does one need to be constructed given the apparent gap identified here?

sources

• [x] Doyle (1979) A Truth Maintenance System: foundational Justification-based Truth Maintenance System (JTMS) paper; the classical approach to belief revision that modern neural-symbolic systems build on. Full text is paywalled on ScienceDirect; consulted via the archived MIT AI Memo version and secondary summaries.
• [x] Doyle (1979) A Truth Maintenance System, MIT AI Memo 521a: publicly accessible archival copy of the same paper, used to verify the justification-based, dependency-directed-backtracking mechanism
• [x] de Kleer (1986) An Assumption-Based TMS: foundational Assumption-based Truth Maintenance System (ATMS) paper; introduces environments and labels for tracking multiple, possibly inconsistent, belief contexts in parallel
• [x] Pan et al. (2024) Unifying Large Language Models and Knowledge Graphs: A Roadmap: includes survey of KG curation and conflict management in LLM-integrated systems
• [x] Onoe et al. (2023) Can Language Models Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge: empirical study of knowledge injection and conflict propagation in LLMs
• [x] Dhingra et al. (2022) Time-Aware Language Models as Temporal Knowledge Bases: temporal conflict detection: how to handle facts that were true at time T but have since been superseded. Source correction: the item's seed URL (arxiv.org/abs/2106.15112) resolved to an unrelated cloud-computing paper; the correct arXiv identifier is 2106.15110, cross-checked against the Transactions of the Association for Computational Linguistics (TACL) publication record.
• [x] Hase et al. (2023) Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models: belief representation and update mechanisms in LLMs relevant to truth maintenance. Source correction: the item's seed URL (arxiv.org/abs/2109.14812) resolved to an unrelated blockchain/Internet of Things (IoT) paper; the correct venue record is the 2023 European Chapter of the Association for Computational Linguistics (EACL) proceedings page above.
• [x] Yao et al. (2023) Editing Large Language Models: Problems, Methods, and Opportunities: survey of knowledge editing methods and their side-effects, directly relevant to ontology fact retraction
• [x] Shi et al. (2024) WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models: lifelong knowledge editing with explicit conflict management for evolving facts
• [x] Jang et al. (2022) Towards Continual Knowledge Learning of Language Models: continual learning approach to adding new knowledge while preserving and correcting existing knowledge
• [x] Xu et al. (2024) Knowledge Conflicts for LLMs: A Survey: taxonomy of context-memory, inter-context, and intra-memory conflicts in LLM-based systems, presented at the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)
• [x] Wilie et al. (2024) Belief Revision: The Adaptability of Large Language Models Reasoning: evaluates LLM belief updating against Alchourron-Gardenfors-Makinson (AGM) rationality postulates using the Belief-R dataset
• [x] AGM-Bench: Do Large Language Models Revise Beliefs Rationally?: benchmark operationalising six AGM postulates plus Darwiche-Pearl iterated-revision postulates across 2,400 synthetic scenarios; submitted to the International Conference on Learning Representations (ICLR) 2026
• [x] Chen et al. KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment: nine-agent pipeline with a dedicated conflict-resolution agent for autonomous KG enrichment from unstructured text
• [x] Detect-Then-Resolve: Enhancing Knowledge Graph Conflict Resolution with Large Language Model: CRDL framework separating explicit conflict detection from LLM-based truth resolution, published in MDPI Mathematics (2024)
• [x] W3C PROV-O: The PROV Ontology: World Wide Web Consortium (W3C) standard for representing provenance as Entity-Activity-Agent relationships in Resource Description Framework (RDF) graphs; the schema underlying justification-chain and retraction tracking
• [x] PROV-AGENT: Unified Provenance for Tracking AI Agent Interactions in Agentic Workflows: extension of PROV-O to capture agent prompts, tool invocations, and delegation chains for auditable agentic memory