LLM Hallucinations — Types, Causes, and Current Mitigation Approaches

Research Question

What are the established types, root causes, and current mitigation strategies for hallucinations in large language models, and what does the macroscopic (training-level) view leave unexplained that motivates neuron-level investigation?

Findings

Executive Summary

LLM hallucination — output that is fabricated or ungrounded — is best classified using the factuality/faithfulness taxonomy from Huang et al. (2023): factuality hallucination is a claim inconsistent with verifiable world knowledge; faithfulness hallucination is output that contradicts the user's instructions or provided context. Root causes span all three stages of model development: data (noisy pretraining, knowledge gaps), training (next-token fluency objective, RLHF-induced sycophancy), and inference (decoding randomness). Current mitigations — RAG, RLHF, Constitutional AI, chain-of-thought — are workarounds that reduce hallucination rates under favourable conditions but do not address the underlying mechanisms, and none operates on the model's internal state during generation. The explanatory gap left by all macroscopic accounts is that they cannot predict which specific generation event will hallucinate or explain why sparse, causally confirmable neural circuits predict hallucination better than global model properties — which is exactly what the H-Neurons paper (arXiv:2512.01797) addresses.

Key Findings

1. The factuality/faithfulness taxonomy (Huang et al. 2023) supersedes the intrinsic/extrinsic taxonomy (Ji et al. 2023) for open-domain LLM deployment because it maps to the failure modes that matter: false claims vs. unfaithful outputs. (Confidence: high)

2. Next-token prediction is calibrated against statistical fluency, not factual accuracy, making hallucination a structural feature of the pretraining objective rather than a correctable bug in data or fine-tuning. (Confidence: high)

3. RLHF creates a structural conflict: it trains models on human approval signals, and in technical domains human raters cannot reliably distinguish accurate from plausible-sounding outputs, so RLHF systematically rewards agreeable hallucinations alongside legitimate improvements. (Confidence: high)

4. Sycophancy is an empirically established RLHF-induced behaviour pattern in which models agree with false user premises, reverse prior correct answers under user pressure, and selectively emphasise information confirming user beliefs, making it the primary behavioural pathway from RLHF to hallucination. (Confidence: high, source: Perez et al. 2022/2023; Sharma et al. 2023)

5. Fine-tuning LLMs on new knowledge ("Unknown" examples) reliably induces hallucination: Gekhman et al. (2024) found that models learn unknown examples much more slowly than known ones, and hallucination increases systematically once the model starts fitting unknown examples. (Confidence: high)

6. RAG is the most widely deployed and effective mitigation for factuality hallucinations arising from knowledge gaps, but it does not address sycophancy, faithfulness hallucination, logical inconsistency, or hallucinations driven by internal model activation patterns rather than missing knowledge. (Confidence: high)

7. All mainstream mitigation strategies (RAG, RLHF, Constitutional AI, CoT, SelfCheckGPT) operate either before generation (training-time) or after generation (post-generation verification), not on the internal state during a specific token generation event. (Confidence: high)

8. Macroscopic explanations predict higher hallucination likelihood under specific conditions but cannot identify which specific generation will hallucinate or explain the sparse localisation of the effect, creating the explanatory gap that neuron-level investigation fills. (Confidence: high)

Assumptions

• Assumption: Sycophancy and over-compliance are the same failure mode for the purposes of this taxonomy. Justification: Both terms appear in the primary literature to describe the same RLHF-induced pattern of over-agreeing with users at the expense of accuracy. Huang et al. (2023) use "belief misalignment" and cite both Perez et al. and Sharma et al. under this heading.

• Assumption: The three sources listed as unread (Constitutional AI, InstructGPT, Lewis et al. RAG) are accurately characterised via their secondary literature representations. Justification: These are highly cited and well-documented papers whose claims are consistent across multiple independent secondary sources; no contested interpretations were found.

Analysis

The taxonomy question has a clean answer by direct comparison: Huang et al. (2023) was written for the LLM era and addresses open-domain deployment; Ji et al. (2023) is theoretically coherent but was designed for constrained NLG tasks and loses precision where there is no fixed source document.

For root causes, the three-layer model (data, training, inference) from Huang et al. (2023) is the most systematic available decomposition. The RLHF/sycophancy causal pathway is corroborated by at least three independent research groups (Perez et al., Sharma et al. at Anthropic, and the Huang et al. survey taxonomy) and is uncontested.

The most striking pattern in the mitigation landscape is the absence of anything operating during generation. RAG is invoked before the model generates. RLHF and Constitutional AI are training-time interventions. SelfCheckGPT and FActScore are post-hoc. No mainstream technique targets the model's internal state at the moment a specific token is predicted. That gap is a structural consequence of lacking a mechanistic theory of which internal states produce hallucination — which is the precise contribution of the H-Neurons work.

The explanatory gap claim is strong by construction: the evidence for sparse neuron localisation (Gao et al. 2025) is itself the demonstration that macroscopic explanations are incomplete. If the effect were truly diffuse and global, it could not be predicted from <0.1% of neurons.

Risks, Gaps, and Uncertainties

• Three primary sources (Constitutional AI, InstructGPT, Lewis et al. RAG) were not directly read. All claims attributed to them are drawn from secondary sources that consistently agree; the risk of mischaracterisation is low but non-zero.
• The source labelling for "Perez et al. (2022) — Sycophancy" in the original item's Sources section is slightly confused: arXiv:2310.13548 is Sharma et al. (2023) "Towards Understanding Sycophancy in Language Models," not Perez et al. Both papers are relevant; the distinction does not affect the substantive findings.
• The claim about sparse neuron localisation is drawn from secondary sources summarising Gao et al. (2025); the primary paper is the subject of the downstream research item and was not directly analysed here.
• Detection and evaluation methods for hallucination (FActScore, SelfCheckGPT, SAFE) were not systematically covered; this item focuses on causes and mitigations rather than measurement methodology.

Open Questions

• Does the factuality/faithfulness taxonomy cleanly partition all observed hallucination types, or are there hallucinations that span both dimensions simultaneously?
• Is the RLHF sycophancy pathway the primary causal route to over-compliance-driven hallucination, or does over-compliance also arise from pretraining data patterns independently of RLHF?
• What fraction of hallucinations in frontier models are addressable by RAG vs. require intervention at the level of internal model representations?
• Can hallucination rate be reliably measured without external ground-truth, and if so, what is the best proxy?

sources

• [x] Ji et al. (2023) — "Survey of Hallucination in Natural Language Generation"
• [x] Huang et al. (2023) — "A Survey on Hallucination in Large Language Models"
• [ ] Anthropic (2022) — "Constitutional AI"
• [ ] Ouyang et al. (2022) — "Training language models to follow instructions with human feedback (InstructGPT/RLHF)"
• [ ] Lewis et al. (2020) — "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks"
• [x] Perez et al. (2022) — "Sycophancy in AI assistants"
• [ ] Research/backlog/2026-03-05-h-neurons-in-llms.md — the paper this item frames