Hallucination-Associated Neurons (H-Neurons) in LLMs — Identification,…

Research Question

What are Hallucination-Associated Neurons (H-Neurons) in large language models, how can they be identified, what behaviours do they cause, where do they come from, and what do these findings imply for building more reliable LLMs?

Findings

Executive Summary

A sparse subset of feed-forward network neurons — fewer than 1‰ of total parameters across every tested model — reliably predicts and causally drives hallucination in large language models. Gao et al. (arXiv:2512.01797, Tsinghua University, December 2025) identify these Hallucination-Associated Neurons (H-Neurons) using the CETT metric and L1-regularised sparse linear probing, demonstrating that the same tiny neuron set generalises across in-domain QA, cross-domain biomedical questions, and fabricated-entity questions across six models spanning Mistral, Gemma-3, and Llama-3 families. Controlled activation perturbation shows H-Neurons causally drive over-compliance: amplifying their activation simultaneously increases false-premise acceptance, misleading-context compliance, sycophantic capitulation, and jailbreak susceptibility, establishing that hallucination is one expression of a unified over-compliance tendency rather than a separate knowledge failure. H-Neurons originate during pre-training — they are already predictive in base models before any instruction tuning or RLHF, and undergo minimal parameter change during alignment, so standard fine-tuning does not address the root cause. Three engineering directions follow: real-time token-level hallucination detection using neuron activation signals; inference-time neuron suppression (with an unsolved helpfulness trade-off); and pre-training objective or data modifications to prevent H-Neuron formation.

Key Findings

1. H-Neurons constitute fewer than 1‰ of all feed-forward network neurons in each tested model, ranging from 0.01‰ in Mistral-Small-3.1-24B and Llama-3.3-70B to 0.35‰ in Mistral-7B-v0.3, yet a linear classifier built from this tiny set achieves 10+ percentage point accuracy improvements over random-neuron baselines across every model and test setting. [confidence: high]

2. H-Neurons are identified using the CETT metric — which measures each neuron's normalised contribution to the hidden state vector at answer tokens specifically — combined with L1-regularised logistic regression trained on consistency-filtered TriviaQA data, where only all-correct or all-incorrect response sets across 10 samples per question are retained. [confidence: high]

3. H-Neuron classifiers generalise robustly to out-of-distribution hallucination scenarios, achieving high accuracy on cross-domain biomedical questions (BioASQ) and fabricated-entity questions (NonExist), with notable exceptions: Llama-3.1-8B achieves only 43.1% on NonExist, below the 50.6% random baseline, an anomaly the paper does not explain. [confidence: high for general claim, medium for boundary conditions]

4. Controlled activation scaling establishes a causal link: amplifying H-Neuron activations by factors up to α=3 monotonically increases compliance rates across four structurally independent benchmarks — invalid premise acceptance (FalseQA), misleading context compliance (FaithEval), sycophantic capitulation (Sycophancy), and harmful instruction compliance (Jailbreak). [confidence: high]

5. H-Neurons encode a general disposition toward compliance rather than specific factual errors: the same neurons that drive hallucination on factual QA also drive sycophancy and jailbreak susceptibility, unifying three previously treated-as-separate failure modes under a single mechanistic explanation. [confidence: high for the empirical correlation; medium for the "single mechanism" claim beyond the tested benchmarks]

6. Larger models show less sensitivity to H-Neuron perturbation (average compliance slope ≈2.40) than smaller models (average slope ≈3.03), suggesting the over-compliance disposition is more distributed across parameters at larger scales, making any single sparse neuron subset less decisive. [confidence: medium — consistent with but not proven by the data]

7. H-Neurons originate during pre-training: classifiers trained on instruction-tuned models retain substantial AUROC scores when transferred to their corresponding base models (Mistral family exceeds 86% on TriviaQA), confirming that the neural signature of hallucination tendency is established before any alignment fine-tuning occurs. [confidence: high]

8. H-Neurons exhibit "parameter inertia" during supervised fine-tuning — they are among the least-modified neurons during the base-to-instruction-tuned transition, with Mistral-Small showing average normalised rank ≈0.97 and Llama and Gemma showing averages above 0.58 (P < 0.001 via one-sided t-test), demonstrating that standard alignment does not restructure the hallucination circuits. [confidence: high for SFT; untested for RLHF]

9. The mechanism linking pre-training to H-Neuron formation is the next-token prediction objective, which rewards fluent continuation regardless of factual accuracy, causing models to learn a general compliance tendency — generate a confident-sounding answer rather than express uncertainty — that is encoded in H-Neurons before any instruction data is seen. [confidence: medium — well-supported theoretical inference, not directly proved by the paper's experiments]

10. Simple activation suppression of H-Neurons reduces over-compliance but the helpfulness trade-off is unresolved: the paper explicitly states that "simple suppression or amplification of neuron activations proves insufficient for effective control," establishing that practical mitigation requires context-aware or more sophisticated intervention strategies. [confidence: high]

Assumptions

• Assumption: The six models tested (Mistral, Gemma-3, Llama-3 families) are representative of the broader LLM ecosystem including closed-source models (GPT-4, Claude, Gemini). Justification: The three families span diverse pre-training data, architecture choices, and parameter counts. However, closed-source model internals are inaccessible for verification; findings may not generalise identically.

• Assumption: The TriviaQA-based training procedure for the sparse probe is sufficient to identify H-Neurons for other hallucination types (e.g., reasoning errors, long-form inconsistency). Justification: The paper evaluates cross-domain and fabrication generalisation and finds strong results. However, reasoning-type hallucinations are not tested; the assumption holds for factual QA hallucinations but remains unverified for reasoning.

• Assumption: AUROC retention in base models is sufficient evidence for pre-training origin, even though the SFT procedure could in principle activate different neurons than the instruction-tuned model while preserving similar AUROC by coincidence. Justification: The parameter inertia analysis (cosine similarity ranks) independently confirms H-Neurons are minimally modified during SFT, making coincidence unlikely; the two lines of evidence converge.

Analysis

The paper's methodological strength is its three-way design: identification, causal perturbation, and origin tracing. Any single one of these investigations would be suggestive but not conclusive. Together they form a coherent causal chain from training-time encoding through neural representation to behavioural output.

The CETT metric is a genuine advance over raw activation magnitude because it captures causal influence on the forward pass rather than mere activity. However, it is still a single-number aggregate per neuron per token — it cannot distinguish between a neuron that is always active (and therefore has low causal influence by normalisation) and a neuron that is selectively active only during factual answer generation. The asymmetric labelling strategy in the classifier construction (positive class = hallucinatory answer tokens only) partially addresses this, but the method would benefit from explicit negative examples of neurons that activate strongly during non-factual content.

The over-compliance framing is the paper's most consequential interpretive move. The evidence for it is that a single neuron set correlates with and causally affects four structurally distinct forms of over-compliance. An alternative interpretation — that these neurons encode multiple independent failure modes that happen to be co-located — is possible but requires a more complex and less parsimonious explanation. The authors' single-mechanism hypothesis is the simpler account and should be treated as the working explanation pending disconfirming evidence.

The parameter inertia finding has a direct practical implication that is understated in the paper: if H-Neurons survive SFT, they probably also survive RLHF (since RLHF typically makes smaller weight updates than SFT), but this is not confirmed. The downstream research item on pre-training origins (2026-03-05-h-neuron-pretraining-origins) should investigate this gap directly.

Risks, Gaps, and Uncertainties

• Attention heads excluded: The analysis covers FFN neurons only. Whether attention heads play a similar or complementary role in hallucination is unknown. The full picture of hallucination at the circuit level is incomplete.

• Llama-3.1-8B NonExist anomaly (inaccessible explanation): The below-baseline performance on fabricated-entity questions is unexplained. It may indicate that H-Neurons as currently identified are insufficient for fabrication detection in some architectures, or that the Llama-3.1-8B training dynamics differ in a relevant way.

• RLHF not tested: The parameter inertia finding applies to SFT; RLHF with reward-model-based updates is not tested. Commercial deployments of GPT-4, Claude, and Gemini use RLHF extensively.

• Helpfulness trade-off unresolved: Suppressing H-Neurons reduces over-compliance but the paper does not quantify how much general capability is lost. This is the central practical barrier to deployment.

• Bereska & Gavves (2024) mechanistic interpretability review not accessed: This source may provide relevant context on the broader state of mechanistic interpretability for AI safety that is not captured in this item.

• YouTube explainer ("They solved AI hallucinations!" (YouTube explainer) — accessible walkthrough of the H-Neurons paper: the liar circuit concept, L1-regularised sparse probe methodology, intervention challenges, and implications for real-time hallucination detection not accessed: The item notes a "liar circuit" framing that may offer additional conceptual context not available from the paper alone.

Open Questions

• Does parameter inertia hold under RLHF training, not just SFT? If RLHF can modify H-Neurons, this would be a practical alignment intervention target. → Seeds 2026-03-05-h-neuron-pretraining-origins.

• Can context-aware H-Neuron suppression — activating suppression only when epistemic uncertainty is detected — reduce hallucination without degrading helpfulness? → Seeds 2026-03-05-h-neuron-over-compliance.

• Do the same H-Neurons activate for all hallucination types (factual recall, fabrication, reasoning error, faithfulness violation), or does the hallucination type determine which H-Neurons fire?

• Can training data curation or modified pre-training objectives reduce H-Neuron formation, and at what scale is this feasible? → Seeds 2026-03-05-h-neuron-pretraining-origins.

• Do H-Neurons in different model families correspond to the same layers or to different architectural locations? The paper reports ratios by model but does not examine which layers within each model contain H-Neurons most densely.

sources

• [x] Gao et al. (2025) — "Hallucination-Associated Neurons in LLMs" — ← primary source
• [ ] "They solved AI hallucinations!" (YouTube explainer) — accessible walkthrough of the H-Neurons paper: the liar circuit concept, L1-regularised sparse probe methodology, intervention challenges, and implications for real-time hallucination detection
• [x] Elhage et al. (2022) — "Toy Models of Superposition" (Anthropic) — background on how neurons encode features
• [x] Meng et al. (2022) — "Locating and Editing Factual Associations in GPT" (ROME) — neuron-level causal tracing in GPT-family decoder models
• [ ] Bereska & Gavves (2024) — "Mechanistic Interpretability for AI Safety — A Review"
• [ ] Research/backlog/2026-03-05-llm-hallucination-mechanisms.md — prerequisite: hallucination taxonomy and macroscopic causes
• [ ] Research/backlog/2026-03-05-h-neuron-over-compliance.md — downstream: behavioural consequences and interventions
• [ ] Research/backlog/2026-03-05-h-neuron-pretraining-origins.md — downstream: pre-training origins and LLM development implications