Pearl's Causal Hierarchy

Pearl's Causal Hierarchy: Formal Information-Theoretic Limits on Deriving Interventional and Counterfactual Reasoning from Observational Data

2026-05-18 · llm-reasoning ai-architecture causal-reasoning formal-epistemology · synthesis medium · source → · wiki →

key claims

Pearl's Causal Hierarchy distinguishes three query classes, association, intervention, and counterfactuals, by the forms `P(y|x)`, `P(y|do(x), z)`, and `P(y_x|x', y')`, and each class corresponds to a different kind of causal information rather than a different notation for the same informationPearl (n.d.)Bareinboim et al. (2022)
The Causal Hierarchy Theorem states that the hierarchy almost never collapses, and formally that the subset of Structural Causal Models in which any collapse occurs has measure zero, so lower-layer data generically fail to determine higher-layer factsBareinboim et al. (2022)
Level 2 never collapses to Level 1, because for any Structural Causal Model there exists another model with the same observational theory but a different intervention theory, which means observational equivalence is never enough to fix causal effects by itselfBareinboim et al. (2022)
Level 3 almost never collapses to Level 2, so even a system that knows intervention distributions still generally lacks enough information to answer unit-level alternate-world questions without additional assumptions or richer model structureBareinboim et al. (2022)
Unobserved confounding can make observational conditioning disagree with intervention effects, including reversing the apparent sign of treatment benefit in Bareinboim et al.'s worked example, which shows that `P(Y|X)` and `P(Y|do(X))` are not interchangeable objectsBareinboim et al. (2022)
Do-calculus is complete for identifiable causal effects, meaning every successful reduction of an intervention query to observational quantities can be derived by Pearl's three rules together with standard probability manipulations, so identifiability depends on structure rather than on ad hoc algebraic tricksValtorta (2006)
Passive machine-learning systems trained only on logged observations are normally confined to Level 1 unless they are given extra structural assumptions, intervention data, or sufficiently rich environment variation, because passive fitting does not itself supply mechanism-replacement semantics or counterfactual world comparisonsPearl (2018)Scholkopf (2017)Valtorta (2006)
Research Question 2.1 becomes the learning-theory corollary of the theorem, because ERM can control observational risk under one distribution without identifying the intervention-sensitive structure needed to remain correct when the environment changesResearch (2026)Bareinboim et al. (2022)

Research Question

What are the formal information-theoretic boundaries that prevent a model trained exclusively on observational data (Level 1 on Pearl's Ladder of Causation) from ever executing or predicting the outcomes of structural interventions (Level 2) or counterfactuals (Level 3)?

Findings

Executive Summary

Observational data alone do not determine intervention or counterfactual answers in general, because Pearl's Causal Hierarchy shows that lower-layer data almost always underdetermine higher-layer facts.

The hierarchy distinguishes association, intervention, and counterfactual reasoning through the probability objects P(y|x), P(y|do(x), z), and P(y_x|x', y'), and those objects require progressively richer structural information in the generic case.

Unobserved confounding is the standard mechanism behind the gap between conditioning and intervention, while Markovian no-confounding models and do-calculus identify the exceptional cases in which that gap can be bridged.

This theorem unifies Research Questions 2.1, 2.2, and 2.3 because ERM's causal blindness, observational underdetermination, and perturbation fragility are all what one should expect from learning that never leaves Level 1.

Key Findings

Pearl's Causal Hierarchy distinguishes three query classes, association, intervention, and counterfactuals, by the forms P(y|x), P(y|do(x), z), and P(y_x|x', y'), and each class corresponds to a different kind of causal information rather than a different notation for the same information.
The Causal Hierarchy Theorem states that the hierarchy almost never collapses, and formally that the subset of Structural Causal Models in which any collapse occurs has measure zero, so lower-layer data generically fail to determine higher-layer facts.
Level 2 never collapses to Level 1, because for any Structural Causal Model there exists another model with the same observational theory but a different intervention theory, which means observational equivalence is never enough to fix causal effects by itself.
Level 3 almost never collapses to Level 2, so even a system that knows intervention distributions still generally lacks enough information to answer unit-level alternate-world questions without additional assumptions or richer model structure.
Unobserved confounding can make observational conditioning disagree with intervention effects, including reversing the apparent sign of treatment benefit in Bareinboim et al.'s worked example, which shows that P(Y|X) and P(Y|do(X)) are not interchangeable objects.
Do-calculus is complete for identifiable causal effects, meaning every successful reduction of an intervention query to observational quantities can be derived by Pearl's three rules together with standard probability manipulations, so identifiability depends on structure rather than on ad hoc algebraic tricks.
Passive machine-learning systems trained only on logged observations are normally confined to Level 1 unless they are given extra structural assumptions, intervention data, or sufficiently rich environment variation, because passive fitting does not itself supply mechanism-replacement semantics or counterfactual world comparisons.
Research Question 2.1 becomes the learning-theory corollary of the theorem, because ERM can control observational risk under one distribution without identifying the intervention-sensitive structure needed to remain correct when the environment changes.
Research Question 2.2 becomes the epistemic corollary of the theorem, because multiple rival mechanisms remain live whenever Level 1 evidence does not identify the higher-layer facts that would otherwise break observational equivalence among those mechanisms.
Research Question 2.3 becomes the deployment corollary of the theorem, because a predictor chosen from Level 1 information alone can look adequate on seen data and still fail once perturbations expose the mechanism that the learner never had enough information to recover.

Assumptions

The passive machine-learning application in this item is about systems trained only on observational traces and evaluated without explicit intervention semantics, not about every possible interactive learning setup.
The prior repository items correctly represent the portions of Phase 2 they summarise here, because this item uses them as completed syntheses rather than re-deriving every subordinate proof from scratch.

Analysis

The strongest part of the case is the negative theorem-level claim, because the hierarchy definitions, the Causal Hierarchy Theorem, the Markovian exception, and the completeness of do-calculus all come from primary technical sources.

The central trade-off is between generic impossibility and structured identifiability: without assumptions the hierarchy does not collapse, but with the right graphical constraints some intervention queries do become observationally identifiable.

A rival interpretation would say that better optimisation, larger models, or more data might dissolve the barrier, but the theorem blocks that move because it is about what lower-layer information determines, not about how efficiently an algorithm uses that information.

The Phase 2 synthesis is therefore coherent: ERM's blind spot, observational underdetermination, and perturbation fragility are not three unrelated failures, but one generic consequence of trying to answer higher-layer questions with lower-layer evidence.

Risks, Gaps, and Uncertainties

The exact size of the practical evidence budget needed to identify a higher-layer query in any specific machine-learning application remains outside this item's scope, because the theorem gives a generic non-collapse result rather than a task-by-task sample-complexity bound.
The machine-learning conclusion is medium confidence rather than high confidence because it combines the theorem with broader learning-theory interpretation, even though Pearl's paper strongly supports the negative direction.
The counterfactual non-collapse claim is strong at the theorem level but thinner at the worked-example level in this item, because the main accessible source is the theorem chapter rather than a separate family of open-access counterfactual case studies.

Open Questions

What minimum combination of interventions, environment changes, or structural assumptions is enough to identify action-relevant structure in current foundation-model systems?
Which benchmark family best distinguishes genuine Level 2 competence from improved Level 1 pattern matching under richer observational coverage?
How should one measure partial progress toward Level 3 counterfactual competence in systems that can answer some intervention queries but still lack stable unit-level counterfactual grounding?