The forty-five-minute elicitation gave you a CPDAG: directed edges where the expert was confident, undirected edges where she was not, and absent edges for non-causal pairs. This is structured, it is grounded, and it is incomplete.
The expert elicitation in Chapter 16 leaves you with a partially directed acyclic graph—a CPDAG. Some edges are oriented: the expert committed on direction. Some edges are undirected: the expert acknowledged causality but would not or could not specify which way the arrow runs. Some pairs have no edge at all, because the expert said there was no direct causal relationship. The CPDAG is the distillation of everything the expert knows, expressed in a form that the causal inference machinery can operate on. But there is a sharp problem: a counterfactual query—what would retention be if we changed the activation flow?—requires a fully directed graph. The undirected edges in the critical path of your analysis are not a minor inconvenience. They are the specific places where your model cannot yet produce an answer. Orienting them is the first task of this chapter.
Without full orientation, you cannot run counterfactuals, rank interventions by effect size, or produce actionable recommendations. A carelessly maintained graph in production is the same as one built carelessly.
Getting from the CPDAG to a fully directed acyclic graph is the work of this chapter. The algorithms do some of the orientation work; the resolution protocol does the rest; and governance ensures the result stays trustworthy over time. Why does full direction matter? Counterfactual queries require it. Intervention ranking requires it. Recommendation systems require it. Practitioners who skip this step or treat the final graph as static often end up with models that were built carefully and maintained carelessly—which, in production, is identical to having built them carelessly. The graph is not a fixed artifact. It is a living one, updated as the domain evolves, as new data arrives, and as expert understanding deepens. This chapter gives you three tools: diagnosis of where disagreements come from, a protocol for resolving them, and a governance framework for keeping the graph trustworthy over time.
The diagnostic question—which source is producing a given undirected edge?—determines whether you invest in more data or more expert sessions.
Undirected edges in a CPDAG come from two distinct sources, and the source determines what can be done about them. The first is Markov equivalence. Two directed graphs are Markov-equivalent if they imply exactly the same conditional independence relationships. Observational data can distinguish graphs that are not equivalent—different structures imply different independencies, and independence tests can tell them apart—but cannot distinguish graphs within the same equivalence class. If two directed graphs are Markov-equivalent, no amount of observational data will ever orient the edge between them. The only resolutions are expert input, someone who knows the mechanism, or experimental intervention, which breaks the equivalence by design. The second source is insufficient data. Independence tests require statistical power. With a small sample or high-dimensional variable space, tests may not have enough power to detect a dependence that exists or distinguish edges whose conditional distributions are nearly identical. Here the edge is undirected not because orientation is structurally unknowable, but because the current data does not have the resolution to determine it. More data—more observations or targeted observations—can resolve this ambiguity. The diagnostic question determines your investment strategy.
When two graphs are Markov-equivalent, they encode identical conditional independencies. Observational data is blind to the difference between them.
Two directed acyclic graphs are Markov-equivalent if they encode the same set of conditional independence relationships among the variables. Because causal discovery algorithms work by testing for conditional independence in the data, they cannot distinguish between Markov-equivalent graphs. No amount of observational data, no matter how large the sample or how many variables you measure, will ever tell you which member of an equivalence class is the true one. This is not a failure of the algorithm or the data. It is a fundamental limit imposed by the structure of the problem. The solution is not methodological—you cannot engineer your way out of this with cleverness or computing power. The solutions are external: bring in expert knowledge about the mechanism, ask someone who has seen the system operate, or run a randomized experiment that actively intervenes on one of the variables. Intervention breaks Markov equivalence by design, because it changes the causal structure. If you randomize a variable, it is no longer subject to the influence of its incoming edges. The resulting graph is no longer equivalent to the original.
The edge exists in the true causal structure; the data just lacks the statistical power to confirm it yet. More observations—targeted at the specific conditional distribution you need to resolve—will restore power and allow correct detection.
The second source of undirected edges is insufficient data. Independence tests are statistical procedures, and statistical procedures require power. In a small sample, conditional independence tests may fail to detect a dependence that is real, not because the test is broken but because there is not enough information in the data to be confident in rejecting the null hypothesis of independence. Similarly, when two conditional distributions are very nearly identical, the tests may not have enough power to distinguish them. In these cases, the edge is undirected not because orientation is impossible or because two competing structures are Markov-equivalent. The edge is undirected because the current dataset does not have the resolution to determine direction. The solution is direct: gather more data. More observations increase the power of the tests. Observations targeted at the specific conditional distribution you are trying to resolve add resolution where you need it most. This is a data-gathering problem, not a causal-learning problem. It is fundamentally solvable with resources. The practical question is whether the cost of gathering that data exceeds the value of resolving the ambiguity.
Each algorithm makes different assumptions and returns a different kind of output. The choice depends on your domain and data situation.
Every causal discovery algorithm does one of two things at its core: it tests for conditional independence relationships in the data and infers structure from them, or it searches over possible graph structures for the one that best fits the data according to some criterion. The first approach is constraint-based; the second is score-based. This chapter focuses on four algorithm families that dominate applied work: PC, which uses constraint-based reasoning to identify edges and leave undirected those whose orientation is fundamentally unresolvable; GES, which greedily searches over directed acyclic graphs to maximize a score function; NOTEARS, which treats the search as a differentiable optimization problem and uses gradient descent; and FCI, which extends constraint-based reasoning to handle latent confounders. Each has different assumptions, different guarantees, and different failure modes. PC leaves you with the most honest picture of equivalence classes—it tells you explicitly which edges cannot be oriented from data. GES and NOTEARS return fully directed graphs, which is useful for applications but can hide uncertainty. FCI is designed for situations where you know some variables are unmeasured. The choice of algorithm is the choice of what assumptions you are willing to make.
Constraint-based algorithms are conservative: they only claim an edge is absent if the data strongly supports it. Undirected edges mean the data could not decide.
Constraint-based algorithms like PC work by testing for conditional independence relationships in the data and using those tests to build the graph. If two variables are conditionally independent given a set of other variables, the algorithm concludes there is no edge between them, or marks the edge as oriented away from that conditioning set. If two variables are not conditionally independent, the algorithm marks an edge between them. The algorithm makes several strong assumptions. It assumes the graph is acyclic—that there are no cycles in the causal structure. It assumes causal sufficiency: that all common causes of the measured variables are themselves measured. It assumes faithfulness: that independence in the data reflects independence in the true graph, not accidental statistical independencies arising by chance. And it assumes the conditional independence tests are correctly calibrated and powered. When these assumptions hold, constraint-based algorithms are honest about uncertainty. Edges are left undirected precisely when the data cannot determine orientation. This honesty is valuable—it tells you where the limits of your data are. When the assumptions are violated, constraint-based algorithms can make serious errors.
GES uses greedy local search; NOTEARS applies gradient descent on a differentiable acyclicity constraint. Both return a fully directed graph, but this advantage comes with a cost: equivalence classes and uncertainty are hidden inside score differences rather than made explicit.
Score-based algorithms take a different approach. They define a scoring function—typically a likelihood-based score like the Bayesian information criterion or a regularized likelihood—and search over the space of possible directed acyclic graphs to find the one with the highest score. GES, the greedy equivalence search, uses local search: it considers small changes to the current graph, moves to the neighbor with the highest score, and repeats until no neighbor is better. NOTEARS, no-tears, reframes the acyclicity constraint as a differentiable penalty and uses gradient descent to search the space. Score-based methods have advantages: they return a fully directed graph, which is immediately usable for interventions and counterfactuals. They can incorporate domain knowledge through priors or constraints. They naturally handle the case where multiple data sources are available. But they have a significant drawback: they hide uncertainty. Two graphs with nearly identical scores may imply very different causal conclusions, but the algorithm returns only one. The uncertainty that a constraint-based algorithm would make visible is buried inside the score gap.
Disagreements are not failures. They are signals that uncertainty remains. The resolution protocol tells you how to address it.
The expert elicitation in Chapter 16 produces a CPDAG. The algorithms in this chapter produce their own graphs. These will sometimes disagree. The expert says edge A→B is present and directed; the algorithm finds no edge. The expert says A and B have no direct connection; the algorithm finds one. These disagreements are not failures of either method. They are signals that uncertainty remains, and they demand investigation. The chapter presents a three-step resolution protocol. First, diagnose: what exactly is the disagreement about? Is it an edge's presence, its direction, or its existence in the undirected edges? Second, classify: what kind of disagreement is this? Can more data resolve it, or does it require expert input? Is it a Markov equivalence issue, where only experiment or expertise can help? Or is it insufficient data, where more observations can answer the question? Third, resolve: take the appropriate action. Gather more data, run more expert sessions, or design a targeted experiment. Some disagreements are aleatory—they reflect genuine randomness that no investigation will resolve. Some are epistemic—they reflect ignorance that you can reduce with effort.
Governance turns the graph from a one-time artifact into an accountable, evolving decision system that can be inspected and corrected.
The validated graph is not a static artifact. It is a living one. It will be updated as new data arrives, as expert understanding deepens, as the domain evolves, and as errors are discovered. Governance is the infrastructure that keeps the graph trustworthy over time. Version control tracks every iteration of the graph, every change to its structure, and the reasoning behind each change. Who made the change? When? What data or expert input drove it? A version-controlled graph is auditable and reversible. Drift monitoring watches for cases where the graph's causal assumptions no longer hold. You deploy the graph, it makes recommendations, and the outcomes diverge from the predictions. That divergence is a signal. It may mean the domain has changed, or that the graph had an error. Re-elicitation triggers establish routines: when domain knowledge changes, when data distributions shift, when drift appears, you return to the experts and run the elicitation protocol again. Auditability means documenting why each edge is present, why it is oriented the way it is, and what assumptions underlie it. A graph whose structure is documented, tracked, and reviewable can be trusted, corrected, and improved over time.
Resolving the graph is not linear. It is an iterative cycle: diagnosis → investment → validation → governance. Each cycle brings the graph closer to the truth and your confidence higher.
This chapter has introduced four families of algorithms, each with its own assumptions and guarantees. It has shown you how to diagnose the sources of undirected edges and disagreements between expert and algorithmic output. The thread that binds them together is the diagnostic question: which source of uncertainty am I looking at? Markov equivalence is a structural fact—a property of the graph space itself. No amount of observational data will change it. If you face Markov equivalence, you invest in expertise and experiment. You run controlled trials. You ask domain experts who have seen the system operate. These are expensive, but they are the only thing that works. Insufficient data is a resource problem. The structure is knowable; the data just does not have the power to see it yet. If you face insufficient data, you invest in data collection. More observations. Targeted sampling. These are also expensive, but often cheaper than running experiments, and they compound over time as your dataset grows. The algorithms do the initial search. The diagnostic question tells you what to do next. Some edges will remain ambiguous, bounded by equivalence classes or by the power of your data. That ambiguity is not a failure. It is an honest statement of what is knowable.
Algorithms reveal what data can tell you; experts reveal what it cannot. The resolution protocol weaves these insights together and governance ensures the result remains auditable over time.
The expert elicitation in Chapter 16 gives you a partially directed graph and the best understanding the expert can muster. The algorithms in this chapter give you a data-driven perspective on what causal structure is consistent with the observations. Neither is complete. Neither is final. The expert's graph may be overconfident in some places, missing data-driven evidence. The algorithm's graph may be missing knowledge that only an expert has. The resolution protocol bridges that gap. It asks: where do they disagree? Can data resolve it, or does it require expertise? Should you gather more data, run more expert sessions, or design an experiment? The path forward depends on the nature of the disagreement. Some edges will remain in equivalence classes, bounded by Markov equivalence—and that is honest, not a limitation. Some will be resolved by more data. Some will require expert judgment or controlled intervention. The final graph is not produced by either method alone. It is the result of iteration: expert input shapes the initial CPDAG; algorithms refine and challenge it; diagnosis points to the next investment; resolution follows. Governance ensures that the graph you deploy today remains auditable, improvable, and trustworthy over time.
Living Models · Chapter 17 · Resolving the Graph