// SLIDE 01 — HOOK

CASE STUDIES RANGE FROM METICULOUS TO SLOPPY.

MeticulousExplicit assumptions · Clear diagrams · Appropriate methods · Sensitivity checks · Acknowledged limits
SloppyLoose claims · Unstated assumptions · Ritually applied methods · Missing limitations
NARRATION

The theory spine ends here. The rest of the book is nine case studies, each applying the methods of preceding chapters to problems graduate students found worth studying. But case studies—published ones as well as student ones—vary enormously in quality. Some are meticulous. The assumptions are explicit, the diagrams are drawn, the analysis is appropriate to the question, the sensitivity checks are performed, the limits are acknowledged. Others are sloppy. Causal claims are made in loose language, assumptions go unstated, methods are applied ritualistically without attention to whether they fit the problem, and the limitations sections are either missing or perfunctory. This chapter is about telling the difference. The goal is not to reject every case study that falls short of perfection—no real analysis is perfect—but to evaluate fairly. A well-executed case study that admits its limits is more trustworthy than a flashy one that hides them.

// SLIDE 02 — THE STAKES

LEARNING TO READ THEM CHANGES YOUR JUDGMENT.

Central challenge: Causal claims are powerful but must be backed by solid analysis. Your ability to read them critically directly determines what you believe.
NARRATION

This chapter is a practical guide, not new methodology. It is about developing structured skepticism—the ability to evaluate case studies fairly without rejecting everything that isn't perfect. You will encounter causal case studies everywhere: in published literature, in this book, in your own future work. The ability to distinguish a well-executed analysis from a sloppy one, to see through loose language and recognize when key steps are skipped, is crucial. By the end of this chapter, you should have a checklist you can apply to any case study you encounter, anywhere, and a sharper eye for what causal language can and cannot legitimately claim. Reading the literature critically requires seeing through terminological games and distinguishing the real work from the marketing.

// SLIDE 03 — DEFINITION

A CAUSAL CASE STUDY IS AN APPLICATION.

Methodological paperDevelops or refines a method
Textbook chapterExplains methods in general
Descriptive studyReports patterns without causal interpretation
Case studyApplies existing methods to new data and commits to causal interpretation
NARRATION

A causal case study is specific. It takes a real-world problem, applies the methods of causal inference to it, and tries to answer a specific causal question about that problem. Unlike a methodological paper, which develops or refines a method, a case study applies existing methods to new data. Unlike a textbook chapter, which explains methods in general, a case study shows methods in action. Unlike a purely descriptive empirical study, which reports patterns in data without trying to explain them causally, a case study commits to causal interpretation. This commitment to causal interpretation is what makes case studies hard. A descriptive study can hide behind correlation. A case study must posit causal structure, argue for its plausibility, apply appropriate methods, and interpret results causally. Each step introduces opportunities for error. A well-constructed case study has a specific shape. It begins with a causal question, proposes a causal model, discusses identification, applies an appropriate method, reports results, conducts sensitivity analyses, and discusses limitations.

// SLIDE 04 — CAUSAL QUESTION

THE FIRST STEP IS THE CAUSAL QUESTION.

Essential requirement: Answerable in one clear sentence. What causal effect? What treatment? What outcome? What population?

A case study that cannot state its question clearly does not know what it is trying to estimate, and the rest will be confused. "We examine the relationship between X and Y" is not a causal question. "We estimate the effect of X on Y in population P" is.

NARRATION

The checklist begins with the most fundamental question: What is the causal effect being estimated? What is the treatment? What is the outcome? What is the target population? This must be answerable in one clear sentence. Why? Because if the case study cannot state its causal question clearly, it does not know what it is trying to estimate, and the rest of the analysis will be correspondingly confused. Watch for slippery formulations. "We examine the relationship between X and Y" sounds like analysis but is not a causal question. "We estimate the effect of X on Y in population P" is. The difference is subtle but critical. The first could mean almost anything. The second is specific enough to be wrong, which makes it worth checking. Clarity about the causal question is the foundation for everything that follows.

// SLIDE 05 — CAUSAL MODEL

THE MODEL ENCODES ALL ASSUMPTIONS.

Real problemCausal model (explicit or implicit)Assumptions made testable

The model shows how X, Y, and other variables relate. It can be a diagram or described in text, but it must exist. A case study that skips the causal model leaves the reader unable to check what the author assumed—a major red flag.

NARRATION

A well-constructed case study proposes a causal model, often explicitly in a diagram, sometimes implicitly in text. This model encodes assumptions about how X, Y, and other relevant variables relate. Why does this matter? Because the model makes the author's assumptions visible and testable. It answers the question: given this problem, what does the author believe the causal structure is? One of the most common shortcuts in poor case studies is skipping the causal model entirely. When you cannot see what the author assumed, you cannot check whether those assumptions are plausible, and you cannot evaluate the rest of the analysis. A case study without an explicit model is doing you—and itself—a disservice. The model does not need to be perfect. But it must exist, and it must be visible to the reader.

// SLIDE 06 — IDENTIFICATION

IDENTIFICATION BRIDGES MODEL TO ESTIMATION.

Causal modelWhat assumptions allow estimation?Available data

A case study must discuss identification: given the causal model, what specific assumptions would allow us to estimate the causal effect from the available data? This step is where theory becomes practice.

NARRATION

Once the causal model is on the table, the case study must discuss identification. The identification argument asks: given this causal model and these assumptions, can we estimate the causal effect from the data we have? This is the critical bridge between theoretical assumptions and practical estimation. Different identification strategies require different assumptions. Adjustment requires ignorability—that there is no unmeasured confounding. Matching requires unconfoundedness and overlap. Instrumental variables require exclusion restrictions—that the instrument affects the outcome only through the treatment. The case study must name these assumptions explicitly and argue for their plausibility in the specific context. Skipping the identification argument is another common shortcut. Without it, the reader cannot tell whether the method being applied actually fits the problem. The method might be technically correct in the abstract, but wrong for this case.

// SLIDE 07 — METHODS & ROBUSTNESS

APPLY METHOD, THEN CHECK ROBUSTNESS.

MethodAdjustment, matching, weighting, instrumental variables, or combination thereof
ResultsReport point estimates and confidence intervals
Sensitivity analysisCheck whether conclusions hold under different reasonable assumptions
LimitationsDiscuss what could go wrong and why the analysis does not claim perfection
NARRATION

A case study applies a method appropriate for the identification strategy and assumptions. This might be adjustment, matching, weighting, instrumental variables, or some combination. It reports the results. But reported results alone are not enough. A well-constructed case study conducts sensitivity analyses. These ask: if I relax the assumption slightly, do my conclusions still hold? If confounding is slightly larger than I think, does the effect still matter? If the instrument is imperfectly excludable, does the sign of the effect change? Sensitivity analysis is not pessimism. It is diligence. It shows the reader how robust your conclusions are to violations of assumptions. A case study that skips sensitivity analysis leaves the reader unable to assess robustness. That is the third common shortcut. Finally, the case study discusses limitations. What could go wrong? What assumptions matter most? What populations does this analysis not cover? A case study that acknowledges limits is more trustworthy than one that hides them.

// SLIDE 08 — RED FLAGS

WATCH FOR THREE COMMON SHORTCUTS.

Skipped elementsNo causal model · No identification argument · No sensitivity analyses · Missing limitations
Signs of rigorExplicit model · Clear identification · Robustness checks · Honest about limits
NARRATION

The most common shortcuts in weak case studies are three. First: skipping the causal model. The author makes causal claims but never lays out the assumed causal structure. You have no way to check what assumptions are being made. Second: skipping the identification argument. The author applies a method but never explains why it is appropriate for this problem. The method might be technically correct in the abstract, but wrong here. Third: skipping sensitivity analyses. The author reports results but does not check whether conclusions are robust to reasonable alternative assumptions. A case study that does all the steps well is doing real work. One that skips one or more is often pretending. When you read a case study, look for these three elements. If any is missing, treat the analysis with skepticism. If all three are present and done thoughtfully, the analysis deserves serious attention even if it is not perfect.

// SLIDE 09 — LANGUAGE PROBLEMS

CAUSAL LANGUAGE IS IMPRECISE.

"Control for" / "Condition on" / "Adjust for"Used interchangeably but technically different
"Causal effect"Floats around without specifying which effect: total, direct, indirect
"Confounding"Means different things in epidemiology, econometrics, and statistics
"Causal AI"Buzzword marketing machine learning as breakthrough when it is implementation of old ideas
NARRATION

Every reader of causal inference will encounter a serious problem: the terminology of the field is a mess. "Control for," "condition on," and "adjust for" are used interchangeably in casual writing, but they have different technical meanings. "Causal effect" floats around without qualifiers specifying which effect—the total effect? The direct effect? The indirect effect? The effect in this subgroup? "Confounding" means different things in different disciplines. Epidemiologists and econometricians and statisticians have inherited different conventions. Most damagingly, a new generation of buzzwords markets old ideas as conceptual breakthroughs. "Causal AI" is the worst offender. It obscures what machine-learning implementations actually do and what the human analyst still has to provide. Reading the literature critically requires seeing through these terminological games. When you see "Causal AI," ask: what is actually new here? Usually, the answer is nothing—the learning model is new, but the causal reasoning is old.

// SLIDE 10 — SYNTHESIS

WELL-CONSTRUCTED STUDIES HAVE CLEAR SHAPE.

1. Causal questionWhat causal effect is being estimated, in what population
2. Causal modelHow do X, Y, and other variables relate
3. IdentificationWhat assumptions allow estimation from available data
4. Method & resultsApply appropriate method, report findings
5. Robustness & limitsCheck sensitivity, discuss what could go wrong
NARRATION

A well-constructed case study has a recognizable shape. It begins with a clear causal question. It proposes a causal model encoding its assumptions. It discusses identification—what would allow estimation from available data. It applies a method appropriate to that identification strategy. It reports results. It conducts sensitivity analyses and discusses limitations. These five steps are not arbitrary. They follow from the logic of causal inference itself. Question before model, because the question determines what model matters. Model before identification, because you cannot say what allows estimation until you know what you are assuming. Identification before method, because the method must fit the identification strategy. Results before sensitivity, because you need to know what you are checking the robustness of. And limitations always, because no analysis is perfect and admitting this is part of being trustworthy. When you read a case study, use this shape as a guide.

// SLIDE 11 — THESIS

FAIR EVALUATION BEATS PERFECTION.

A well-executed case study that admits its limits is more trustworthy than a flashy one that hides them.

The goal is not to reject every case study that falls short of perfection—no real analysis is perfect—but to evaluate fairly, with structured skepticism. You will learn to recognize which studies have done the hard work and which are pretending.

NARRATION

The central argument of this chapter is simple: your job as a reader is not to demand perfection but to evaluate fairly. A well-executed case study that admits its limits—that acknowledges the assumptions being made, shows the sensitivity of conclusions, discusses what could go wrong—is more trustworthy than a flashy one that hides these things. A case study that skips the hard work of laying out its model and identification argument is almost always weak, no matter how impressive the results sound. Reading causal analysis critically means developing structured skepticism. You should be skeptical of claims made in loose language, of methods applied without explanation of why they fit, of results presented without sensitivity checks, of limitations sections that are missing or perfunctory. But you should also recognize good work when you see it. A case study that does all the steps well, even imperfectly, deserves your attention and respect.

// SLIDE 12 — CLOSE

YOUR CRITICAL EYE IS ESSENTIAL.

CASE STUDY//STRUCTURED SKEPTICISM//CAUSAL INFERENCE

Causal Inference · Chapter 9 · How to Read a Causal Case Study

NARRATION

You now have the framework to read any causal case study critically. Start with the checklist: Is there a clear causal question? Is the causal model explicit? Is the identification argument made? Is the method appropriate? Are there sensitivity analyses? Are limitations acknowledged? Use these questions as you read the nine case studies in the rest of this book, and as you encounter causal analysis in the wider literature. The ability to distinguish careful work from careless work, to ask the right questions, and to develop judgment about what you can and cannot believe—that is what this chapter aims to give you. This is the end of the theory spine. Everything that follows applies these ideas.

01 / 12
Causal Inference · Ch.9 · Nik Bear Brown