// SLIDE 01 — HOOK

OCTOBER IS RISKY. BUT IS THE DIAGNOSIS CALENDAR OR CRISIS.

The problem: A disruption model trained on logistics learns that October is dangerous. October is also high-demand season. The correlation is real. But when the model flags October at 80% risk, is it reading genuine operational stress or just the date?
NARRATION

A model trained on 730 days of supply-chain records learns quickly that October is dangerous. October is indeed the month when shipment volumes peak and inventory thins—the correlates are real. But here is the confound: October is also high-demand season naturally, so the feature data rises in October whether the network is under genuine stress or simply doing its seasonal job. When the system raises a disruption flag in October with 80 percent confidence, is it responding to something genuinely happening—delays piling up, suppliers wavering, inventory dropping below safety thresholds—or is it reading the calendar? That is the single most important diagnostic question in supply-chain forecasting. The Supply Chain Collapse Predictor was built to ask it, and to answer it not by eyeballing the correlations but by intervention.

// SLIDE 02 — THE STAKES

WRONG DIAGNOSIS IS EXPENSIVE. CONFUSING SEASONALITY WITH STRESS DESTROYS SIGNAL.

If the model is tracking OReal operational fragility · Early warning · Actionable intervention
If it's just tracking TFalse alarms in October · Ignored in March · Wasted mitigation budget

The Supply Chain Collapse Predictor asks: how can we tell? The answer lies in counterfactual intervention on the model's input.

NARRATION

The cost of confusing seasonality with stress is operational and financial. If the model is truly tracking operational fragility—real suppliers under pressure, real port congestion, real weather impact—then a flag in October is an early warning system. The operations team can intervene: diversify suppliers, buffer inventory, expedite key shipments. That intervention prevents tens of thousands in losses. But if the 80 percent flag is merely the model learning that October is busy, the team has false alarms in October, missed signals in March, and wasted mitigation budget. The difference between a true positive and a seasonal artifact determines whether the model is trusted or ignored. That trust is not earned by accuracy on a holdout test; it is earned by proving the model reads the genuine causes, not the calendar.

// SLIDE 03 — THE CAUSAL GRAPH

FIVE NODES IN TWO LAYERS. ONE HIDDEN CONFOUNDER AT THE DATA LEVEL.

T (Time-of-year)Month, quarter, seasonal calendar; encodes demand cycles
O (Unobserved stress)Supplier fragility, port congestion, weather impact—causes real disruptions
S (Observable signals)Delays, inventory, lead times—the features the model ingests
Y (Realized disruption)True operational breakdown; what we observe in the field
P̂ (Predicted probability)Model output; the flagged risk the system reports

T and O both feed S. The model only sees S. That is the source of the confound.

NARRATION

The causal structure has five nodes. T represents time-of-year: month, quarter, the calendar. O is unobserved operational stress: supplier fragility, port disruptions, weather impact, and other real causes of breakdown. S is the observable signal layer: measured delays, inventory levels, lead times, demand index, transport stress, supplier reliability, and weather risk. These are the features the model ingests. Y is realized disruption—true operational breakdown, what happens in the field. P̂ is the model's predicted probability of disruption. The edges matter: T drives S because October naturally has higher shipments and thinner inventory. O also drives S, because real stress also raises delays and depletes inventory. S determines P̂ because the model maps features to probability. O causes Y because genuine stress causes genuine breakdowns. And T can directly cause Y because seasonal demand can produce disruptions even without underlying fragility. The confound sits at S.

// SLIDE 04 — THE CONFOUND

AT NODE S TWO CAUSES CONVERGE. THE MODEL CANNOT DISTINGUISH THEM.

Seasonality (T→S)October ships more inventory. Lead times stretch. Demand peaks. Features rise.
Operational stress (O→S)A struck port delays shipments. Suppliers wobble. Weather hits. Features rise the same way.

A calm July with baseline shipment rates looks identical to a crisis October after the model normalizes for season—unless the model explicitly encodes T and learns to rely on it.

NARRATION

At node S, two distinct upstream causes push the features in the same direction. A July with stable baseline shipments looks like a July with baseline delays and normal inventory. An October with high seasonal demand looks like an October under real operational stress: both show rising delays, falling inventory, stretched lead times, rising demand indices. From the model's vantage point, they are indistinguishable at the feature level. The model only sees S. It cannot see whether S rose because the calendar drove it up, or because genuine stress drove it up. Both upstream causes point to the same direction in the data. This is confounding: the model learns a strong association between certain feature configurations and disruption, but that association is attached to two conflicting causal pathways, one of which—the calendar—has nothing to do with genuine risk.

// SLIDE 05 — THE INTERVENTION

REMOVE TIME FROM INPUT. REFIT THE MODEL WITHOUT THE CALENDAR.

Original featuresDrop month and quarterRefit modelRe-score same days

The counterfactual is what the model predicts when T is unavailable as an input, but everything else—the rolling averages of delay, inventory, demand index, transport stress, supplier reliability, weather, and lead time—remains intact.

NARRATION

Pearl's do-operator, applied to the model's input, means removing time-of-year information and refitting. The intervention drops month, quarter, and any explicit temporal encodings from the feature set. Every other input is held fixed: the seven-day rolling averages of delay, inventory, lead time, demand index, transport stress, weather risk, supplier reliability, plus the three-day velocity measures. The model is retrained on the same 730 days with the reduced feature set. It is then re-scored on those same days. The readout is the change in predictions: how many high-confidence flags survive, how far they shift, whether the isotonic calibration holds. This is a counterfactual question rather than a confounding-adjustment question. We are not estimating the effect of a treatment on an outcome. We are asking what the model would do under intervention on its own input.

// SLIDE 06 — PREDICTION PERSISTENCE

PREDICTIONS PERSIST OR COLLAPSE AFTER ABLATION. THAT ANSWER TELLS THE DIAGNOSIS.

Predictions persist or shift slightlyModel relied on O · Seasonal noise filtered out · Diagnosis: real stress
Predictions collapse eighty percent to thirtyModel relied on T · Seasonal signal is the diagnosis itself · Diagnosis: false alarm

The persistence or collapse of a prediction under ablation is binary: the model either learned genuine stress or learned the calendar.

NARRATION

If the high-confidence predictions persist after removing T—if days the original model flagged at 80 percent still trigger at 75 or 70 percent—the model was using the remaining features S in a way that does not depend on T being explicit. That means the model learned to read operational stress from the signals that survive ablation. The diagnosis is: real operational fragility. But if the high-confidence predictions collapse—if those 80 percent flags drop to 30 percent or below—the original predictions were carried by T, not by O working through S. The model was reading the calendar, not the crisis. The binary outcome is interpretable. The survival or collapse of a prediction under ablation tells the operator whether the model is trustworthy in that month and context.

// SLIDE 07 — CALIBRATION BEHAVIOR

CALIBRATION SHIFTS UNDER ABLATION. THAT REVEALS THE DIAGNOSIS.

0.80
BEFORE ABLATION
0.30
AFTER ABLATION
COLLAPSE
T WAS THE DRIVER

If the isotonic calibration of the probability estimate depends on T being present in the feature set, the probabilities themselves will drift when T is removed. Magnitude and direction of drift indicate whether the model's confidence was earned or inherited from seasonality.

NARRATION

Calibration is the alignment between predicted probability and observed frequency. An 80 percent prediction should see disruptions 80 times in 100 such cases. The isotonic calibration used in the Collapse Predictor adjusts the raw model scores to match observed frequencies in the training window. When T is removed, the raw scores change. The calibration mapping may not hold. If the predictions drop from 80 to 30 percent and the disruption rate among those dropped-probability days was indeed 30 percent, calibration preserved the honest relationship between probability and outcome—but the probabilities themselves were illusory because they depended on T. If the predictions drop and the disruption rate among them was 70 percent, the model depended heavily on T for calibration but not for ranking. Calibration behavior under ablation is a second diagnostic: it reveals whether T was a crutch for probability estimation or a foundation for causal reasoning.

// SLIDE 08 — THE LIMITATION

THE TEST IS BOUNDED. ROLLING AVERAGES STILL ENCODE TIME.

The limit: Removing T at the input level does not remove T's information entirely. The seven-day rolling average of demand index is itself partly a function of time-of-year. If the model can re-learn seasonality from features that survive ablation, the ablation test understates the dependence on T. This is a feature-level diagnostic, not a representational one.

The test catches one side of the problem: whether the model is using explicit time encodings. It cannot prove the model never learned seasonality from the temporal structure in the other features.

NARRATION

The test is bounded, and that boundary must be named. Removing T at the input level does not remove T's information entirely. The seven-day rolling average of demand index is itself partly a function of time-of-year. High demand in October is different from high demand in March, even if both raise the demand-index feature by the same amount. If the model can re-learn seasonality from features that survive ablation—by learning the temporal structure of the demand indices, or the correlation of inventory drops with certain demand levels—the ablation test understates the dependence on T. This is a feature-level diagnostic, not a representational one. It bounds the problem from one side: if removing explicit time encodings does not matter, we know the model found another way to read seasonality. But if removing explicit time encodings does matter, we cannot conclude the model is free of all seasonality bias.

// SLIDE 09 — RESULTS FRAMEWORK

WHAT DOES THE SYSTEM MEASURE. DRIFT STABILITY AND DISRUPTION INJECTION.

Adversarial validation (AUC)Compares training and current distributions; near 0.5 is indistinguishable, above 0.75 indicates significant drift
Population Stability IndexReported per feature; below 0.1 is stable, above 0.25 is a significant shift between reference and current window
Disruption injectionSynthetic stress events added to roughly 4 percent of days: delays multiply 2.5 to 5 times, inventory drops to 10 to 40 percent, supplier reliability falls to 30 to 60 percent

These measurements form the apparatus for observing what the model learns and how it drifts. The ablation test is wired as a capability, waiting to report which predictions dissolve under removal of month and quarter.

NARRATION

The system measures three things. Adversarial validation uses AUC to compare the training distribution and current distribution: a score near 0.5 means they are indistinguishable; above 0.75 indicates significant drift. Population Stability Index is reported per feature, comparing reference and current windows: below 0.1 is stable, above 0.25 is a significant shift. Disruption injection is synthetic: roughly 4 percent of days have injected risk events. Shipment delays multiply 2.5 to 5 times, inventory drops to 10 to 40 percent of normal, supplier reliability falls to 30 to 60 percent of baseline. These injections create visible operational stress so the model's response can be observed. The counterfactual confound test is described as a capability of the system. The README states that the test exists and that a prediction which dissolves under removal of month and quarter was never real evidence. The load-bearing measurement—the actual delta in predictions before and after ablation on the 730-day record—is where a published case would report the empirical result.

// SLIDE 10 — SYNTHESIS

THE PREDICTION EITHER PERSISTS OR COLLAPSES. EACH OUTCOME IS ACTIONABLE.

Prediction survives ablationTake the flag seriously. The model is reading genuine operational signals. Investigate inventory, delays, supplier status in real time.
Prediction collapses under ablationTreat the flag as a seasonal artifact. Suppress alerts in high-demand seasons unless other concurrent signals confirm stress. Invest in deseasoning features.

Counterfactual ablation allows the operations team to convert a diagnostic question into an actionable split: is this prediction worth trusting, or is it calendar noise.

NARRATION

In production, the ablation test either validates or invalidates a prediction. If a flag in October persists when T is removed, the operations team has good reason to trust it. The model is reading genuine signals: inventory status, delay patterns, supplier health indices. The team can intervene with supplier diversification, inventory buffering, expedited shipment routes. If a flag collapses under ablation, treat it as a seasonal artifact. Suppress October alerts unless other concurrent signals—a struck port, a weather event, a supplier bankruptcy—confirm genuine stress. The binary outcome converts a diagnostic question into actionable guidance. Counterfactual ablation allows the system to tell the operator not just whether a prediction is accurate, but why it is making the prediction, and whether that why is trustworthy in the field.

// SLIDE 11 — THESIS

COUNTERFACTUAL ABLATION DIAGNOSES CONFOUNDS. IT REVEALS WHETHER MODEL READS CALENDAR OR CRISIS.

The central claim is that removing time-of-year information from a trained model reveals whether its high-confidence predictions depend on seasonal patterns or genuine operational stress. This is a feature-level diagnostic—not a proof of representational independence, but a practical tool for deciding when a model's alarms merit intervention in the field.

Causal reasoning applied to model diagnostics asks not what is the effect of seasonality on predictions but what would this model do if the date were not in its input. The counterfactual intervention is on the algorithm, not the world. And that distinction is the discipline the chapter teaches: how to frame questions precisely enough that the answer is actionable, and how to respect the boundary between what you know about a model and what you know about the supply chain.

NARRATION

The central claim is that removing time-of-year information from a trained model reveals whether its high-confidence predictions depend on seasonal patterns or genuine operational stress. This is a feature-level diagnostic—not a proof of representational independence, but a practical tool for deciding when a model's alarms merit intervention in the field. Causal reasoning applied to model diagnostics asks not what is the effect of seasonality on predictions but what would this model do if the date were not in its input. The counterfactual intervention is on the algorithm, not the world. And that distinction—between a causal claim about the system and a claim about the field—is the discipline the chapter teaches: how to frame questions precisely enough that the answer is actionable, and how to respect the boundary between what you know about a model and what you know about the supply chain.

// SLIDE 12 — CLOSE

COUNTERFACTUAL//FEATURE ABLATION//CONFOUNDING

Causal Inference · Chapter 20 · Case: Supply Chain Collapse Predictor

NARRATION

01 / 12
Causal Inference · Ch.20 · Nik Bear Brown