// SLIDE 01 — HOOK

EIGHTY QUESTIONS IN / FORTY MINUTES.

45 min
LLM ELICITATION
Creative Brief
WITH CAUSAL STRUCTURE

A founder completed a brand strategy interview yielding positioning, archetypes, and strategic insights in forty-five minutes—a task that would normally require three to five hours of consulting across two weeks. The system was engineered as software, not facilitated as conversation.

NARRATION

A founder arrives at what appears to be a routine interview. She faces a chat interface with a direct prompt: describe your company in two or three sentences. She types her answer. The system reflects back, confirms understanding, then proceeds with the next question. Over forty minutes, she answers approximately eighty questions spanning customers, competitors, purchase moments, regret, recommendation language, complaint language. The system refuses to let her speak in marketing abstractions. When she says her brand is authentic, it demands specificity. At the end, the system produces a creative brief containing brand archetypes, positioning statements, and tone-of-voice guidance. Some of it she recognizes—tacit knowledge finally articulated. Some surprises her—connections the systematic questioning surfaced that she had not consciously seen. A skilled human brand strategist would have required three to five hours across two weeks to reach equivalent quality. Nina compressed it to forty-five minutes and eliminated the need for a human strategist in the elicitation step. This architectural move—structured interviews as executable software—transfers directly to causal elicitation in Living Models.

// SLIDE 02 — THE STAKES

THE BOTTLENECK IS NOT THE / ALGORITHMS.

AlgorithmCausal DAGElicitation Gap

Chapter 17's algorithms run fast and are fully automated. But they require a causal DAG as input. The bottleneck at scale is not computation—it is extraction of expert knowledge into the structural form the algorithms need.

NARRATION

The Living Models architecture has solved the algorithmic problem. The inference mechanisms of Chapter 17 are fast, deterministic, and automated. They accept a causal directed acyclic graph and produce decisions, counterfactual reasoning, and uncertainty quantification. But this entire pipeline starts with a structural input: the causal graph itself. And the causal graph must be elicited from human experts. Chapter 14 documented that traditional elicitation engagements take weeks to months. A trained facilitator must work with domain experts to extract variables, edges, and structural constraints. The facilitator is a scarce resource. The experts must be scheduled. The process is expensive, slow, and subject to the cognitive biases Chapter 15 enumerated: availability bias, representativeness, anchoring, collider blindness, and feedback-loop simplification. Until elicitation becomes fast, systematic, and cognitively robust, the Living Model architecture remains a research project. The algorithms do their job perfectly. But the knowledge engineering step precedes them, and that step is where deployment at scale has historically broken down.

// SLIDE 03 — CONCEPT

INTERVIEW IS NOT ART, IT IS / SOFTWARE.

Traditional ModelExpert facilitator interprets answers, steers conversation, translates to structure—a scarce resource taking weeks to months
Nina FrameworkStructured sequence of questions written as executable logic; expert knowledge extracted through disciplined protocol

The Nina framework demonstrates that elicitation is not an art form requiring rare facilitation talent. It is a deterministic process that can be formalized, parameterized, and executed by a system. The architectural pattern transfers directly to causal elicitation.

NARRATION

Traditional causal elicitation relies on a trained facilitator—a person with years of expertise in both probability theory and the substantive domain. This person sits with experts, asks questions from a structured protocol like IDEA or Delphi, interprets answers, translates responses into structural commitments on a diagram, and steers the conversation forward. The entire process depends on the facilitator's judgment, experience, and ability to recognize when an answer requires clarification or when the conversation should pivot. This makes the facilitator a bottleneck. They are scarce, expensive, and the process takes weeks or months. The Nina framework in brand strategy revealed an alternative architecture: the interview itself is a piece of software that can be written. A structured sequence of questions, conditional logic, and consistency checks can be formalized. The system asks, interprets, probes for specificity, prevents abstraction, and builds a structural output without human facilitation. The quality matches human-facilitated work in a fraction of the time. This architectural pattern transfers directly to causal elicitation. The protocol becomes formal. The expert's tacit knowledge remains the input. But the apparatus for extraction—the sequence and logic of questioning—becomes deterministic and scalable.

// SLIDE 04 — CONCEPT

ELICITATION DECOMPOSES INTO / FOUR PHASES.

1. Variables2. Edges3. Equivalence4. Confidence

Each phase has a clear input, a specific mechanism, and a verifiable output. No phase can be skipped. The system executes them in order. The expert moves through them once, and the first-pass DAG emerges.

NARRATION

Causal elicitation is not a monolithic conversation. It is a sequence of four distinct phases, each with its own input, mechanism, and output. Phase one: variable identification. The system asks the expert to identify what decisions, outcomes, and factors matter in their domain. The output is a set of variables that define the decision problem. Phase two: edge elicitation. For each pair of variables, the system asks whether one causally influences the other, using temporal-precedence substitution and post-edge probes to surface the expert's implicit causal reasoning. The output is a complete set of edges. Phase three: equivalence resolution. Because of Markov equivalence, multiple directed acyclic graphs can encode the same conditional independence statements. The system shows the expert which structures are equivalent and asks which representation matches their domain knowledge. The output is a fully oriented DAG. Phase four: confidence calibration. The system asks the expert to express uncertainty about the edges they have identified—which are robust, which are tentative, which rest on assumptions that future data might challenge. The output is a probability distribution over plausible structures. The forty-five-minute benchmark assumes all four phases complete in a single elicitation session.

// SLIDE 05 — CONCEPT

PHASE ONE ASKS: WHAT IS THE / DECISION PROBLEM?

VariablesDecisions, outcomes, factors, or states that matter in the expert's domain
Scope BoundaryWhat is in scope for the model; what is outside; what is given versus what varies
Temporal FramingOver what time horizon does causation occur; when do decisions take effect; when do outcomes manifest

Variable identification is not open-ended brainstorming. The system asks concrete questions about decisions, outcomes, and uncertainty. The expert's answers define the decision problem the DAG will encode.

NARRATION

Phase one of causal elicitation surfaces the decision problem itself. This is not a free-form brainstorm. The system asks concrete, directed questions. What decision or intervention is being made? What are the outcomes that matter? What information is uncertain at the time of decision? What happens after the decision, and how is success measured? The expert's answers define the boundary of the model. Some variables are included because they are decision-relevant. Some are excluded because they are beyond scope or orthogonal to the problem. Some are held constant because they are given; others are left variable because they are uncertain. The system also establishes temporal framing: over what horizon does causation operate? If the decision is a marketing campaign, does the model predict immediate response, week-long effects, seasonal patterns, or long-term brand equity? If the decision is a medical intervention, does causation unfold over minutes, months, or years? Variable identification produces a list of domain-relevant variables and a clear specification of the decision problem they encode. This output becomes the input to phase two.

// SLIDE 06 — CONCEPT

TEMPORAL PRECEDENCE AND TWO / POST-PROBES.

Temporal-Precedence Substitution: Instead of asking 'does X cause Y,' ask 'if X had not occurred before Y, would Y still occur?' This reframes causation as temporal constraint, making implicit causal reasoning explicit.

After identifying temporal precedence, the system deploys two follow-up probes. Post-Probe One: 'What would have to be true for this edge to be false?' Post-Probe Two: 'What evidence would change your mind?' These surface the cognitive failures Chapter 15 documented.

NARRATION

Phase two asks whether one variable causally influences another. But causal reasoning is tacit and often unconscious. The system uses temporal-precedence substitution to make it explicit. Instead of asking directly 'does X cause Y,' the system asks 'if X had not occurred before Y, would Y still occur?' This reframes causation as a temporal constraint. If X must precede Y for causation to operate, then the expert can reason about temporal order as a proxy for causal structure. Once an edge is tentatively identified, the system deploys two post-edge probes. Post-Probe One: 'What would have to be true for this edge to be false?' This surfaces assumptions the expert has not articulated. It exposes reasoning that rests on implicit domain constraints. Post-Probe Two: 'What evidence would change your mind about this edge?' This calibrates confidence and reveals whether the edge rests on robust theory or on recent examples. Chapter 15 documented five systematic cognitive failures: availability bias, representativeness, anchoring, collider blindness, and feedback-loop simplification. These two post-probes systematically surface each category. They are not optional conversational refinements. They are mechanisms that correct for documented cognitive failures.

// SLIDE 07 — CONCEPT

MARKOV EQUIVALENCE MEANS MULTIPLE / DAGS ENCODE IDENTICAL CONDITIONALS.

Directed EdgesImply causal direction: A → B means A comes before B; B depends on A
Equivalence ProblemSome edge orientations produce identical independence structures; expert judgment must resolve which is real

Chapter 7 showed that a CPDAG contains both directed and undirected edges. Undirected edges can be oriented either way without changing conditional independence. Phase three asks the expert which orientation matches the domain causal structure.

NARRATION

Phases one and two produce a complete set of edges between variables. But not all edge orientations are resolved. Due to Markov equivalence, multiple directed acyclic graphs can encode the exact same conditional independence statements. A~B might be oriented as A→B or B→A, and both would satisfy the conditional independence constraints. The completed partially directed acyclic graph or CPDAG contains both directed edges (A→B, settled) and undirected edges (A~B, ambiguous). Chapter 7 established that directed edges imply causal direction and temporal precedence. Undirected edges remain structurally ambiguous. Phase three is equivalence resolution. For each undirected edge, the system asks the expert: which orientation matches your domain knowledge? Does A causally precede B, or vice versa? The expert's answer cannot be determined from conditional independence alone. It requires domain knowledge about temporal order, mechanism, or intervention feasibility. Some edges are empirically resolvable—if data can distinguish them, the expert notes that. Others are theoretically settled—the mechanism dictates direction. Still others remain ambiguous—the expert acknowledges that data must resolve them.

// SLIDE 08 — CONCEPT

FOUR AGENTS EXECUTE THE / ELICITATION PROTOCOL.

InterviewerConducts four-phase elicitation, asks questions, probes for specificity, maintains rigor
ConsistencyChecks that answers cohere; if new edges contradict earlier statements, flags for expert review
EquivalenceHandles phase three; presents equivalent DAGs; asks which matches domain knowledge

A fourth agent, Bias-Watch, monitors all phases for the cognitive failures documented in Chapter 15. The agents run in parallel. No agent performs multiple roles. This separation of concern is essential because consistency-checking and bias-mitigation are fundamentally different tasks.

NARRATION

The multi-agent architecture separates concerns into four distinct roles. The Interviewer agent conducts all four phases of elicitation. It asks questions from the protocol, interprets responses, probes for specificity, prevents abstraction, and maintains logical rigor. The Interviewer is not responsible for checking consistency. The Consistency agent monitors all responses across all phases and flags incoherence. When the expert says X causes Y in minute thirty but implies the opposite in minute forty, the Consistency agent surfaces the contradiction and asks the expert to resolve it. The Consistency agent also checks that the DAG being constructed has no logical inconsistencies: circular dependencies, temporal paradoxes, or impossible conditionals. The Equivalence agent handles phase three. It identifies which edges remain undirected and presents the expert with equivalent DAG structures. It asks which representation matches the expert's domain knowledge. The fourth agent, Bias-Watch, monitors the entire process for systematic biases. It watches for availability bias, representativeness heuristic, anchoring effects, collider blindness, and feedback-loop simplification. These agents cannot be consolidated. Consistency is structural; bias-detection is statistical. Both differ from elicitation logic itself. Running them in parallel catches errors in real-time.

// SLIDE 09 — CONCEPT

COGNITIVE BIASES ARE NOT / INDIVIDUAL FAILURES.

Availability BiasRecent or vivid examples overweight in probability judgment—the agent looks for pattern clustering over time
RepresentativenessPatterns matched to categories without base-rate reasoning—the agent flags when reasoning lacks statistical grounding
Collider BlindnessConditioning on outcomes and inferring spurious causation—the agent flags edges involving outcome variables

The Bias-Watch agent does not tell the expert they are biased. It detects systematic patterns in reasoning that Chapter 15 showed are reliably misleading. It surfaces them for expert review and explicit judgment.

NARRATION

Chapter 15 enumerated five systematic cognitive failures that occur in causal elicitation even among domain experts. Availability bias causes recent or emotionally vivid examples to overweight in probability and causation judgments. Representativeness causes patterns to be matched to categories without base-rate reasoning. Anchoring causes early judgments to persist despite new information. Collider blindness causes experts to infer spurious causation when they condition on outcome variables. Feedback-loop simplification causes dynamic systems to be flattened into linear causal chains. The Bias-Watch agent does not assume the expert is irrationally biased. Rather, it recognizes that these are systematic features of human reasoning under uncertainty. When it detects patterns consistent with availability bias—the expert cites the same three recent examples repeatedly as evidence for multiple causal edges—it flags the pattern for explicit review. When it detects representativeness—the expert matches a current situation to a category without statistical grounding—it asks what base rates would support the analogy. When it detects collider blindness—edges flowing toward or from outcome variables—it surfaces these for explicit judgment. A Bias-Watch flag does not override expert judgment. It asks the expert to make their reasoning explicit and conscious. The expert may confirm the edge after reflection. But the edge is now a deliberate commitment, not an unreflective output of heuristic reasoning.

// SLIDE 10 — SYNTHESIS

THE FORTY-FIVE-MINUTE BENCHMARK / IS CONDITIONAL.

Completes in 45 MinutesModerate scope · Clear variables · Simple structure · Experienced expert
Extends Beyond 45 MinutesLarge scope · Contested variables · Ambiguous edges · Causal cycles · Inexperienced expert

The forty-five-minute benchmark represents what a well-specified protocol can accomplish for problems of bounded complexity. It assumes the expert can articulate causal reasoning. It does not mean all elicitations complete in forty-five minutes.

NARRATION

The forty-five-minute benchmark emerged from the Nina framework applied to brand strategy, where a single interview produces a creative brief in less than an hour. This success suggests structured elicitation might reach causal domains in similar time. But the benchmark is conditional on several assumptions. First: the problem has moderate scope. A model with four to eight variables completes in forty-five minutes. A model with thirty variables requires longer because phase two—edge elicitation—scales quadratically with the number of variables. Second: the variables are clearly defined. If the expert and system spend ten minutes debating what a variable means, the clock runs longer. Third: the causal structure is mostly simple. If the domain contains feedback loops, causal cycles, or bidirectional influences, equivalence resolution becomes harder. Fourth: the expert is experienced and can articulate causal reasoning. A domain expert with years of implicit knowledge can answer questions faster than a novice who must reconstruct reasoning from memory. These conditions define the class of problems the forty-five-minute benchmark addresses. Beyond that class—large-scope problems, contested definitions, complex feedback structures—the elicitation extends. But even then, the four-phase protocol and multi-agent architecture remain valid. Time extends; structure remains fixed.

// SLIDE 11 — THESIS

THE LIVING MODEL BOTTLENECK / BECOMES SOLVABLE.

LLM-guided elicitation compresses the knowledge-engineering bottleneck from weeks to minutes, making causal inference deployable at scale.

When elicitation becomes fast, systematic, and cognitively sound, the Living Model architecture scales from research to production. The algorithms of Chapter 17 are fast and automatic. Chapter 15 identified what breaks in causal reasoning. This chapter specifies the system that prevents it from breaking. Scale follows.

NARRATION

The Living Model architecture depends on a causal DAG as its foundation. The DAG is not learned from data alone—that is the subject of Chapter 17. It is elicited from expert knowledge, combined with data, and used to guide inference. For years, this elicitation step was the deployment bottleneck. Chapter 14 showed that traditional elicitation takes weeks to months. Chapter 15 showed why: systematic cognitive failures undermine expert reasoning. A trained facilitator can mitigate some of these failures through careful questioning, but the facilitator is a scarce resource. This chapter presents an alternative: elicitation as a computable process. Four phases of questioning extract variables, edges, orientations, and confidence. Four agents running in parallel check for consistency, handle equivalence, and surface cognitive biases. The whole process completes in forty-five minutes for typical problems. The expert's tacit knowledge is preserved. The cognitive biases that derail traditional elicitation are identified and made explicit. The DAG emerges clean and ready for Chapter 17's algorithms. This is not a minor optimization. This is the transformation that makes Living Models a production system rather than a research framework. When elicitation is fast, experts can iterate. When it is systematic, errors are caught and corrected. When it is cognitively sound, the knowledge is reliable. The algorithms have been solved. This chapter solves the engineering bottleneck that precedes them.

// SLIDE 12 — CLOSE

ELICITATION//CAUSAL STRUCTURE//MULTI-AGENT DESIGN

Living Models · Chapter 16 · The Machine That Interviews the Expert

NARRATION

01 / 12
Living Models · Ch.16 · Nik Bear Brown