Three engineers, three meanings, one word. The problem is not that any of them is wrong—it is that the word is too small to carry the information that matters.
When you encounter a claim that a system is real-time, you are hearing a word that means entirely different things depending on who is saying it and what they are responsible for. The high-frequency trader means microseconds—the speed of light through fiber optic cable is a design constraint. The retail analyst means the dashboard refreshes more often than it used to—perhaps four hours instead of twenty-four. The loan officer means the model scores applications immediately upon receipt rather than batching them overnight. These are not different degrees of the same thing. They are different things entirely. Each engineer is right about what she or he means. But the word real-time has been asked to do the work of three separate, independent specifications, and it fails at all three by collapsing them into one. As a practitioner, your job is to decompose that claim. The word is a placeholder. Behind it are three independent numbers—three latencies that operate independently, and any one of which can be your actual bottleneck.
When executives are told a system is real-time, they form a mental model of currency and recency. If that model is wrong—if they think they have live data when they have batched data from six hours ago—the cost is not aesthetic. It is decisions made against the world as it was, not the world as it is.
The stakes of this distinction are practical and immediate. A decision-maker who believes a system is real-time will ask different questions of it and trust its answers more readily. They will approve faster loans. They will place larger orders. They will act with more confidence. But their confidence is founded on a word that may mean nothing at all. If the system's data is stale, its model is out of date, or its organizational processes are slow, then the decision-maker is operating under a misconception. The system appears to be current but is not. The label says real-time but the label is a lie—not because anyone intended to deceive, but because the word has been asked to cover territory it cannot cover. The practical harm is not that executives get slower decisions. It is that they act faster than they should. They believe they can move at the speed the word promises, but the system cannot deliver at that speed. Understanding how to decompose that claim is not an academic exercise. It is the difference between decision-makers who have an accurate mental model of their system's currency and those who do not.
Data latency is the elapsed time between an event happening in the world—a transaction, a click, a sensor reading—and that data becoming available to the system for use in a decision. Every step in that pipeline adds delay.
Data latency measures the time between an event occurring in the world and the information describing that event reaching the system. An event happens—a customer swipes a credit card, someone visits a website, a temperature sensor reports a reading. That event generates data. The data travels through collection systems, ingestion pipelines, transformation and cleaning steps, and joins against other data sources. At the end of that journey, the data lands somewhere the system can access it. Each step in that process introduces delay. The collection system may batch events. The ingestion pipeline may buffer data. Cleaning may require lookups against external sources. Joins may require waiting for late-arriving data. Deduplication and validation add more cycles. By the time all of this is complete, the event that triggered the data generation may be hours or days old. The system may report that it is real-time, but if the data it operates on is from hours ago, the system is not real-time. It is real-time at reporting, not at knowing. Data latency is the first independent specification: how old is the information the system is actually using?
A model scores applications at the speed of light, but if that model was trained six months ago, the latency is six months. Currency of the model is independent of speed of scoring.
Model latency is the time between the world changing and the model being retrained to reflect that change. A machine learning system might score loan applications in milliseconds. That speed is genuine. But if the model powering that score was trained six months ago, the actual latency of the system is six months. Model latency is independent of scoring latency. The model may execute at microsecond speeds while remaining months or years out of date. Economic conditions shift. Customer behavior changes. Populations drift. A model trained on 2019 data has accumulated half a decade of epistemic debt by 2024. The engineer deploying the model may not mention this. The system may report that it is real-time, and the scoring itself is real-time, but the model is stale. This is the second independent specification: how current is the model? When was it last retrained? Under what conditions was it trained, and have those conditions changed? Has the model's calibration on current applicants been measured? A high-speed scoring system running a stale model is a fast way to make slow decisions.
Even with fresh data and a current model, organizational and epistemic processes move slowly. The third latency is often the longest because it is human, institutional, and cultural—not technical.
Decision latency is the time between the system producing a recommendation and a human decision-maker acting on it. This is the layer where the system is no longer a system—where humans, organizations, and institutions take over. Even if data arrives at the speed of light and the model is perfectly current, a human decision-maker still needs to see the output, understand it, consider context, consult with colleagues, and choose a course of action. That process takes time. A loan officer may receive a score in milliseconds, but approving a loan application may take hours or days because the officer needs to review supporting documentation, consult with supervisors, or escalate to a committee. A manufacturer may receive an equipment alert in real-time, but responding to that alert requires a technician to walk to the machine, diagnose the problem, and authorize a repair or shutdown. A trader may receive a signal in microseconds, but executing a trade may be delayed by regulatory requirements, risk checks, or organizational approval. This layer is often the longest, and it is the least technical. It is where psychology, bureaucracy, and human judgment dominate. A system can be real-time at every technical layer and still be glacial in its actual decision-making.
You can have millisecond data with a six-month-old model. You can have a fresh model with stale data. You can have both with glacially slow human decision-making. Each layer is its own specification.
The critical insight is that data latency, model latency, and decision latency are independent. You can have one short while the others are long. You can have fast data arriving constantly while operating a model trained years ago. You can have a model that is retrained daily while the data feeding it is buffered in nightly batches. You can have both data and model that are current, but the decision-making process can still be slow because humans are slow. Or you can optimize one layer and ignore the others. A trading firm might have microsecond data latency and negligible decision latency because orders are placed algorithmically, but if the model was trained on historical data from a different market regime, the model latency is enormous. A bank might have excellent decision latency—decisions are made in seconds—but if the data is from six hours ago and the model is from six months ago, the system is not real-time by any meaningful measure. The fact that the three latencies are independent is both the problem and the solution. It is the problem because a claim of real-time could mean anything. It is the solution because once you know all three numbers, you know exactly what the system can do and what it cannot.
Just as water flows through a pipe at the rate of the narrowest segment, a decision system operates at the speed of its slowest latency component. All three must be short enough for the system to be genuinely real-time.
Think of the three latencies as segments of a pipeline. Water flows through a pipe at the rate of the narrowest segment, no matter how wide the other segments are. Decision systems work the same way. The effective latency of the system is determined by whichever component is slowest. Fast data flowing into a stale model is still stale. Fresh data and a current model flowing through a glacially slow decision process is still glacial. You cannot speed up the system by optimizing just one component. If data latency is one hundred milliseconds but model latency is six months, the effective latency is six months. If model latency is one day but decision latency is two weeks, the effective latency is two weeks. Until all three are short enough relative to the decision being made, the system is not real-time. Practitioners often focus on the most visible layer—usually data latency or scoring speed—and ignore the others. They brag about the pipeline that processes millions of events per second while running a model that has not been retrained in years. They brag about a model that is retrained daily while the decisions based on its outputs sit in a queue waiting for human approval. The bottleneck is whichever component moves slowest.
The same word—real-time—is being used to mean microseconds, hours, and months. Each interpretation is rational within its context, but the contexts are incomparable.
Real-time at a high-frequency trading firm means something completely different than real-time at a retail analytics company or at a bank evaluating loan applications. The HFT engineer means latency measured in microseconds—the time between a price tick arriving at an exchange and a quote being generated and dispatched is measured in millionths of a second. The location of the firm's servers relative to the exchange's matching engine is a design constraint. The speed of light through copper and fiber optic cable are fundamental parameters. This is real-time in the strictest engineering sense. The retail engineer means the dashboard refreshes every four hours rather than every twenty-four. Compared to the quarterly reports the company was previously running, this is a genuine improvement. But the data on the dashboard might be from this morning or last night. The label does not specify. This is real-time as a marketing comparison to the prior state, not a technical specification. The loan officer means the model scores applications as they come in rather than batching them overnight. The application is scored in seconds. She does not mention that the model was trained six months ago, or that economic conditions have shifted since then, or that the model's performance on current applicants is unknown. This is real-time in the sense of scoring speed, not in the sense of model currency. Three meanings, one word.
The same update frequency can be adequate for one decision and catastrophic for another. What matters is not absolute speed—it is speed relative to how fast the world is changing for that particular decision.
Calibration is the final piece of the puzzle: adequacy is always relative to the rate of world change for a specific decision. A model retrained monthly might be adequate for forecasting product demand, where the underlying market changes slowly. The same monthly retrain would be catastrophic for fraud detection, where fraud patterns shift daily. A dashboard updated every four hours might be adequate for strategic inventory planning, where decisions play out over days or weeks. The same four-hour update would be disastrous for emergency room triage, where decisions must be made in minutes. The question is not whether a system is real-time in absolute terms. The question is whether the system is real-time relative to the decision you are asking it to support. A credit card company might track balances with nightly batch updates—data is twenty-four hours old—and that is adequate because fraud decisions are made at the point of transaction, using real-time signal processing on the transaction itself. A manufacturing plant tracking equipment health with sensor readings that are only current to the last hourly batch might miss a failure developing over minutes. The adequacy is not an intrinsic property of the system. It is a relationship between the system's latency and the world's rate of change. Understanding that relationship is how you deploy a system that works.
These layers are not metaphors. They are three separate audits that must each be completed before you can answer the question: is this system actually real-time?
The three latencies are distinct, but they are also part of a larger system that has three separable layers. The technical layer includes all the infrastructure: data pipelines, ingestion systems, transformation and cleaning, joins and aggregations. The organizational layer includes model development, training, deployment, retraining cadence, and the decision-making processes that humans use to act on the system's outputs. The epistemic layer includes the relationship between the model and the world it is supposed to describe—how well the model performs under current conditions, whether the fundamental assumptions of the model still hold, whether the world has shifted in ways the model cannot see. Many failures look technical but are organizational. A system might have world-class data infrastructure but a model that is retrained once per year and never revalidated. A slow pipeline might not be the bottleneck at all—slow decision-making is. Conversely, many failures look organizational but are epistemic. A model might be retrained weekly, and the decision process might be fast, but the underlying world might have shifted so fundamentally that the model is useless. An auditing protocol for a real-time claim must touch all three layers. Technical audit alone is insufficient. Organizational audit alone is insufficient. You need all three to know what the system can actually tell you.
Each latency is determined by different constraints and operates at different timescales. The slowest one determines the system's effective speed. Until you have measured all three, you do not know what the system can actually tell you or how quickly you can act on it. Decision-makers who understand this distinction make better choices. Systems designed with this distinction in mind are more trustworthy.
The central argument of this chapter is that real-time is not a single specification. It is three independent specifications that must all be met for a system to be genuinely current. Data latency is the time from event to available data. Model latency is the time from world change to model update. Decision latency is the time from system output to human action. Each is determined by different forces, operates at a different timescale, and has different tradeoffs. A system can be very fast in one and very slow in another. The slowest layer is always the bottleneck. Most claims of real-time collapse these three independent measures into a single word that means nothing. The cost is not academic. It is executives who believe they can move faster than their systems actually allow. It is loan officers who trust stale models. It is manufacturers who think their systems are warning them of hazards in real-time when the warnings are already obsolete. The decomposition into three latencies is not difficult, but it requires discipline. It requires asking specific questions about each layer and demanding specific answers. It requires measuring, not assuming. Once you have those three numbers, the word real-time becomes meaningful.
Living Models · Chapter 3 · What We Mean When We Say "Real-Time"
Three latencies. One word doing three jobs. Decompose the claim, measure all three, and you will know what your system can actually do.