Outcome Horizon

Short Definition

The outcome horizon is the time delay between a model-driven decision and the point at which its true outcome can be reliably observed.

Definition

The outcome horizon defines how long it takes for the real-world consequences of a model’s prediction or action to materialize and become measurable. It determines when labels are considered mature and when performance can be evaluated truthfully.

The horizon separates prediction from truth.

Why It Matters

Many machine learning systems operate on short feedback cycles while optimizing for long-term outcomes. When the outcome horizon is ignored, models are trained, evaluated, and updated using incomplete or misleading signals, leading to biased learning and false confidence.

Time defines correctness.

Examples of Outcome Horizons

Outcome horizons vary widely by domain:

fraud detection: weeks to months (chargebacks)
credit risk: months to years (default)
recommender systems: days to months (retention)
churn prediction: weeks to quarters
medical diagnosis: months or longer

Short horizons are the exception, not the rule.

Relationship to Labels and Ground Truth

Labels observed before the outcome horizon completes are often:

incomplete
censored
biased toward negative or neutral outcomes

True ground truth exists only after the horizon passes.

Minimal Conceptual Illustration

Decision → (Outcome Horizon) → Observable Outcome → Label

Outcome Horizon vs Label Latency

Label latency refers to delays in data collection or processing
Outcome horizon refers to when the outcome actually occurs

Latency can be reduced; horizons are intrinsic.

Impact on Evaluation

Ignoring the outcome horizon leads to:

inflated short-term metrics
misleading validation results
incorrect model comparisons
premature claims of improvement

Evaluation must align with outcome timing.

Impact on Proxy Metrics

Proxy metrics are often used to approximate long-horizon outcomes. The longer the horizon, the weaker and riskier the proxy tends to be.

Proxy reliability decays with horizon length.

Relationship to Delayed Feedback Loops

Outcome horizon is the structural cause of delayed feedback loops. Systems with long horizons must explicitly account for delay in monitoring, retraining, and governance.

Delay is unavoidable; misinterpretation is optional.

Relationship to Model Update Policies

Update policies must define:

how long to wait before evaluating updates
which data is mature enough for retraining
how proxies are used during the horizon
when recalibration is required

Updates before horizon completion are speculative.

Horizon Mismatch

Horizon mismatch occurs when:

models are optimized on short-term proxies
decisions have long-term consequences
retraining cycles outpace outcome realization

Mismatch creates systemic bias.

Handling Long Outcome Horizons

Common strategies include:

explicit horizon-aware evaluation windows
delayed or staggered retraining
survival or time-to-event modeling
conservative policy updates
periodic long-term audits

Time-aware design is essential.

Common Pitfalls

treating early signals as final outcomes
retraining on immature labels
comparing models with unequal horizons
ignoring censoring effects
assuming faster feedback implies better learning

Speed does not equal truth.

Summary Characteristics

Aspect	Outcome Horizon
Defines	When truth is known
Typical length	Weeks to years
Affects	Evaluation, retraining, governance
Proxy reliance	Increases with horizon
Risk if ignored	Systemic bias

Neural Network Lexicon