Short Definition
Proxy metrics are indirect measurements used to approximate true objectives or outcomes that are difficult, delayed, or costly to observe.
Definition
A proxy metric is a measurable signal used in place of a true target metric when the true metric is unavailable in real time, delayed, expensive to compute, or difficult to measure directly. Proxy metrics are commonly used in training, evaluation, monitoring, and online experimentation to provide faster feedback.
Proxies stand in for what really matters.
Why It Matters
Many real-world objectives—such as long-term revenue, user satisfaction, safety outcomes, or default risk—cannot be measured immediately. Proxy metrics enable iteration and monitoring, but if poorly chosen, they can misrepresent true performance and drive harmful optimization.
Optimizing the proxy does not guarantee optimizing the goal.
Common Reasons for Using Proxy Metrics
Proxy metrics are used when:
- true outcomes are delayed (label latency)
- business impact is long-term
- labels require human verification
- experimentation needs rapid feedback
- deployment constraints limit observability
Proxies trade fidelity for speed.
Examples of Proxy Metrics
Depending on the domain, proxy metrics may include:
- click-through rate as a proxy for engagement
- short-term conversion as a proxy for lifetime value
- prediction confidence as a proxy for correctness
- early fraud flags as a proxy for confirmed fraud
- surrogate loss as a proxy for business cost
Each proxy encodes assumptions.
Minimal Conceptual Illustration
Decision → Proxy Metric → (time) → True Outcome
Relationship to Offline Metrics vs Business Metrics
Offline metrics are often proxies for business outcomes. The farther a proxy is from the real objective, the greater the risk of misalignment between technical performance and real-world impact.
All offline metrics are proxies by design.
Proxy Metrics and Delayed Feedback
Proxy metrics are frequently used to bridge delayed feedback loops. However, proxies may be biased toward short-term signals and systematically misrepresent long-term outcomes.
Delay amplifies proxy risk.
Risks of Proxy Metrics
Poorly chosen proxies can lead to:
- Goodhart’s Law effects (“when a measure becomes a target…”)
- optimization toward unintended behavior
- overfitting to short-term signals
- neglect of rare but costly failures
- erosion of long-term performance
Proxies shape behavior.
Proxy Drift
Over time, the relationship between a proxy metric and the true objective may change due to:
- distribution shift
- user adaptation
- policy changes
- feedback loops
- model updates
Proxy drift breaks assumed correlations.
Evaluation Implications
When using proxy metrics:
- validate correlation with true outcomes
- monitor divergence over time
- avoid treating proxies as ground truth
- re-evaluate proxies after model updates
- complement proxies with delayed outcome audits
Proxies require continuous validation.
Relationship to Online Evaluation
Online experiments often rely on proxy metrics due to latency constraints. Decisions based solely on proxies must be revisited once true outcomes become available.
Online success may be temporary.
Common Pitfalls
- optimizing proxies without validating alignment
- treating proxy improvements as business success
- ignoring proxy drift
- using single proxies for multi-objective systems
- reporting proxies without uncertainty or caveats
Proxies simplify reality—sometimes too much.
Summary Characteristics
| Aspect | Proxy Metrics |
|---|---|
| Purpose | Approximate true objectives |
| Availability | Immediate or early |
| Fidelity | Lower than true metrics |
| Risk | Misalignment |
| Monitoring need | High |
Neural Network Lexicon 