Neural Network Lexicon

Expected Cost Curves

Short Definition

Expected cost curves visualize the expected prediction cost across decision thresholds or operating conditions.

Definition

Expected cost curves are evaluation tools that plot the expected cost of a classifier as a function of decision thresholds or class prevalence. They incorporate misclassification costs directly into evaluation, showing how total expected cost changes as operating conditions vary.

Unlike accuracy-based curves, expected cost curves explicitly reflect real-world consequences of errors.

Why It Matters

In many applications, minimizing error count is less important than minimizing harm or financial loss. Expected cost curves allow practitioners to:

  • compare models under realistic cost assumptions
  • select decision thresholds that minimize expected cost
  • understand how cost trade-offs change with class prevalence

They provide a practical bridge between model evaluation and decision-making.

How It Works (Conceptually)

  • Define costs for false positives and false negatives
  • Estimate class prevalence or operating conditions
  • Compute expected cost for each threshold
  • Plot expected cost against threshold or prevalence

The optimal operating point is where expected cost is minimized.

Conceptual Formula

Expected Cost =
P(Positive) × [FN Rate × Cost_FN] +
P(Negative) × [FP Rate × Cost_FP]

Minimal Python Example

Python
expected_cost = (
p_positive * fn_rate * cost_fn +
p_negative * fp_rate * cost_fp
)


Common Pitfalls

  • Using unrealistic or unjustified cost values
  • Ignoring uncertainty in prevalence estimates
  • Treating cost curves as static across environments
  • Comparing curves without consistent cost assumptions

Related Concepts

  • Cost-Sensitive Learning
  • Decision Thresholding
  • Precision–Recall Curve
  • ROC Curve
  • Evaluation Metrics