Short Definition
Learning curves show how model performance changes as training progresses.
Definition
Learning curves plot training and validation metrics (such as loss or accuracy) as a function of training time, number of epochs, or amount of data. They provide a visual summary of how a model learns and whether it is improving, stagnating, or degrading over time.
Learning curves are one of the most important diagnostic tools in machine learning.
Why It Matters
Learning curves help diagnose:
- underfitting
- overfitting
- insufficient data
- ineffective optimization
They allow practitioners to understand why a model behaves as it does, not just how well it performs.
How It Works (Conceptually)
- Training metrics reflect how well the model fits seen data
- Validation metrics reflect generalization to unseen data
- The gap and trends between curves reveal learning behavior
Different curve shapes correspond to different training regimes.
Minimal Python Example
Python
plot(epoch, train_loss, val_loss)
Common Pitfalls
- Looking only at final values
- Overreacting to short-term noise
- Ignoring validation curves
- Comparing curves across inconsistent setups
Related Concepts
- Training Dynamics
- Training Monitoring
- Overfitting
- Underfitting
- Generalization
Neural Network Lexicon 