Short Definition
Label noise occurs when training labels are incorrect.
Definition
Label noise arises when some training examples are mislabeled. This can be due to human error, ambiguous cases, or automated labeling processes.
Why It Matters
Neural networks can memorize noisy labels, harming generalization and calibration.
How It Works (Conceptually)
- Incorrect targets distort loss
- Gradients push model in wrong directions
- Overfitting becomes more likely
Minimal Python Example
y_true = [0, 1, 0, 1, 0] # one label is wrongCommon Pitfalls
- Assuming labels are perfect
- Overtraining on noisy data
- Ignoring uncertainty in labels
Related Concepts
- Generalization
- Robustness
- Regularization
- Loss Functions
Neural Network Lexicon 