Short Definition
Model capacity measures how complex a function a model can represent.
Definition
Model capacity reflects the flexibility of a neural network to fit data. It is influenced by the number of parameters, layers, and neurons. Higher capacity enables modeling complex relationships but increases overfitting risk.
Choosing appropriate capacity depends on dataset size and task complexity.
Why It Matters
Both underpowered and overpowered models fail to generalize well.
How It Works (Conceptually)
- More parameters → higher capacity
- More layers → deeper representations
- Capacity must match data complexity
Minimal Python Example
capacity = num_layers * neurons_per_layerCommon Pitfalls
- Overbuilding models
- Ignoring dataset size
- Confusing capacity with performance
Related Concepts
- Bias–Variance Tradeoff
- Overfitting
- Architecture Design
Neural Network Lexicon 