Short Definition
Random search samples hyperparameter configurations randomly from a defined space.
Definition
Random search is a hyperparameter optimization method that evaluates randomly chosen combinations of hyperparameters. Unlike grid search, it does not exhaustively cover the space and instead relies on random sampling.
Random search is often more efficient than grid search, especially when only a few hyperparameters significantly affect performance.
Why It Matters
Many hyperparameters have little impact on performance. Random search allocates more trials to exploring important dimensions and avoids wasted computation.
It often finds good configurations faster than grid search.
How It Works (Conceptually)
- Define probability distributions for hyperparameters
- Sample configurations randomly
- Train and evaluate each sampled configuration
- Track and compare results
Randomness improves exploration efficiency in high-dimensional spaces.
Minimal Python Example
Python
lr = random.uniform(1e-4, 1e-1)batch = random.choice([16, 32, 64])
Common Pitfalls
- Poorly chosen sampling distributions
- Too few trials
- Lack of reproducibility
- Treating randomness as optimization strategy alone
Related Concepts
- Hyperparameter Optimization
- Grid Search
- Bayesian Optimization
- Training Budget
- Experiment Tracking
Neural Network Lexicon 