Neural Network Lexicon

Rolling Window Sampling

Short Definition

Rolling window sampling trains and evaluates models using a fixed-size, moving time window.

Definition

Rolling window sampling is a time-aware data selection strategy in which a model is trained on the most recent fixed-duration slice of data and evaluated on the immediately following period. As time advances, the window “rolls” forward, discarding older data and incorporating newer observations.

This approach prioritizes recency while preserving temporal causality.

Why It Matters

In non-stationary environments, older data can become misleading due to concept drift or changing user behavior. Rolling window sampling limits the influence of stale data and aligns training with the most current distribution.

It is especially useful when the data-generating process evolves over time.

How Rolling Window Sampling Works

A typical workflow:

  1. Choose a window size (e.g., last 30 days)
  2. Train the model on data within the window
  3. Evaluate on the next time slice
  4. Advance the window forward
  5. Repeat across time

Window size controls the bias–variance trade-off.

Minimal Conceptual Example

# conceptual rolling window
train = data[(time > T - window) & (time <= T)] 
test = data[(time > T) & (time <= T + horizon)]

Rolling Window vs Expanding Window

  • Rolling window: fixed size, discards old data
  • Expanding window: grows over time, retains all history

Rolling windows adapt faster; expanding windows are more stable.

Choosing the Window Size

Key considerations include:

  • rate of concept drift
  • seasonality and cycles
  • data volume and label availability
  • model capacity and training cost

Too small a window increases variance; too large a window reintroduces staleness.

Common Pitfalls

  • selecting a window size without drift analysis
  • leaking future information via features or preprocessing
  • ignoring delayed or censored labels
  • comparing results across different window sizes without disclosure
  • assuming rolling windows guarantee robustness

Window design is a modeling decision.

Relationship to Time-Series Validation

Rolling window sampling is a core building block of time-series validation. It defines which data is eligible for training at each evaluation step, while validation protocols define aggregation and reporting.

Relationship to Rolling Retraining

Rolling window sampling underpins rolling retraining pipelines by specifying the historical scope of each retraining cycle.

Relationship to Generalization

Rolling window sampling estimates near-term generalization under evolving distributions. It does not measure long-term stability or robustness to sudden shifts.

Related Concepts

  • Data & Distribution
  • Time-Aware Sampling
  • Forward-Chaining Splits
  • Time-Series Validation
  • Rolling Retraining
  • Concept Drift
  • Distribution Shift