Neural Network Lexicon

Short Definition

Concept drift occurs when the relationship between inputs and outputs changes over time.

Definition

Concept drift refers to a change in the underlying mapping between input features and target labels after a model has been trained. Even if input data appears similar, the meaning of patterns can evolve, causing previously learned relationships to become invalid.

Unlike simple data distribution changes, concept drift alters what the correct prediction should be.

Why It Matters

Models trained on historical data assume that past relationships remain valid. When concept drift occurs, predictions can degrade even if input distributions look stable.

Concept drift is a common cause of failure in long-running, real-world systems.

Common Types of Concept Drift

• Sudden drift: abrupt changes in the target relationship
• Gradual drift: slow, continuous evolution
• Incremental drift: small changes accumulating over time
• Recurring drift: previously seen concepts reappear

Each type requires different detection and response strategies.

How Concept Drift Arises

Concept drift can be caused by:

• changing user behavior
• evolving preferences or norms
• market or environmental changes
• policy or regulatory updates
• feedback loops from deployed models

These changes often reflect real-world dynamics rather than data errors.

How Models Are Affected

• Predictions become systematically incorrect
• Confidence estimates lose meaning
• Decision thresholds become misaligned
• Performance degrades despite stable input features

Models cannot adapt without retraining or updates.

Minimal Conceptual Example

# conceptual illustration
P(y | x, time=t1) != P(y | x, time=t2)

This highlights that the label-generating process has changed.

Detecting Concept Drift

Common approaches include:

• monitoring prediction errors over time
• tracking changes in feature–label relationships
• using sliding-window evaluation
• comparing recent performance to historical baselines

Detection often requires labeled feedback.

Common Pitfalls

• Confusing concept drift with covariate shift
• Relying solely on static test sets
• Ignoring feedback loops
• Delaying model updates too long

Concept drift is inevitable in dynamic systems.

Relationship to Distribution Shift

Concept drift is a specific type of distribution shift where the conditional relationship between inputs and outputs changes. Not all distribution shifts involve concept drift, but all concept drift implies a shift in the underlying data-generating process.

Related Concepts

• Data & Distribution
• Distribution Shift
• Data Distribution
• Model Monitoring
• Generalization
• Retraining