Imputation

Short Definition

Imputation is the process of filling in missing data with estimated values.

Definition

Imputation refers to techniques used to replace missing values in a dataset with plausible substitutes based on observed data. Rather than discarding incomplete samples, imputation preserves data volume and structure by estimating missing entries using statistical rules or learned models.

Imputation is a data preprocessing choice that directly shapes the learning signal.

Why It Matters

Missing data is common in real-world datasets. Naively removing incomplete samples can introduce bias, reduce effective sample size, and distort data distributions. Imputation enables models to learn from partially observed data while controlling for the impact of missingness.

Poor imputation, however, can introduce misleading patterns and false confidence.

Common Imputation Strategies

Imputation methods range from simple to complex:

Constant imputation: replace with a fixed value (e.g., 0 or “unknown”)
Statistical imputation: mean, median, or mode
Conditional imputation: values inferred from other features
Model-based imputation: learned estimators or predictive models
Multiple imputation: generate several plausible values to reflect uncertainty

The appropriate strategy depends on the data and missingness mechanism.

Imputation and Missingness Assumptions

Imputation methods implicitly assume a type of missingness:

MCAR: simple methods may suffice
MAR: conditional or model-based methods are preferred
MNAR: imputation is risky and may bias results

Incorrect assumptions can invalidate downstream analysis.

How Imputation Affects Models

alters feature distributions
reduces variance introduced by missing values
can hide uncertainty from missingness
may introduce artificial correlations

Imputed values are estimates, not observations.

Minimal Conceptual Example

			
# conceptual example
x_filled = impute(x_missing, strategy="mean")

Common Pitfalls

imputing before train/test splitting
using global statistics that include test data
ignoring patterns in missingness
treating imputed values as ground truth
overconfident predictions on heavily imputed samples

Imputation must be applied carefully and transparently.

Imputation vs Model-Based Handling

Some models can handle missing values natively or learn to ignore them. Imputation shifts responsibility from the model to the data pipeline. The choice depends on model capabilities, data structure, and interpretability requirements.

Relationship to Data Quality and Bias

Imputation affects data quality by introducing assumptions about missing values. Systematic missingness combined with naive imputation can amplify sampling bias and distort generalization.

Neural Network Lexicon