Neural Network Lexicon

Short Definition

Resilience testing evaluates whether a machine learning system continues to operate reliably under stress, failure, or unexpected conditions.

Definition

Resilience testing systematically subjects ML systems to adverse scenarios—such as traffic spikes, component failures, distribution shifts, and resource constraints—to verify that reliability mechanisms (admission control, degradation, fallback, and recovery) function as intended. The goal is not peak performance, but predictable behavior under duress.

Resilience is proven under failure, not normal operation.

Why It Matters

In production ML systems:

• failures are inevitable
• load and data distributions change
• adaptive models increase variance
• latent weaknesses surface only under stress

Untested resilience is assumed resilience.

Core Principle

A system is reliable only if it behaves acceptably when things go wrong.

Normal-case success is insufficient.

Minimal Conceptual Illustration

Normal load → System OK
Stress / Failure → Test Injection
                     ↓
              Controlled degradation
                     ↓
               SLA preserved

What Resilience Testing Covers

Resilience testing typically evaluates:

• latency behavior under overload
• tail-latency stability
• admission control effectiveness
• fallback and degradation paths
• recovery after failure
• correctness under constrained modes

Reliability is multi-faceted.

Common Stress Scenarios

Effective tests include:

• traffic bursts and flash crowds
• sustained near-capacity load
• partial hardware or node failures
• dependency outages (databases, caches)
• distribution shift or harder inputs
• cold starts and cache invalidation

Stress must be realistic.

Relationship to Capacity Headroom Planning

Headroom defines safety margins; resilience testing verifies that those margins are sufficient under real stress patterns.

Planning without testing is guesswork.

Relationship to Admission Control

Resilience testing validates whether admission control:

• triggers early enough
• preserves critical traffic
• avoids oscillatory rejection behavior

Admission must fail gracefully.

Interaction with Graceful Degradation

Resilience testing ensures that:

• degradation activates when expected
• degraded outputs remain acceptable
• degradation does not introduce bias or instability

Degradation paths must be exercised.

Evaluation Metrics

Resilience should be evaluated using:

• SLA violation rates under stress
• p95 / p99 latency stability
• rejection and fallback activation rates
• recovery time after failure
• correctness in degraded modes

Accuracy alone is insufficient.

Testing Approaches

Common approaches include:

• load and stress testing
• chaos engineering (fault injection)
• traffic replay from production
• canary deployments under load
• scenario-based simulations

Failures should be intentional.

Failure Modes Without Resilience Testing

Systems lacking resilience testing often exhibit:

• cascading failures
• unpredictable latency spikes
• emergency rollbacks
• brittle recovery behavior

Most outages are untested paths.

Practical Design Guidelines

• test beyond expected peak load
• inject failures regularly, not once
• test with adaptive features enabled
• validate recovery, not just failure
• document known failure modes

Resilience is a continuous practice.

Common Pitfalls

• testing only average conditions
• disabling safeguards during tests
• ignoring tail latency
• assuming infra teams cover resilience
• treating testing as a one-time activity

Reliability decays without practice.

Summary Characteristics

Related Concepts

• Generalization & Evaluation
• Capacity Headroom Planning
• Admission Control
• Graceful Degradation
• SLA-Aware Inference Policies
• Tail Latency Metrics
• Latency Drift Monitoring
• Fallback Models