Shadow Deployment

A shadow deployment sends production requests to the current service and copies the same requests to a candidate model whose response is logged but not shown to users. It tests model-serving integration, latency, resource use, and output distributions before a canary deployment exposes users.

Side-effect-free shadowing

The stable path remains authoritative. The shadow path must be side-effect free: no emails, charges, database writes, recommendation impressions, or policy actions. Every copied request should carry the same correlation ID so observability can compare stable and shadow behavior.

flowchart TD
  Request[Production request] --> Stable[Stable model: authoritative response]
  Stable --> User[User]
  Request --> Shadow[Shadow model: side-effect free]
  Shadow --> Logs[Log scores and latency with a correlation ID]
  Logs --> Compare[Compare stable versus shadow behavior]

Artifact: Shadow Routing Policy

endpoint: fraud-score-prod
production_variant:
  name: stable-v41
  initial_weight: 1.0
shadow_variants:
  - name: candidate-v42
    sampling_percentage: 20
    capture:
      fields: [request_id, model_version, score, latency_ms, error]
      destination: s3://ml-observability/fraud-shadow/2026-07-11/
side_effect_policy:
  allow_writes: false
  allow_external_calls: false

The useful comparison is not only “did it crash?” but “where do scores differ and why?” Pair shadow logs with monitoring dashboards for latency, timeout rate, output quantiles, and missing-feature errors.

Limits

Shadowing cannot estimate user reaction because users never see the candidate output. It also cannot detect policies triggered only after exposure, such as feedback loops in recommenders. When shadow results look safe, the next step is a limited canary with explicit rollbacks.

References

• Amazon SageMaker: Testing models with shadow variants
• OpenTelemetry Signals

Nav

Section — ML Engineering and MLOps

← Docker Canary Deployment →