Rolling Updates, Rollback & Blue-Green for AI Inference

Three deployment strategies — and the readiness-probe contract

You already know how to author a Deployment (last lab). Now you're going to update one — replace v1 with v2 — without dropping inference traffic. Kubernetes gives you three native strategies for this. Picking the right one depends on what your workload tolerates during the cutover window.

A fourth pattern, canary, is two Deployments behind one Service with the traffic split coming from replica counts (9 v1 + 1 v2 = 10% canary). It's a useful primitive but real production canaries usually go through a service-mesh layer (Istio, Linkerd) for percentage-precise traffic splitting; we'll skip it for this lab and focus on the three Kubernetes-native ones.

The readiness probe is non-optional

The single most important field for any of these strategies is the pod template's readinessProbe. Without it, Kubernetes considers a pod Ready as soon as its container starts — not when the application inside is actually serving. For an inference server that needs 30 seconds to load weights from disk, that 30-second window is when:

1. The Service routes incoming requests to the pod (because Endpoints sees Ready=True).
2. The application returns 500s because model.from_pretrained() is still running.

Result: a successful "rolling update" that actually drops every Nth request through the rollout.

A readiness probe is one of three forms:

readinessProbe:
  # 1. HTTP — the most common, tests an app endpoint
  httpGet:
    path: /v1/models
    port: 8000
  initialDelaySeconds: 5
  periodSeconds: 5

  # 2. Exec — tests by running a command in the container
  exec:
    command: [test, -f, /tmp/ready]   # the app touches this file when serving
  initialDelaySeconds: 5
  periodSeconds: 2

  # 3. TCP — tests that a port is listening (weakest)
  tcpSocket:
    port: 8000
  initialDelaySeconds: 5
  periodSeconds: 5

In this lab we'll use exec probes against a /tmp/ready marker file — our stub container "loads its model" by sleeping a few seconds, then touch /tmp/ready to signal it's serving. The Service-routing flow is the same as production HTTP probes; we just don't run a real HTTP server.

Read what's there

Run the step's command (or click Run). It dumps:

1. Every Deployment in the cluster.
2. CoreDNS's strategy block — the cluster's own rolling-update settings, a baseline for what defaults look like.
3. A summary of the three strategy fields you'll touch in the next steps.

Note the default rolling-update settings (maxSurge: 25%, maxUnavailable: 25%). For a 4-replica Deployment, that means at most 5 pods exist during a rollout (4 + 25%) and at most 1 can be Unavailable at a time (4 - 25%). For inference workloads, those defaults are usually fine.

Two production-relevant trade-offs

maxSurge controls cost during rollout. A larger value = faster rollout = more replicas in flight = more GPU/CPU. For 8-replica GPU inference, maxSurge: 1 (rather than 25% = 2) keeps you from briefly running 10 GPU pods.

maxUnavailable controls availability margin. A larger value = more pods can be down at once = lower capacity during the rollout. For a service near its capacity ceiling, maxUnavailable: 0 forbids any drop in available replicas during rollout (and maxSurge >= 1 is required to make progress).

Why this matters in practice: inference deployment is the most-frequent action a platform engineer takes. Knowing which strategy + which readinessProbe shape + which surge/unavailable trade-off your workload needs is the basic motor skill the NCA-AIIO exam tests under AI Infrastructure & Operations.