SyncTrix logoSyncTrix
All articles
Platform10 min read

Blue-green, canary or rolling: choose the deploy strategy your rollback can support

The deploy strategy question is really a rollback question. Here's what each approach actually guarantees, and why the database usually decides for you.

By Lena Voss
Blue-green, canary or rolling: choose the deploy strategy your rollback can support

Teams argue about deploy strategies as though they were interchangeable options with different price tags. They are not - they differ in one thing that matters, which is how quickly and how completely you can undo a bad release. Pick the strategy by working backwards from your worst realistic failure and asking what it would take to reverse it, because that is the only property you will care about at the moment it matters.

01Rolling: the default, and the one with the messiest middle

Rolling updates replace instances in batches - take down two of ten, start two new ones, wait for health checks, repeat. It needs no extra capacity beyond one batch, it is what Kubernetes does out of the box, and for the vast majority of stateless services it is entirely adequate.

The property to understand is that during the roll, both versions serve production traffic simultaneously, and the roll takes as long as it takes. A ten-instance service with slow startup can spend six or seven minutes in a mixed state. Rollback means rolling again in the opposite direction, so your worst-case exposure is roughly twice the roll duration - which is the number to compare against your other options, not the deploy duration itself.

  • Both versions serve traffic for the whole roll - the API contract must be compatible in both directions.
  • Rollback time is a second full roll, not an instant switch.
  • Readiness probes must genuinely reflect readiness, or you roll broken instances into service.
  • Set maxUnavailable and maxSurge deliberately; the defaults optimise for speed, not safety.

02Blue-green: instant rollback, double the infrastructure

You run two complete environments. Blue serves production, green receives the new version, you verify green in isolation, then flip the router. The property you are buying is that rollback is the same flip in reverse, executing in seconds rather than minutes, with no partially-updated fleet in between.

The costs are concrete: you pay for double capacity during the overlap, and the cutover is all-or-nothing, so a bug that only appears under real production load hits 100% of users at once. Blue-green gives you the fastest recovery and the worst blast radius - it is the correct choice when your failures are the kind that show up immediately and obviously, and the wrong choice when they are subtle regressions that take twenty minutes of real traffic to surface.

03Canary: smallest blast radius, most machinery

Canary sends a small fraction of traffic - typically 1% to 5% - to the new version, compares its error rate and latency against the baseline, then progressively increases the share if the comparison holds. Rollback means routing that fraction back, which affects only the users already on the canary.

This is the only strategy that limits exposure while gathering real production evidence, and it is also the only one that requires genuine machinery: traffic splitting at the router or mesh, metrics labelled by version so you can compare like with like, and an automated analysis step with a defined abort rule. Canary without automated analysis is just a slow rolling deploy with someone watching a dashboard, and the person watching gets distracted at exactly the wrong moment.

StrategyRollback timeBlast radius on failureExtra capacityMachinery required
RollingOne full roll (minutes)Grows as the roll proceedsOne batchHealth checks
Blue-greenSeconds - reverse the flip100% at cutover2x during overlapRouter switch, duplicate env
CanarySeconds for the canary share1-5% initiallyOne extra instanceTraffic splitting, per-version metrics, analysis
Feature flagSeconds, no redeployWhatever cohort you targetedNoneFlag service, flag hygiene
Deploy strategies by the properties that matter

04The database decides more than the strategy does

Every one of these strategies assumes you can revert the code. Schema changes are not revertible on the same timescale, and a data migration that has already run cannot be undone by flipping a router. This is why teams with instant code rollback still have hour-long incidents: the code went back and the data did not.

The discipline that makes deploy strategy meaningful is decoupling schema changes from code changes entirely. Ship the additive schema change in its own deploy, ship the code that uses it later, and never let a single release contain both a destructive migration and the code that depends on it. If your migration is not backwards compatible with the previous release, your rollback story is fiction regardless of which strategy the deploy pipeline is running.

05What to measure during a canary, and for how long

Compare the canary against a concurrent baseline, not against yesterday. Traffic mix, cache warmth and upstream behaviour all vary by time of day, so a comparison against historical data will flag differences that have nothing to do with your release. The metrics worth automating on are error rate, latency at p95 and p99, saturation signals like CPU and pool utilisation, and one or two business counters such as successful checkouts per minute.

The bake time has to be long enough for the failure modes you actually experience. Memory leaks, connection leaks and cache-related regressions do not appear in ninety seconds - they appear after twenty minutes at load. A canary that promotes after two minutes catches crash-loops and nothing else, which is worth something, but do not mistake it for having tested the release.

  • Compare against a concurrent baseline, never against a historical window.
  • Label metrics by version at emission - retrofitting this during an incident is not possible.
  • Include at least one business metric; technical metrics can all look fine while conversions drop.
  • Set an automatic abort rule and let it fire without a human in the loop.

06Feature flags are the fourth strategy, and often the right one

Deploying code and releasing behaviour are separate operations. Ship the new code path behind a flag defaulting to off, deploy it with a boring rolling update, then enable it for 1% of users and increase from there. Rollback is a flag change taking effect in seconds without touching the deploy pipeline at all.

This gives you canary-grade blast radius control with rolling-grade infrastructure, and it works at the level of individual behaviours rather than whole releases, so you can enable one risky change while ten safe ones ship normally. The cost is discipline: flags must have owners and removal dates, because a codebase carrying two hundred stale flags has an untestable combinatorial state space and every one of those branches is a path that can break.

07Choosing, concretely

For a stateless internal service where a few minutes of degradation is acceptable, use rolling and spend the saved effort elsewhere. For a service where failures are immediate and obvious - it starts, or it does not - blue-green gives the fastest recovery for modest cost. For a high-traffic customer-facing path where a subtle regression is expensive, canary with automated analysis is worth the machinery, and feature flags on top of it are worth more still.

The one combination to avoid is elaborate progressive delivery on top of an unrevertible data layer, because it produces confidence that the rollback story does not support. Fix the migration discipline first; the deploy strategy is the cheaper half of the problem and it depends entirely on the other half being solved.

Topics

blue green vs canary deploymentrolling deployment kubernetesprogressive delivery rollbackcanary analysis metricsdeployment strategy comparisonzero downtime deploy database

Lena Voss

Lead Architect · SyncTrix

Writes about the engineering decisions behind production systems - architecture, delivery and the trade-offs that only show up at scale.

Building something like this?

SyncTrix engineers AI, SaaS, platform and cloud systems for enterprises and high-growth teams. Tell us what you're shipping and we'll scope it with you.