Progressive Delivery — Trivia & Interesting Facts¶
Surprising, historical, and little-known facts about progressive delivery.
The term "progressive delivery" was coined by James Governor in 2018¶
James Governor of RedMonk coined "progressive delivery" to describe the evolution beyond continuous delivery. While CD focuses on getting code to production quickly, progressive delivery adds controlled exposure: canary releases, feature flags, A/B testing, and traffic shifting. The insight was that deploying and releasing are different actions — you can deploy code to production without exposing it to any users.
Canary releases are named after coal mine canaries¶
The term comes from the practice of coal miners bringing canaries into mines to detect carbon monoxide. If the canary died, miners evacuated. In software, a canary release exposes a small percentage of users (the "canary") to a new version. If the canary shows elevated errors, the rollout stops before it affects everyone. The metaphor is grimly accurate — the canary traffic absorbs the impact of any bugs.
Blue-green deployments require double the infrastructure¶
In a blue-green deployment, you maintain two identical production environments. The "blue" runs the current version, the "green" runs the new version. Traffic is switched at the load balancer level. The advantage is instant rollback (switch back to blue). The disadvantage is cost — you need 2x the infrastructure during deployment. This is why blue-green is common for small services but impractical for large-scale systems, where canary deployments are preferred.
Argo Rollouts extends Kubernetes Deployments with canary and blue-green strategies¶
Kubernetes native Deployments only support rolling updates and recreate strategies. Argo Rollouts (a CNCF project by Intuit) adds canary deployments with traffic splitting, blue-green with traffic switching, analysis runs for automated rollback, and integration with service meshes and ingress controllers. It works by replacing the Deployment resource with a Rollout CRD that the Argo Rollouts controller manages.
Flagger automates canary analysis using Prometheus metrics¶
Flagger (a CNCF project by Weaveworks, now part of Flux) automates progressive delivery by monitoring canary metrics (error rate, latency percentiles, custom queries) from Prometheus, Datadog, or CloudWatch. If metrics exceed thresholds, Flagger automatically rolls back. If they stay healthy through all progressive steps, Flagger promotes the canary to full production. The entire process can run unattended, enabling true continuous delivery without human gatekeepers.
Feature flags decouple deployment from release entirely¶
Feature flags (LaunchDarkly, Unleash, Flagsmith, Split.io) allow code to be deployed to all servers but activated for specific users, percentages, or segments. This means a feature can exist in production code for weeks without any user seeing it. Netflix reportedly has over 1,000 active feature flags at any given time. The risk: flag debt — orphaned flags that nobody removes, creating a combinatorial explosion of code paths.
Netflix pioneered progressive delivery before the term existed¶
Netflix's deployment system (Spinnaker, launched internally around 2012, open-sourced in 2015) implemented canary analysis, automated rollback, and multi-region progressive rollouts years before "progressive delivery" was coined. Netflix's canary system compares real-time metrics between the canary and baseline using statistical analysis (Mann-Whitney U tests) rather than simple threshold checks, making it significantly more accurate at detecting regressions.
Traffic mirroring (shadowing) tests with real traffic and zero user impact¶
Traffic mirroring duplicates production requests to a new version but discards the responses. Users only see responses from the current version. This lets you test the new version's behavior, performance, and error rates with real traffic patterns — far more realistic than synthetic tests. Istio, Envoy, and NGINX all support traffic mirroring natively. The caveat: the mirrored service must not produce side effects (write to databases, send emails, charge credit cards).
The "blast radius" concept drives progressive delivery architecture¶
Blast radius — the maximum impact of a failed deployment — is the core metric of progressive delivery. A 1% canary has a blast radius of 1% of users. A feature flag enabled for internal employees has a blast radius of zero customers. Every progressive delivery technique is fundamentally a blast radius minimization strategy. Organizations with mature progressive delivery target blast radii of less than 0.1% for initial exposure of any change.
Automated rollback is harder than it sounds¶
Automatically rolling back when metrics degrade seems straightforward, but the implementation is complex. Questions include: How long do you wait before deciding metrics are bad? (Statistical significance requires time.) What about metrics that naturally fluctuate? (False positives cause unnecessary rollbacks.) What if the rollback itself introduces errors? (Stateful changes may not be reversible.) Most teams start with automated detection and manual rollback before trusting full automation.
GitOps and progressive delivery are complementary, not competing¶
GitOps (desired state in git, reconciled by a controller) defines what should be deployed. Progressive delivery defines how it is exposed. ArgoCD + Argo Rollouts, and Flux + Flagger are the canonical pairings. Git contains the new version, the GitOps controller deploys it, and the progressive delivery controller gradually shifts traffic. The combination provides both auditability (who changed what, when) and safety (controlled exposure).
Dark launches expose new code paths without visible UI changes¶
A dark launch deploys new backend functionality and routes real production traffic through the new code path, but the results are only logged — not shown to users. Facebook famously used dark launches before major feature releases, processing millions of real user requests through the new code path to validate performance and correctness at scale before any user saw the feature. This is different from feature flags, which control visibility; dark launches control execution.