Skip to content

Anti-Primer: Docker

Everything that can go wrong, will — and in this story, it does.

The Setup

A startup is rushing to ship a customer-facing feature by end of sprint. The lone DevOps engineer is building Docker images for a new microservice while the backend team pushes last-minute code changes. No staging environment exists yet.

The Timeline

Hour 0: Running as Root

Ships a Dockerfile with no USER directive because it 'works in dev'. The deadline was looming, and this seemed like the fastest path forward. But the result is container escape vulnerability exposed during a security audit two weeks later.

Footgun #1: Running as Root — ships a Dockerfile with no USER directive because it 'works in dev', leading to container escape vulnerability exposed during a security audit two weeks later.

Nobody notices yet. The engineer moves on to the next task.

Hour 1: Using latest Tag

Deploys myapp:latest to production because pinning versions 'slows us down'. Under time pressure, the team chose speed over caution. But the result is a node restart pulls a newer image; half the fleet runs untested code.

Footgun #2: Using latest Tag — deploys myapp:latest to production because pinning versions 'slows us down', leading to a node restart pulls a newer image; half the fleet runs untested code.

The first mistake is still invisible, making the next shortcut feel justified.

Hour 2: No .dockerignore

COPY . . sends the entire repo including .env and .git into the image. Nobody pushed back because the shortcut looked harmless in the moment. But the result is API keys are baked into image layers and pushed to a shared registry.

Footgun #3: No .dockerignore — COPY . . sends the entire repo including .env and .git into the image, leading to API keys are baked into image layers and pushed to a shared registry.

Pressure is mounting. The team is behind schedule and cutting more corners.

Hour 3: Mounting Docker Socket

Mounts /var/run/docker.sock into a CI runner for convenience. The team had gotten away with similar shortcuts before, so nobody raised a flag. But the result is a compromised CI job gains root-equivalent access to the host.

Footgun #4: Mounting Docker Socket — mounts /var/run/docker.sock into a CI runner for convenience, leading to a compromised CI job gains root-equivalent access to the host.

By hour 3, the compounding failures have reached critical mass. Pages fire. The war room fills up. The team scrambles to understand what went wrong while the system burns.

The Postmortem

Root Cause Chain

# Mistake Consequence Could Have Been Prevented By
1 Running as Root Container escape vulnerability exposed during a security audit two weeks later Primer: Non-root USER directive and security scanning
2 Using latest Tag A node restart pulls a newer image; half the fleet runs untested code Primer: Immutable image tags and digest pinning
3 No .dockerignore API keys are baked into image layers and pushed to a shared registry Primer: .dockerignore with secrets and build artifacts excluded
4 Mounting Docker Socket A compromised CI job gains root-equivalent access to the host Primer: Rootless Docker or dedicated build infrastructure

Damage Report

  • Downtime: 1-4 hours of degraded container builds or deployments
  • Data loss: None directly, but vulnerable images may remain in production
  • Customer impact: Delayed releases; potential security exposure from unscanned images
  • Engineering time to remediate: 8-12 engineer-hours for remediation and pipeline repair
  • Reputation cost: Security team flags container practices; additional review gates added

What the Primer Teaches

  • Footgun #1: If the engineer had read the primer, section on running as root, they would have learned: Non-root USER directive and security scanning.
  • Footgun #2: If the engineer had read the primer, section on using latest tag, they would have learned: Immutable image tags and digest pinning.
  • Footgun #3: If the engineer had read the primer, section on no .dockerignore, they would have learned: .dockerignore with secrets and build artifacts excluded.
  • Footgun #4: If the engineer had read the primer, section on mounting docker socket, they would have learned: Rootless Docker or dedicated build infrastructure.

Cross-References