Skip to content

🎓 Mission Debrief: Fix the Deployment

What Happened

Your deployment was configured with replicas: 0, which tells Kubernetes: "I want ZERO instances of this application running."

This is technically valid configuration - just not useful for serving traffic!

How Kubernetes Behaved

Here's the flow:

  1. Deployment Controller read your spec: "replicas: 0"
  2. ReplicaSet was created with desired count = 0
  3. No pods were created (working as designed!)
  4. Deployment status showed: 0/0 ready ✅ (from K8s perspective)

Kubernetes did exactly what you asked - but you probably wanted at least 1 replica running.

The Correct Mental Model

Key Concepts:

  1. Deployments manage ReplicaSets
  2. Deployment = desired state (how many pods, which image, etc.)
  3. ReplicaSet = ensures that many pods exist

  4. Pods = actual running containers

  5. Replicas = High Availability

  6. replicas: 0 = nothing running (maintenance mode)
  7. replicas: 1 = one pod (no redundancy)

  8. replicas: 3 = three pods (survives failures)

  9. Deployments are mutable

  10. Unlike pods, you CAN edit deployments
  11. Changes trigger rolling updates

  12. Old ReplicaSet scaled down, new one scaled up

  13. Multiple ways to scale 

    # Imperative (quick fix)
kubectl scale deployment web --replicas=3 -n k8squest

# Declarative (proper way)
# Edit the YAML and kubectl apply

# Interactive
kubectl edit deployment web -n k8squest

What You Should Remember:

  • Deployments > Pods for production workloads
  • replicas: 0 is valid but means "nothing running"
  • You can edit deployments live (unlike pods)
  • Check readyReplicas not just replicas (pods might be crashing!)

Real-World Incident Example

Scenario: Black Friday Disaster

What happened: An e-commerce company was preparing for Black Friday. A DevOps engineer was testing autoscaling and accidentally committed:

spec:
  replicas: 0  # TODO: test HPA from 0

The change went through CI/CD and deployed to production at 11:45 PM on Thanksgiving.

Impact: - At midnight (Black Friday start), all checkout services scaled to ZERO - Website showed "Service Unavailable" - 15 minutes of downtime - Estimated $2.3M in lost sales - Made tech news headlines

Root cause: - No review on the replica count change - No minimum replica validation in CI/CD - No alerts for "deployment has zero replicas"

How it was fixed: 

# Emergency fix (1 minute)
kubectl scale deployment checkout --replicas=10 -n production

# Permanent fix:
# 1. Added git pre-commit hook: replicas must be >= 1
# 2. Added admission webhook: block replicas: 0 in production
# 3. Added alert: deployment.spec.replicas < 1

Lesson: - Always have minimum replica counts for critical services - Use admission controllers to prevent dangerous configs - Monitor desired vs actual state

How This Applies to Your Career

Interview Questions You Can Now Answer:

Q: "What's the difference between a Pod and a Deployment?"

A: - Pod = single instance, ephemeral, can't self-heal - Deployment = manages multiple pods, ensures desired count, handles rolling updates, self-healing

In production, you almost never create pods directly. Deployments handle pod lifecycle.

Q: "How do you scale an application in Kubernetes?"

A: 

# Horizontal scaling (more pods)
kubectl scale deployment <name> --replicas=5

# Or update the deployment YAML
spec:
  replicas: 5

# For automatic scaling
kubectl autoscale deployment <name> --min=2 --max=10 --cpu-percent=70

Q: "A deployment shows 0/3 ready. What could be wrong?"

A: Could be: 1. Image pull errors (wrong image name/tag) 2. Pods crashing (bad config, missing env vars) 3. Health checks failing (readiness probe issues) 4. Resource limits (not enough CPU/memory on cluster)

Debug with: - kubectl get pods - see pod states - kubectl describe deployment <name> - see events - kubectl logs <pod-name> - see application logs

Commands You Mastered

# View deployment status
kubectl get deployment <name> -n <namespace>
kubectl get deployment <name> -n <namespace> -o wide

# See detailed events
kubectl describe deployment <name> -n <namespace>

# Scale imperatively
kubectl scale deployment <name> --replicas=N -n <namespace>

# Edit declaratively
kubectl edit deployment <name> -n <namespace>

# Watch rollout status
kubectl rollout status deployment/<name> -n <namespace>

# See the ReplicaSets created by deployment
kubectl get rs -n <namespace>

# See the pods managed by deployment
kubectl get pods -l app=<label> -n <namespace>

Next Steps

You now understand: - ✅ How Deployments manage pods - ✅ The replica count and its importance - ✅ Different ways to scale applications - ✅ The relationship: Deployment → ReplicaSet → Pods

Next challenge: We'll explore more complex deployment scenarios with health checks and rolling updates.

💡 Pro tip: In production, use Horizontal Pod Autoscaler (HPA) to automatically scale based on CPU/memory usage. Manual replica counts are for when you know exactly what you need.