Skip to content

How We Got Here: Kubernetes Itself

Arc: Platform Eras covered: 5 Timeline: ~2010-2025 Read time: ~12 min

The Original Problem

By 2010, containers were becoming practical (LXC was working, Google was running everything in containers internally), but nobody outside Google had a good answer for: "I have 1000 containers. How do I schedule them across 100 machines? How do I restart them when they crash? How do I route traffic to the right ones? How do I update them without downtime?" Running one container was easy. Running a fleet was an unsolved problem for the rest of the industry.

Google had solved this internally with Borg (2003-2004), but Borg was deeply entangled with Google's infrastructure and was not open-sourceable. The rest of the world needed its own container orchestrator.

Era 1: Google Borg and Internal Orchestration (~2003-2013)

The Solution

Borg (named after the Star Trek species) was Google's internal cluster management system. It ran virtually everything at Google — from Gmail to Search to MapReduce. Borg managed resource allocation, scheduling, health checking, and service discovery across millions of containers on hundreds of thousands of machines. Omega (2013) was an evolution that explored alternative scheduling architectures.

What It Looked Like

// Borg job specification (simplified, based on published papers)
job {
  name: "frontend"
  priority: PRODUCTION
  container {
    image: "//production/frontend:release-2013.03.15"
    resources {
      cpu: 2.0  // cores
      ram: 4GB
      disk: 10GB
    }
    ports {
      name: "http"
      port: 8080
    }
  }
  replicas: 500
  constraints {
    spread_across: "rack"    // anti-affinity
    require_machine_class: "highcpu"
  }
  update_strategy {
    max_unavailable: 10
    canary_count: 5
    canary_duration: "10m"
  }
}

Why It Was Better

  • Ran all of Google's production workloads for a decade+
  • Bin-packing: optimal resource utilization across the fleet
  • Self-healing: automatic rescheduling on failure
  • Priority-based preemption: batch jobs yield to production
  • Service discovery and load balancing built in

Why It Wasn't Enough

  • Proprietary: only Google had it
  • Deeply coupled to Google's infrastructure (Colossus, Chubby, Stubby)
  • Configuration was complex and Google-specific
  • The rest of the industry had no equivalent

Legacy You'll Still See

Borg's concepts appear directly in Kubernetes: pods (Borg allocs), ReplicaSets (Borg tasks), labels and selectors (Borg labels), namespaces (Borg cells). The "Large-scale cluster management at Google with Borg" paper (2015) is required reading for understanding Kubernetes design decisions. Several Kubernetes founders (Craig McLuckie, Joe Beda, Brendan Burns) worked on Borg.

Era 2: Apache Mesos and Marathon (~2011-2016)

The Solution

Apache Mesos (2009, from UC Berkeley's AMPLab; Apache TLP 2013) offered "datacenter-scale resource management." Mesos abstracted the entire datacenter as a single pool of resources. Marathon (2013, by Mesosphere/D2IQ) ran on top as an orchestration framework for long-running services. Together, they were the first widely available open-source answer to "how do I run containers at scale?"

What It Looked Like

{
  "id": "/production/frontend",
  "cmd": null,
  "cpus": 1.0,
  "mem": 2048,
  "instances": 10,
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "registry.example.com/frontend:v2.3.1",
      "network": "BRIDGE",
      "portMappings": [
        { "containerPort": 8080, "hostPort": 0, "protocol": "tcp" }
      ]
    }
  },
  "healthChecks": [
    {
      "protocol": "HTTP",
      "path": "/healthz",
      "portIndex": 0,
      "gracePeriodSeconds": 30,
      "intervalSeconds": 10,
      "maxConsecutiveFailures": 3
    }
  ],
  "upgradeStrategy": {
    "minimumHealthCapacity": 0.9,
    "maximumOverCapacity": 0.1
  }
}

# Deploy via Marathon REST API
curl -X PUT http://marathon.example.com:8080/v2/apps/production/frontend \
  -H "Content-Type: application/json" \
  -d @frontend.json

Why It Was Better

  • Open source and available to anyone
  • Two-level scheduling: Mesos offered resources, frameworks decided how to use them
  • Ran diverse workloads: containers, Spark, Hadoop, Cassandra on the same cluster
  • Proven at scale: Twitter, Airbnb, Apple ran Mesos in production
  • Marathon provided rolling deployments, health checks, and service discovery

Why It Wasn't Enough

  • Complex architecture: Mesos master, Mesos agents, ZooKeeper, Marathon, Chronos
  • Two-level scheduling was powerful but hard to reason about
  • Docker support was added later and always felt second-class
  • The ecosystem was smaller than what Kubernetes would build
  • Mesosphere/D2IQ pivoted to Kubernetes, signaling the end

Legacy You'll Still See

Mesos is in maintenance mode but still running at some large companies. DC/OS (Mesosphere's commercial Mesos distribution) was discontinued. The conceptual influence persists — Mesos's resource offer model informed scheduler design across the industry. If you encounter Mesos, the organization is likely mid-migration to Kubernetes.

Era 3: Docker Swarm (~2014-2018)

The Solution

Docker Swarm (2014, rewritten as "Swarm Mode" in Docker 1.12, 2016) was Docker's native orchestration answer. It extended the Docker CLI with clustering, scheduling, and service management. If you knew docker run, you could learn docker service create in minutes. It was simple by design — the anti-Kubernetes.

What It Looked Like

# Initialize a Swarm cluster
docker swarm init --advertise-addr 192.168.1.10

# Join worker nodes
docker swarm join --token SWMTKN-1-abc123... 192.168.1.10:2377

# Deploy a service
docker service create \
  --name frontend \
  --replicas 10 \
  --publish 80:8080 \
  --update-parallelism 2 \
  --update-delay 10s \
  --restart-condition on-failure \
  registry.example.com/frontend:v2.3.1

# Scale
docker service scale frontend=20

# Update
docker service update --image registry.example.com/frontend:v2.4.0 frontend

# docker-compose.yml with deploy section (Swarm mode)
services:
  frontend:
    image: registry.example.com/frontend:v2.3.1
    deploy:
      replicas: 10
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
    ports:
      - "80:8080"

Why It Was Better

  • Radically simpler than Mesos or early Kubernetes
  • Built into Docker — no separate installation
  • docker-compose compatibility (same YAML format, extended for Swarm)
  • Fast setup: a working cluster in 5 minutes
  • Familiar CLI for anyone who used Docker

Why It Wasn't Enough

  • Too simple: lacked Kubernetes' extensibility (no CRDs, no operators)
  • Networking was limited compared to Kubernetes (no network policies)
  • No ecosystem: Helm, Operators, CNCF tooling all targeted Kubernetes
  • Docker Inc.'s business struggles undermined confidence
  • Auto-scaling required third-party tools
  • The industry chose Kubernetes, and ecosystem effects made that irreversible

Legacy You'll Still See

Docker Swarm is still supported in Docker Engine and works fine for small-scale deployments. Some small teams and single-server setups use Swarm because it's simpler than Kubernetes. If you encounter it, the recommendation is usually to evaluate migration to Kubernetes — not because Swarm is broken, but because the ecosystem has moved.

Era 4: Kubernetes Ascendance (~2014-2022)

The Solution

Kubernetes (Google, 2014; v1.0 and CNCF donation 2015) drew from Borg's lessons but was designed for the open-source ecosystem. It used a declarative model (desired state → reconciliation), an extensible API (Custom Resource Definitions), and a pluggable architecture (CNI for networking, CSI for storage, CRI for container runtimes). The CNCF ecosystem built around it at extraordinary speed.

What It Looked Like

# Kubernetes Deployment — the canonical workload
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: production
  labels:
    app: frontend
spec:
  replicas: 10
  selector:
    matchLabels:
      app: frontend
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 2
  template:
    metadata:
      labels:
        app: frontend
    spec:
      containers:
        - name: frontend
          image: registry.example.com/frontend:v2.3.1
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 15
            periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
  name: frontend
  namespace: production
spec:
  selector:
    app: frontend
  ports:
    - port: 80
      targetPort: 8080
  type: ClusterIP

Why It Was Better

  • Declarative: describe desired state, the system reconciles
  • Extensible: CRDs let anyone extend the API with new resource types
  • Self-healing: crashed pods are restarted, failed nodes are drained
  • Service discovery and load balancing built in
  • Massive ecosystem: Helm, Operators, CNCF projects, commercial tools
  • Cloud-provider support: EKS, GKE, AKS all manage the control plane
  • Became the industry standard — knowledge was portable across companies

Why It Wasn't Enough

  • Complexity: the learning curve is steep and wide
  • YAML fatigue: hundreds of lines for a simple service
  • Resource overhead: control plane + system components consume significant resources
  • Debugging: multiple layers (pod, node, network, storage) make troubleshooting hard
  • Small teams: Kubernetes overhead doesn't pay off for fewer than ~10 services
  • Security defaults were permissive (pods could do too much by default)

Legacy You'll Still See

Kubernetes is the current standard. EKS, GKE, and AKS are the most common deployment targets for cloud-native applications. The Kubernetes API is the control plane for the entire CNCF ecosystem. If you work in DevOps or SRE, Kubernetes fluency is a required skill.

Era 5: Managed Kubernetes, vcluster, and Beyond (~2020-2025)

The Solution

The overhead of managing Kubernetes itself drove further abstraction. Managed services (EKS, GKE, AKS) handled the control plane. Managed node groups and Fargate/Autopilot handled the worker nodes. vcluster (2021) created lightweight virtual clusters inside a host cluster for multi-tenancy. K3s (Rancher, 2019) provided a minimal Kubernetes for edge and resource-constrained environments.

What It Looked Like

# EKS with managed node groups and Fargate
eksctl create cluster \
  --name production \
  --region us-west-2 \
  --managed \
  --node-type t3.large \
  --nodes 5 \
  --nodes-min 3 \
  --nodes-max 15

# Or fully serverless with Fargate
eksctl create cluster \
  --name production \
  --region us-west-2 \
  --fargate

# GKE Autopilot — Google manages everything
gcloud container clusters create-auto production \
  --region us-central1
# No node configuration, no node management, pay per pod

# vcluster — virtual cluster inside a host cluster
# Each team gets their own "cluster" without real infrastructure
apiVersion: v1
kind: Namespace
metadata:
  name: team-frontend

# Create a virtual cluster
# vcluster create team-frontend -n team-frontend
# Team gets full cluster admin, isolated from other teams
# But shares the same physical nodes

# K3s — Kubernetes for edge and IoT
curl -sfL https://get.k3s.io | sh -
# Full Kubernetes API, single binary, 512MB RAM
# Runs on Raspberry Pi, edge locations, CI environments

Why It Was Better

  • Control plane management eliminated (managed K8s)
  • Node management eliminated (Fargate, Autopilot)
  • Multi-tenancy without cluster sprawl (vcluster)
  • Edge and resource-constrained environments supported (K3s)
  • Cost optimization: Autopilot and Fargate bill per pod, not per node
  • Developer experience: virtual clusters for development and testing

Why It Wasn't Enough

  • Managed K8s still requires deep K8s knowledge for workloads
  • Cost management in Kubernetes is a discipline (FinOps for K8s)
  • Multi-cloud Kubernetes is harder than vendors claim
  • vcluster adds another abstraction layer to debug
  • Platform teams still needed to build IDPs on top
  • Kubernetes networking remains complex (CNI, Ingress, Gateway API)

Legacy You'll Still See

This is the current state. EKS, GKE, and AKS dominate production. K3s is used for edge, CI, and development. vcluster is growing for multi-tenancy. The "how much Kubernetes do I actually need to know?" question is answered differently at different organizations — platform teams know everything, application developers know kubectl get pods.

Where We Are Now

Kubernetes has won the orchestration war. The question is no longer "should we use Kubernetes?" but "how much Kubernetes complexity should we expose to developers?" Managed services handle the control plane. Platform engineering abstracts the workload management. The Kubernetes API is the universal control plane — not just for containers, but for infrastructure (Crossplane), networking (Cilium), security (Kyverno), and more. The challenge has shifted from "running Kubernetes" to "building a good developer experience on top of Kubernetes."

Where It's Going

WebAssembly workloads on Kubernetes (SpinKube, wasmCloud) will add a new runtime option alongside containers. The Gateway API will replace Ingress as the standard for traffic management. Platform engineering tools (Backstage, Humanitec) will hide Kubernetes from most developers. Kubernetes itself will become invisible — like the Linux kernel, essential but not directly interacted with by most engineers.

The Pattern

Every orchestration generation is a bet on the right level of abstraction between "hardware" and "application." Borg bet on internal infrastructure, Mesos on datacenter-level resources, Swarm on Docker simplicity, Kubernetes on extensible APIs. The winner was the one that attracted the largest ecosystem, because ecosystem breadth is the moat that kills competitors.

Key Takeaway for Practitioners

Learn Kubernetes deeply enough to debug problems, even if you never set up a cluster from scratch. Understand pods, deployments, services, and ingress. Understand resource requests and limits. Understand health checks. These concepts will outlast any specific version of Kubernetes, because they represent fundamental orchestration patterns.

Cross-References