Skip to content

Cluster Upgrade Exercise

Objective

Practice a rolling k3s upgrade using Ansible, following production-safe patterns.

Prerequisites

  • A running k3s cluster (bootstrap with ansible-playbook playbooks/bootstrap-k3s.yml)
  • Application deployed (./devops/scripts/deploy-local.sh)

Steps

1. Check current version

k3s --version
kubectl get nodes -o wide

2. Choose target version

Find available versions at https://github.com/k3s-io/k3s/releases

3. Run the upgrade playbook

cd devops/ansible
ansible-playbook playbooks/upgrade-k3s.yml -e k3s_version=v1.31.0+k3s1

4. What the playbook does

For each node (serial: 1, one at a time):

  1. Cordon — marks node as unschedulable (skipped for single-node)
  2. Drain — evicts workloads to other nodes (skipped for single-node)
  3. Upgrade — runs k3s installer with new version
  4. Restart — restarts the k3s systemd service
  5. Wait — polls until the node reports Ready
  6. Uncordon — marks node schedulable again

5. Verify

# Check version
k3s --version

# Check node status
kubectl get nodes

# Check application is running
kubectl get pods -n grokdevops
curl http://localhost:8000/health  # via port-forward

Single-Node vs Multi-Node

Step Single-Node Multi-Node
Cordon Skipped Yes
Drain Skipped Yes
Upgrade Yes Yes (serial)
Restart Yes Yes
Uncordon Skipped Yes

On single-node clusters, the cordon/drain is skipped because there's nowhere to move workloads. Pods will restart after the k3s service restarts.

Rollback

If the upgrade fails, reinstall the previous version:

ansible-playbook playbooks/upgrade-k3s.yml -e k3s_version=v1.30.4+k3s1