Level 3: Production Kubernetes¶
Core K8s objects, probes, rollouts, services, DNS, scaling.
Concepts¶
pod, deployment, service, endpoints, namespace, labels, configmap, secret, readiness_probe, liveness_probe, horizontal_pod_autoscaler, resource_limits, resource_requests, rollout, replicaset, dns, networkpolicy, ingress, rbac, statefulset, daemonset, persistent_volume, pod_disruption_budget
Failure Patterns You Should Be Able to Resolve¶
- FP-001: crashloopbackoff
- FP-002: readiness_probe_failure
- FP-003: imagepullbackoff
- FP-004: oomkilled
- FP-005: hpa_not_scaling
- FP-006: dns_resolution_failure
- FP-007: networkpolicy_block
- FP-008: service_selector_mismatch
- FP-012: rollout_stuck
- FP-013: ingress_404
- FP-014: rbac_forbidden
Commands You Should Be Fluent With¶
kubectl get pods -n <ns>/-o wide/--show-labelskubectl logs -n <ns> deploy/<name>/--previouskubectl describe pod -n <ns> <pod>kubectl get endpoints -n <ns>kubectl rollout status/undo/historykubectl get hpa/describe hpakubectl top pods/top nodeskubectl exec -n <ns> <pod> -- <cmd>kubectl run <debug> --rm -i --restart=Never --image=busybox -- <cmd>kubectl auth can-ikubectl get events --sort-by=.lastTimestamp
Assets to Complete¶
Read (theory)¶
- training/library/skillchecks/kubernetes.skillcheck.md
- training/kubectl-debugging-cheatsheet.md
- devops/docs/troubleshooting.md
- training/library/skillchecks/kubernetes.under.the.covers.md
Practice (hands-on)¶
- Quest Ladder k8s track: levels 1-15 (
training/interactive/exercises/levels/level-*/k8s-*) - training/interactive/runtime-labs/lab-runtime-01-rollout-probe-failure/
- training/interactive/runtime-labs/lab-runtime-02-hpa-live-scaling/
- training/interactive/runtime-labs/lab-runtime-08-resource-limits-oom/
Runbooks (study)¶
- training/library/runbooks/crashloopbackoff.md
- training/library/runbooks/kubernetes/readiness_probe_failed.md
- training/library/runbooks/kubernetes/imagepullbackoff.md
- training/library/runbooks/oomkilled.md
- training/library/runbooks/kubernetes/hpa_not_scaling.md
- training/library/runbooks/dns_resolution.md