1 Helm debugging issue — it produces invalid YAML.¶

lesson
helm-architecture
charts
templates
values
hooks
release-management
debugging ---# What Happens When You helm install

Topics: Helm architecture, charts, templates, values, hooks, release management, debugging Level: L1–L2 (Foundations → Operations) Time: 45–60 minutes Prerequisites: Basic Kubernetes understanding (see "What Happens When You kubectl apply")

The Mission¶

You type helm install myapp ./chart --values prod-values.yaml and press Enter. Helm says "deployed." But what actually happened between "enter" and "deployed"? How did a directory of templates become running pods?

What Helm Actually Is¶

Helm is a package manager for Kubernetes — it takes a directory of Go templates + values, renders them into Kubernetes YAML, submits them to the API server, and tracks what it deployed (so it can upgrade and rollback).

                ┌──────────────┐
helm install →  │ Chart (templates + values)     │
                │ values.yaml + prod-values.yaml │
                └───────┬──────┘
                        │ render templates
                        ↓
                ┌──────────────┐
                │ Rendered YAML │
                │ (valid K8s manifests) │
                └───────┬──────┘
                        │ kubectl apply (essentially)
                        ↓
                ┌──────────────┐
                │ API Server   │ → etcd → scheduler → kubelet → running pods
                └──────────────┘
                        │
                        ↓
                ┌──────────────┐
                │ Release record │ (stored as K8s Secret)
                └──────────────┘

The Chart Structure¶

mychart/
├── Chart.yaml          # Chart metadata (name, version, description)
├── values.yaml         # Default configuration values
├── templates/          # Go templates that produce K8s YAML
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── ingress.yaml
│   ├── configmap.yaml
│   ├── _helpers.tpl    # Reusable template snippets
│   └── NOTES.txt       # Post-install message
├── charts/             # Dependency charts (sub-charts)
└── .helmignore         # Files to exclude from packaging

Templates are Go templates, not YAML¶

# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "mychart.fullname" . }}
  labels:
    {{- include "mychart.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      {{- include "mychart.selectorLabels" . | nindent 6 }}
  template:
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          ports:
            - containerPort: {{ .Values.service.port }}
          resources:
            {{- toYaml .Values.resources | nindent 12 }}

The {{ }} blocks are Go template directives. .Values comes from values.yaml and overrides. .Chart comes from Chart.yaml.

The Install Sequence¶

When you run helm install myapp ./chart --values prod-values.yaml:

Step 1: Merge values¶

Chart default values.yaml
  + prod-values.yaml (overrides defaults)
  + --set flags (override everything)
  = Final merged values

# See the merged values without installing
helm install myapp ./chart --values prod-values.yaml --dry-run --debug | head -50

Step 2: Render templates¶

Helm runs Go's text/template engine, substituting values into templates. The output is valid Kubernetes YAML.

# See rendered YAML without installing
helm template myapp ./chart --values prod-values.yaml

Step 3: Validate¶

Helm validates the rendered YAML against the Kubernetes API schema. If a resource type doesn't exist (CRD not installed), Helm fails here.

Step 4: Submit to API server¶

Helm sends the rendered YAML to Kubernetes, similar to kubectl apply. The API server processes it through auth, RBAC, admission webhooks, and stores in etcd.

Step 5: Run hooks (if any)¶

Hooks are special resources that run at specific lifecycle points:

annotations:
  "helm.sh/hook": pre-install     # Run before main resources
  "helm.sh/hook": post-install    # Run after main resources
  "helm.sh/hook": pre-upgrade     # Run before upgrade
  "helm.sh/hook-weight": "5"      # Order among hooks
  "helm.sh/hook-delete-policy": hook-succeeded  # Clean up

Common use: database migrations as pre-upgrade hooks.

Step 6: Store release record¶

Helm creates a Secret in the release namespace containing the rendered manifests, values, and chart metadata. This is how Helm knows what to upgrade or rollback.

# See release history
helm history myapp
# → REVISION  STATUS    CHART          DESCRIPTION
# → 1         deployed  mychart-1.0.0  Install complete

# See the secret
kubectl get secret -l owner=helm
# → sh.helm.release.v1.myapp.v1

Common Helm Failures¶

Failure 1: Template rendering error¶

Error: template: mychart/templates/deployment.yaml:15:23:
  executing "mychart/templates/deployment.yaml" at <.Values.image.tag>:
  nil pointer evaluating interface {}.tag

A value is missing. .Values.image.tag is nil because image.tag isn't set in values or overrides.

Fix: Use default:

image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"

Failure 2: Indentation error¶

# BAD — toYaml output not indented correctly
resources:
{{ toYaml .Values.resources }}
# Renders with wrong indentation → invalid YAML

# GOOD — nindent handles indentation
resources:
  {{- toYaml .Values.resources | nindent 12 }}

This is the #1 Helm debugging issue. Go templates and YAML indentation don't compose naturally. Getting the nindent number right requires counting spaces.

Failure 3: Release in failed state¶

helm upgrade myapp ./chart
# Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress

# Check the release
helm list -a
# → NAME   STATUS          REVISION
# → myapp  pending-install  1

# Fix: force the status
helm rollback myapp 0    # Rollback to nothing (delete failed release)
# or
kubectl delete secret sh.helm.release.v1.myapp.v1
helm install myapp ./chart

Helm Upgrade and Rollback¶

# Upgrade (renders new templates, applies changes)
helm upgrade myapp ./chart --values prod-values.yaml

# Rollback to previous revision
helm rollback myapp 1

# See what would change (diff plugin)
helm diff upgrade myapp ./chart --values prod-values.yaml

Gotcha: helm rollback rolls back Kubernetes resources and Helm release record. It does NOT rollback database migrations, config changes, or anything outside Kubernetes. Same limitation as kubectl rollout undo — see "The Rollback That Wasn't."

Debugging Helm¶

# See rendered YAML (don't install)
helm template myapp ./chart --values prod-values.yaml

# Dry run against the actual cluster (validates against API)
helm install myapp ./chart --dry-run --debug

# Lint the chart (catches common errors)
helm lint ./chart --values prod-values.yaml

# See what values are actually being used
helm get values myapp

# See the full rendered manifest of a deployed release
helm get manifest myapp

# See all release info
helm get all myapp

Flashcard Check¶

Q1: Where does Helm store release information?

As Kubernetes Secrets in the release namespace. The Secret contains rendered manifests, values, and chart metadata.

Q2: helm template vs helm install --dry-run — what's the difference?

helm template renders locally (no cluster needed). --dry-run renders and validates against the actual cluster's API (catches missing CRDs, RBAC issues).

Q3: {{ toYaml .Values.resources | nindent 12 }} — what does nindent 12 do?

Adds a newline then indents every line by 12 spaces. Getting this number wrong is the

1 Helm debugging issue — it produces invalid YAML.¶

Q4: helm rollback — does it rollback database migrations?

No. Helm only rolls back Kubernetes resources and its own release record. Database migrations, config changes, and external state are not affected.

Cheat Sheet¶

Task	Command
Install	`helm install NAME ./chart --values values.yaml`
Upgrade	`helm upgrade NAME ./chart --values values.yaml`
Rollback	`helm rollback NAME REVISION`
See rendered YAML	`helm template NAME ./chart`
Dry run	`helm install NAME ./chart --dry-run --debug`
Lint	`helm lint ./chart`
Get deployed values	`helm get values NAME`
Get deployed manifest	`helm get manifest NAME`
Release history	`helm history NAME`
Diff before upgrade	`helm diff upgrade NAME ./chart`
Delete release	`helm uninstall NAME`

Takeaways¶

Helm is template rendering + release tracking. It renders Go templates into YAML, submits to the API server, and stores a release record as a K8s Secret.
helm template is your best friend. See the rendered YAML before installing. Most Helm bugs are visible in the template output.
nindent is the #1 pain point. Go templates and YAML indentation don't mix naturally. Always check rendered output.
Rollback is limited to Kubernetes resources. Database migrations, external config, and state changes are not rolled back by Helm.
Lint and dry-run catch different errors. lint catches template syntax. --dry-run catches API validation (missing CRDs, bad resource specs).

What Happens When You kubectl apply — what happens after Helm submits YAML
Why YAML Keeps Breaking Your Deploys — YAML gotchas in Helm values
The Rollback That Wasn't — why Helm rollback doesn't fix everything
Terraform vs Ansible vs Helm — when to use Helm vs the alternatives