Skip to content

Portal | Level: L2: Operations | Topics: GitOps | Domain: DevOps & Tooling

Runbook: ArgoCD Application OutOfSync

Symptoms

  • ArgoCD UI shows application as OutOfSync
  • Health status may be Degraded or Progressing
  • Pods may be scaling up and down (if HPA conflict)
  • argocd app get <app> shows sync status != Synced

Fast Triage (under 2 minutes)

# Check app status
argocd app get <app-name>

# View the diff
argocd app diff <app-name>

# Hard refresh (re-read from Git)
argocd app get <app-name> --hard-refresh

Causes and Fixes

1. HPA vs Git Replicas (Most Common)

Symptoms: spec.replicas in diff. Pods scaling up/down in a loop.

argocd app diff <app-name>
# Shows: spec.replicas: 3 (Git) vs 8 (live)
kubectl get hpa -n <ns>

Fix: Add ignoreDifferences for replicas.

# Immediate
argocd app set <app-name> --ignore-difference \
  '{"group":"apps","kind":"Deployment","jsonPointers":["/spec/replicas"]}'

# Permanent: add to Application spec in Git

2. Server-Side Defaults

Symptoms: Diff shows fields you didn't set (e.g., clusterIP, strategy).

Fix: Add ignoreDifferences for server-set fields, or explicitly set them in Git.

3. Mutating Webhook Added Fields

Symptoms: Extra annotations, labels, or sidecars in the diff.

Fix: ignoreDifferences with jqPathExpressions for webhook-injected fields.

4. Manual Change Detected (Self-Heal Conflict)

Symptoms: Someone ran kubectl edit or kubectl scale and ArgoCD reverts it.

# Check who made the change
kubectl get events -n <ns> --sort-by=.lastTimestamp

Fix: Make the change in Git instead. ArgoCD is the source of truth.

5. Sync Failed

argocd app sync <app-name>
# Check sync result
argocd app get <app-name> -o json | jq '.status.operationState'

Fix: Check error message, fix manifest in Git, re-sync.

Verification

argocd app get <app-name>
# Status: Synced, Health: Healthy

kubectl get pods -n <ns>
# All pods Running, stable (not restarting)

Prevention

  • Remove spec.replicas from Deployments that use HPA
  • Use ignoreDifferences for known server-set fields
  • Test sync in staging before production
  • Use sync waves for ordering dependencies

Wiki Navigation

Related Content