Skip to content

ArgoCD & GitOps - Primer

Why This Matters

GitOps is the operational model where Git is the single source of truth for infrastructure and application configuration. ArgoCD and Flux are the two dominant GitOps controllers for Kubernetes. Understanding GitOps is essential for production Kubernetes - it gives you audit trails, rollback, and drift detection for free.

Core Principles

Name origin: The term "GitOps" was coined by Alexis Richardson, CEO of Weaveworks, in a March 2017 blog post describing how his team was using Git as the single source of truth for Kubernetes deployments. The methodology was initially called "Operations by Pull Request." Weaveworks created Flux (the first GitOps controller) and helped establish the OpenGitOps project under the CNCF to formalize the four GitOps principles.

The Four GitOps Principles

  1. Declarative - The entire system is described declaratively (YAML/Helm/Kustomize)
  2. Versioned and immutable - The desired state is stored in Git with full history
  3. Pulled automatically - Agents pull desired state from Git (not pushed by CI)
  4. Continuously reconciled - Agents detect and correct drift automatically

Push vs Pull Deployment

PUSH (traditional CI/CD):
  CI builds image -> CI runs kubectl apply -> cluster changes

PULL (GitOps):
  CI builds image -> CI commits manifest change to Git -> ArgoCD detects change -> ArgoCD applies to cluster

Why pull is better: The cluster is self-healing. If someone manually changes something, ArgoCD reverts it. CI never needs cluster credentials.

ArgoCD Architecture

[Git Repo] <--- polls --- [ArgoCD Server]
                                |
                          [Application Controller]
                                |
                          [Repo Server]  (renders manifests)
                                |
                          [kubectl apply] -> [Kubernetes Cluster]

Key Components

Component Role
API Server Web UI + gRPC/REST API
Application Controller Watches apps, compares desired vs live state
Repo Server Clones repos, renders Helm/Kustomize templates
Redis Caching layer
Dex (optional) SSO/OIDC authentication

Who made it: ArgoCD was created at Intuit (the TurboTax company) and open-sourced in 2018. It grew from the Argo Workflows project, which originated at Applatix (acquired by Intuit). ArgoCD became a CNCF incubation project and is now one of the most widely adopted GitOps tools for Kubernetes, with a rich web UI that distinguishes it from the CLI-focused Flux.

Core Concepts

Application

An ArgoCD Application connects a Git source to a cluster destination:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: grokdevops
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/grokdevops.git
    targetRevision: main
    path: devops/k8s
  destination:
    server: https://kubernetes.default.svc
    namespace: grokdevops
  syncPolicy:
    automated:
      prune: true        # Delete resources removed from Git
      selfHeal: true     # Revert manual changes
    syncOptions:
      - CreateNamespace=true

Sync Status

Status Meaning
Synced Live state matches Git
OutOfSync Live state differs from Git
Unknown ArgoCD cannot determine state

Health Status

Status Meaning
Healthy All resources are healthy
Progressing Resources are still rolling out
Degraded A resource has failed (e.g., CrashLoopBackOff)
Missing Resource exists in Git but not in cluster

Sync Policies

syncPolicy:
  automated:
    prune: true       # Remove resources deleted from Git
    selfHeal: true    # Revert manual kubectl edits
  retry:
    limit: 5
    backoff:
      duration: 5s
      factor: 2
      maxDuration: 3m

Source Types

ArgoCD supports multiple manifest formats:

Source When to use
Plain YAML Simple apps, few resources
Helm Apps with values files, shared charts
Kustomize Environment overlays (dev/staging/prod)
Jsonnet Programmatic manifest generation
Directory Mix of YAML files in a directory

Helm Source Example

source:
  repoURL: https://charts.example.com
  chart: my-app
  targetRevision: 1.2.3
  helm:
    valueFiles:
      - values-prod.yaml
    parameters:
      - name: image.tag
        value: "v2.1.0"

Kustomize Source Example

source:
  repoURL: https://github.com/your-org/config.git
  targetRevision: main
  path: overlays/production

App-of-Apps Pattern

Manage many ArgoCD Applications with a single parent Application:

apps/
  grokdevops.yaml     # ArgoCD Application for grokdevops
  monitoring.yaml     # ArgoCD Application for prometheus
  logging.yaml        # ArgoCD Application for loki

# Parent application that manages all child apps
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: platform-apps
  namespace: argocd
spec:
  source:
    repoURL: https://github.com/your-org/platform.git
    path: apps
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd

ApplicationSet

Dynamically generate Applications from templates:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-apps
  namespace: argocd
spec:
  generators:
    - list:
        elements:
          - cluster: dev
            url: https://dev-cluster.example.com
          - cluster: prod
            url: https://prod-cluster.example.com
  template:
    metadata:
      name: 'grokdevops-{{cluster}}'
    spec:
      source:
        repoURL: https://github.com/your-org/grokdevops.git
        path: 'overlays/{{cluster}}'
      destination:
        server: '{{url}}'
        namespace: grokdevops

Image Updater

ArgoCD Image Updater watches container registries and automatically updates image tags in Git:

# Annotate the Application
metadata:
  annotations:
    argocd-image-updater.argoproj.io/image-list: app=ghcr.io/your-org/grokdevops
    argocd-image-updater.argoproj.io/app.update-strategy: semver
    argocd-image-updater.argoproj.io/write-back-method: git

Gotcha: ArgoCD's targetRevision: main (or HEAD) means every commit to main triggers a sync. This is intentional for GitOps, but it means a broken manifest merged to main immediately affects the cluster. Protect your config repo's main branch with required reviews and CI validation (helm template, kustomize build, kubeval) before merge. ArgoCD will faithfully apply whatever is in Git — including your mistakes.

Drift Detection

ArgoCD continuously compares live state to desired state. When drift is detected:

  1. UI shows the resource as OutOfSync with a diff view
  2. If selfHeal: true, ArgoCD automatically reverts the change
  3. If manual sync, an operator must click "Sync" in the UI or run argocd app sync
# Check for drift
argocd app diff grokdevops

# View detailed diff
argocd app diff grokdevops --local devops/k8s/

Flux vs ArgoCD

Aspect ArgoCD Flux
UI Rich web UI No built-in UI (use Weave GitOps)
CRDs Application, AppProject GitRepository, Kustomization, HelmRelease
Multi-cluster Built-in Via Kustomization targeting
Image automation Separate Image Updater Built-in (image-reflector + image-automation)
Notifications ArgoCD Notifications Flux notification-controller
Complexity More moving parts, more features Simpler, more modular

Default trap: ArgoCD defaults to syncPolicy: {} (manual sync, no prune, no self-heal). This means drift is detected but not corrected — someone can kubectl edit a Deployment and the change persists until a manual sync. For production GitOps, always set automated.selfHeal: true and automated.prune: true. Without these, you have "Git as audit log" but not true GitOps reconciliation.

War story: A team enabled prune: true on their ArgoCD Application, then accidentally deleted a directory from their Git repo during a refactor. ArgoCD dutifully pruned all the Kubernetes resources in that directory — including a production database StatefulSet. The fix: use ArgoCD resource annotations (argocd.argoproj.io/sync-options: Prune=false) on critical resources that should never be auto-deleted, even if removed from Git.

Remember: The four GitOps principles mnemonic: DVPC — Declarative, Versioned, Pulled, Continuously reconciled. If any one is missing, it is not true GitOps. CI pushing kubectl apply is declarative and versioned but not pulled or continuously reconciled — it is "CI/CD with Git" but not GitOps.

Interview tip: "What is the difference between GitOps and CI/CD?" is an increasingly common interview question. The key answer: in CI/CD, the pipeline pushes changes to the cluster (push model). In GitOps, an agent in the cluster pulls desired state from Git and continuously reconciles (pull model). The agent detects and corrects drift, CI never needs cluster credentials, and Git history becomes the audit log.

Common Pitfalls

  1. Sync loops - Two tools fighting over the same resource (e.g., HPA and ArgoCD both setting replicas)
  2. Secret management - Don't store plaintext secrets in Git. Use Sealed Secrets, SOPS, or External Secrets
  3. Prune without thinking - prune: true will delete resources you remove from Git. Be careful with namespace-scoped deletes
  4. Helm hook ordering - ArgoCD processes Helm hooks differently than helm install. Test thoroughly
  5. Large repos - ArgoCD clones the entire repo. Keep manifest repos small and focused

Wiki Navigation

Related Content