Remediation: Container Image Vuln Scanner False Positive, Blocks Deploy Pipeline¶

Immediate Fix (Kubernetes Ops — Domain C)¶

The fix involves refreshing the base image cache, rebuilding, and updating the scanning pipeline.

Step 1: Refresh the base image cache¶

# Pull the latest base image from Docker Hub
$ docker pull python:3.11-slim-bookworm
# Push to internal cache
$ docker tag python:3.11-slim-bookworm registry.internal:5000/base-images/python:3.11-slim-bookworm
$ docker push registry.internal:5000/base-images/python:3.11-slim-bookworm

Step 2: Rebuild with fresh base image¶

# Force a rebuild without cache
$ docker build --no-cache --pull -t registry.internal:5000/notification-service:v3.8.1 .
$ docker push registry.internal:5000/notification-service:v3.8.1

Step 3: Re-scan the rebuilt image¶

$ trivy image registry.internal:5000/notification-service:v3.8.1
registry.internal:5000/notification-service:v3.8.1 (debian 12.4)

Total: 0 (CRITICAL: 0)

Clean scan. The fixed version is now correctly detected.

Step 4: Set up automated base image refresh¶

Create a Kubernetes CronJob to refresh base images weekly:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: refresh-base-images
  namespace: ci
spec:
  schedule: "0 4 * * 1"  # Every Monday at 4 AM
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: refresh
            image: docker:24-cli
            command:
            - /bin/sh
            - -c
            - |
              for img in python:3.11-slim-bookworm node:20-slim golang:1.22-bookworm; do
                docker pull $img
                docker tag $img registry.internal:5000/base-images/$img
                docker push registry.internal:5000/base-images/$img
              done
          restartPolicy: OnFailure

Step 5: Deploy the blocked release¶

# Re-run the CI pipeline — scan should pass now
$ gh workflow run ci.yml --ref v3.8.1

Verification¶

Domain A (Security) — Vulnerability resolved¶

$ trivy image --severity CRITICAL registry.internal:5000/notification-service:v3.8.1
Total: 0 (CRITICAL: 0)

Domain B (DevOps Tooling) — Pipeline unblocked¶

$ gh run list --workflow=ci.yml --limit=3
STATUS  TITLE                           BRANCH  EVENT
✓       Deploy notification-service     v3.8.1  push
✗       Deploy notification-service     v3.8.1  push    # (old failure)

Domain C (Kubernetes) — CronJob scheduled¶

$ kubectl get cronjob refresh-base-images -n ci
NAME                   SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE
refresh-base-images    0 4 * * 1     False     0        <none>

Prevention¶

Monitoring: Add a base image age alert that fires when any cached base image is older than 14 days.
Runbook: When a scanner blocks a deploy with a CRITICAL CVE, verify the finding before adding exceptions. Check: (1) Is the installed version actually vulnerable? (2) Is the vulnerability exploitable in this context? (3) Is the scanner reading the correct package version?
Architecture: Use multi-stage Docker builds with --squash or combine the package install and upgrade in a single layer to avoid dpkg status file conflicts. Consider switching to distroless base images where the package database is minimal. Add .trivyignore for confirmed false positives with expiration dates.