Ops Archaeology: The Container That Exits Immediately¶
You've just joined a team. There are no docs. The previous engineer left last month. Something is broken. Here's everything you have to work with.
Difficulty: L1 Estimated time: 15 min Domains: Docker, Container Images, CI/CD
Artifact 1: CLI Output¶
$ docker ps -a --filter name=pdf-renderer
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a3f8b2c1d4e5 registry.corp.io/pdf-renderer:v3 "/app/render" 12 seconds ago Exited (1) 11 seconds ago pdf-renderer
9c7e6d5f4a3b registry.corp.io/pdf-renderer:v3 "/app/render" 47 seconds ago Exited (1) 46 seconds ago pdf-renderer
f1b2c3d4e5a6 registry.corp.io/pdf-renderer:v3 "/app/render" 1 minute ago Exited (1) 1 minute ago pdf-renderer
e5d4c3b2a1f0 registry.corp.io/pdf-renderer:v3 "/app/render" 2 minutes ago Exited (1) 2 minutes ago pdf-renderer
$ docker logs a3f8b2c1d4e5
exec /app/render: exec format error
$ docker inspect a3f8b2c1d4e5 --format '{{.Config.Image}} | Architecture: {{.Architecture}} | OS: {{.Os}}'
registry.corp.io/pdf-renderer:v3 | Architecture: arm64 | OS: linux
$ uname -m
x86_64
Artifact 2: Metrics¶
# No Prometheus metrics available — the pdf-renderer container never starts
# long enough to serve its /metrics endpoint.
# Docker daemon metrics from the host:
engine_daemon_container_states_containers{state="stopped"} 47
engine_daemon_container_actions_seconds_count{action="start"} 312
# Upstream service queue depth (from a different exporter):
pdf_queue_pending_jobs 1847
pdf_queue_oldest_job_age_seconds 7234
Artifact 3: Infrastructure Code¶
# From: Dockerfile (last modified 3 days ago in git)
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY . .
RUN npm run build
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./
COPY --from=builder /app/node_modules ./node_modules
ENTRYPOINT ["/app/render"]
Artifact 4: Log Lines¶
[2024-11-15T09:12:03Z] docker | exec /app/render: exec format error
[2024-11-15T09:11:58Z] ci-runner | Step 8/8 : Successfully built amd64/linux image pdf-renderer:v3 on runner gitlab-runner-arm64-02
[2024-11-15T09:10:44Z] registry | PUT /v2/pdf-renderer/manifests/v3 digest=sha256:8a3b... size=1842 arch=arm64
Your Mission¶
- Reconstruct: What does this system do? What are its components and purpose?
- Diagnose: What is currently broken or degraded, and why?
- Propose: What would you do to fix it? What would you check first?