Skip to content

How We Got Here: Artifact Management

Arc: Deployment Eras covered: 6 Timeline: ~2000-2025 Read time: ~11 min

The Original Problem

In 2000, "shipping software" meant putting a binary on an FTP server and emailing the download link. Internal deployments meant SCP'ing a tarball to a production server. There was no versioning beyond filenames (app-final.tar.gz, app-final-v2.tar.gz, app-final-v2-FIXED.tar.gz). There was no cryptographic verification, no dependency resolution, no way to know what was inside the package or what it depended on. If the FTP server died, the deployment pipeline died.

The gap between "code compiles" and "code is reliably delivered to production" was filled with duct tape and tribal knowledge.

Era 1: FTP Servers and Tarballs (~2000-2005)

The Solution

FTP servers (wu-ftpd, ProFTPD, later vsftpd) were the distribution mechanism. Build artifacts were tarballs or zip files, often with a README explaining how to install them. Internal deployments used SCP or NFS mounts. Package managers existed for OS packages (RPM, dpkg) but not for application artifacts.

What It Looked Like

# "Artifact management" circa 2003
tar czf myapp-1.2.3.tar.gz -C build/ .
scp myapp-1.2.3.tar.gz ftp.internal.example.com:/pub/releases/
# Email: "New release on the FTP server. Please deploy."

# On the production server:
cd /opt
wget ftp://ftp.internal.example.com/pub/releases/myapp-1.2.3.tar.gz
tar xzf myapp-1.2.3.tar.gz
ln -sfn myapp-1.2.3 myapp-current
systemctl restart myapp

Why It Was Better

  • Simple and universally understood
  • No special tooling required beyond tar, scp, and an FTP server
  • Worked for any type of artifact (binaries, configs, data)

Why It Wasn't Enough

  • No metadata: what dependencies does this need? What version of Java?
  • No integrity checking: was the file corrupted or tampered with?
  • No dependency resolution: if myapp needs libfoo 2.3, you're on your own
  • No retention policy: FTP servers filled up until someone cleaned them
  • Naming conventions were the only versioning (and they were inconsistent)

Legacy You'll Still See

Tarball-based deployment persists in embedded systems, game servers, and organizations with extremely simple deployment needs. The wget | tar pattern shows up in Dockerfiles and CI scripts regularly. Many internal tools are still distributed this way.

Era 2: Language Package Managers (~2003-2012)

The Solution

Language ecosystems built their own package managers and registries. Maven Central (2003) for Java, PyPI (2003) for Python, RubyGems (2004) for Ruby, npm (2010) for Node.js. Each solved dependency resolution, versioning, and distribution for their language. Nexus (Sonatype, 2008) and Artifactory (JFrog, 2008) emerged as universal repository managers that could proxy and host multiple formats.

What It Looked Like

<!-- Maven pom.xml — dependency management (~2008) -->
<project>
  <dependencies>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-core</artifactId>
      <version>3.0.5.RELEASE</version>
    </dependency>
  </dependencies>

  <distributionManagement>
    <repository>
      <id>internal</id>
      <url>https://nexus.example.com/repository/releases/</url>
    </repository>
  </distributionManagement>
</project>

# Publishing and consuming
mvn deploy                        # publish to Nexus
pip install mylib==1.2.3          # install from PyPI or private index
gem install rails --version 3.2.0 # install from RubyGems
npm install express@4.17.1        # install from npm registry

Why It Was Better

  • Automatic dependency resolution (Maven, pip, npm handle transitive deps)
  • Semantic versioning provided a common language for compatibility
  • Central registries made discovery easy
  • Nexus/Artifactory provided caching, access control, and proxying
  • Reproducible builds became possible with lock files

Why It Wasn't Enough

  • Each language had its own format, tooling, and registry
  • "Dependency hell" replaced manual dependency management
  • Security was an afterthought (typosquatting, malicious packages)
  • Large organizations needed Nexus/Artifactory to control what was allowed
  • Application deployment still required more than just library installation

Legacy You'll Still See

Language package managers are universal and permanent. npm, pip, Maven, and RubyGems are daily tools. Nexus and Artifactory are standard infrastructure at any organization with more than a handful of developers. The dependency management patterns they established are foundational.

Era 3: OS Packages and System Package Managers (~2008-2015)

The Solution

Teams started packaging their applications as OS packages (RPM, DEB) so they could use the same deployment mechanism as system software. Tools like fpm (2011) made it easy to build OS packages from any source. This brought versioning, dependency tracking, and clean install/uninstall to application deployment.

What It Looked Like

# fpm: build an RPM from a directory
fpm -s dir -t rpm \
  --name myapp \
  --version 1.2.3 \
  --depends "java-11-openjdk" \
  --after-install scripts/post-install.sh \
  -C build/ \
  opt/myapp

# Upload to a yum repository
createrepo /var/www/yum-repo/
cp myapp-1.2.3-1.x86_64.rpm /var/www/yum-repo/

# Install on production
yum install myapp-1.2.3
# Rollback
yum downgrade myapp-1.2.2

Why It Was Better

  • Consistent with OS package management (yum/apt workflows)
  • Built-in dependency resolution at the system level
  • Pre/post install scripts handled service management
  • Versioning and rollback through the package manager
  • Repository-based distribution with GPG signing

Why It Wasn't Enough

  • OS-specific: RPMs for RHEL, DEBs for Debian/Ubuntu — doubled the work
  • Build tooling was complex and fragile
  • Package managers weren't designed for frequent application releases
  • No isolation: packages shared the host filesystem and could conflict
  • Containers made OS packages largely unnecessary for application deployment

Legacy You'll Still See

OS packages are still used for system-level software and in organizations that standardized on RPM/DEB workflows before containers. Ansible and Puppet modules that install applications via packages are common. Infrastructure tooling (monitoring agents, log collectors) is often distributed as packages.

Era 4: Container Images and Registries (~2013-2020)

The Solution

Docker images (2013) bundled the application, its dependencies, and its runtime into a single distributable artifact. Docker Hub and private registries (Amazon ECR, Google GCR, Azure ACR, Harbor) provided storage, distribution, and access control. The image tag became the version identifier.

What It Looked Like

# Build and push a container image
docker build -t myapp:1.2.3 .
docker tag myapp:1.2.3 registry.example.com/myapp:1.2.3
docker push registry.example.com/myapp:1.2.3

# Deploy (Kubernetes)
kubectl set image deployment/myapp myapp=registry.example.com/myapp:1.2.3

# Rollback
kubectl rollout undo deployment/myapp

# Amazon ECR
aws ecr create-repository --repository-name myapp
aws ecr get-login-password | docker login --username AWS --password-stdin \
  123456789.dkr.ecr.us-east-1.amazonaws.com
docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/myapp:1.2.3

Why It Was Better

  • Self-contained: image includes everything needed to run
  • Immutable: same image hash = same behavior everywhere
  • Layered: shared base layers reduce storage and transfer time
  • Registry ecosystem: pull policies, vulnerability scanning, replication
  • Universal: one artifact format for any language or runtime

Why It Wasn't Enough

  • Image sizes were often bloated (multi-stage builds helped, but required discipline)
  • Tag mutability (:latest could mean anything) caused production incidents
  • No standard for non-container artifacts (Helm charts, configs, policies)
  • Registry security (who can push? who can pull?) needed careful configuration
  • Vulnerability scanning was reactive, not preventive

Legacy You'll Still See

Container images are the current standard for application artifacts. Docker Hub, ECR, GCR, and ACR are daily infrastructure. The docker build && docker push workflow is universal. Image scanning (Trivy, Snyk) is standard in CI pipelines.

Era 5: OCI Artifacts and Universal Registries (~2020-2023)

The Solution

The OCI (Open Container Initiative) distribution spec was extended to support arbitrary artifact types. Helm charts, WASM modules, policy bundles, and Sigstore signatures could all be stored in the same registry as container images. ORAS (OCI Registry As Storage) provided a CLI for pushing and pulling any artifact type.

What It Looked Like

# Push a Helm chart to an OCI registry
helm push myapp-1.2.3.tgz oci://registry.example.com/charts

# Pull it
helm pull oci://registry.example.com/charts/myapp --version 1.2.3

# Push arbitrary artifacts with ORAS
oras push registry.example.com/configs/myapp:1.2.3 \
  config.yaml:application/vnd.myapp.config.v1+yaml

# Pull
oras pull registry.example.com/configs/myapp:1.2.3

Why It Was Better

  • One registry for everything: images, charts, configs, signatures
  • OCI standard means interoperability across registries
  • Artifact signing and attestation built into the distribution model
  • Reduces infrastructure: no separate Helm repo, no separate config store
  • Content-addressable: SHA256 digest ensures integrity

Why It Wasn't Enough

  • Tooling support was uneven (not all registries supported all artifact types)
  • Developer experience for non-image artifacts was less polished
  • Discoverability was poor (how do you browse a registry of mixed artifacts?)
  • Migration from existing artifact stores was non-trivial

Legacy You'll Still See

OCI artifact registries are becoming the standard. ECR, GCR, ACR, and Harbor all support OCI artifacts. Helm 3 uses OCI registries natively. The convergence is happening, but many organizations still run separate Nexus/Artifactory instances alongside container registries.

Era 6: SBOMs and Supply Chain Metadata (~2022-2025)

The Solution

Software Bill of Materials (SBOM) requirements (US Executive Order 14028, 2021) and supply chain attacks (SolarWinds 2020, Log4Shell 2021) made it clear that knowing what's inside an artifact is as important as the artifact itself. SPDX and CycloneDX became SBOM standards. Sigstore (cosign, Rekor) provided keyless signing. SLSA (Supply chain Levels for Software Artifacts) defined a maturity framework.

What It Looked Like

# Generate an SBOM for a container image
syft registry.example.com/myapp:1.2.3 -o spdx-json > myapp-sbom.json

# Sign the image with cosign (keyless, via OIDC)
cosign sign registry.example.com/myapp:1.2.3

# Attach the SBOM as an attestation
cosign attest --predicate myapp-sbom.json \
  --type spdxjson registry.example.com/myapp:1.2.3

# Verify before deployment
cosign verify registry.example.com/myapp:1.2.3
cosign verify-attestation --type spdxjson registry.example.com/myapp:1.2.3

# Admission controller enforces in the cluster
# Kyverno policy: only deploy signed images with attached SBOMs

Why It Was Better

  • Verifiable provenance: know who built what, when, and from what source
  • Vulnerability tracking: SBOM enables targeted CVE response
  • Compliance: meets regulatory requirements (US EO 14028, EU CRA)
  • Trust chain: signatures verify integrity from build to deployment
  • Automated enforcement: admission controllers reject unsigned artifacts

Why It Wasn't Enough

  • SBOM generation is imperfect (language-specific gaps, build-time vs runtime deps)
  • Two competing standards (SPDX vs CycloneDX) fragment the ecosystem
  • Tooling is immature — SBOM consumption tools lag behind generation tools
  • "SBOM-washing" — generating SBOMs to check a compliance box without using them
  • The operational burden of managing signing keys and verification policies

Legacy You'll Still See

This is the current frontier. SBOM requirements are spreading through government and regulated industries. Sigstore adoption is growing rapidly. Most organizations are in the "we should be doing this" phase rather than "we are doing this well."

Where We Are Now

Container images in OCI registries are the standard artifact format. Language package managers remain essential for development-time dependencies. SBOM and signing are becoming requirements rather than nice-to-haves. The artifact lifecycle — build, sign, scan, store, distribute, verify, deploy — is well-understood but unevenly implemented.

Where It's Going

Artifact management will converge on OCI registries as the universal store. Every artifact type — images, charts, configs, SBOMs, signatures, policy bundles — will be content-addressable and cryptographically verifiable. The "trust on first use" model will be replaced by "verify everything, always." The remaining challenge is making this invisible to developers while mandatory for the platform.

The Pattern

Every generation adds metadata to the artifact: from bare files to versioned packages to layered images to signed, attested, and inventoried artifacts. The artifact becomes more trustworthy as it carries more proof of its provenance and contents.

Key Takeaway for Practitioners

Use container images as your primary artifact format. Tag them with immutable identifiers (SHA or semver, never :latest in production). Start generating SBOMs and signing images now — it's easier to adopt before it's mandatory than after.

Cross-References