Skip to content

The CI/CD Pipeline Rewrite

Category: The Migration Domains: ci-cd, jenkins Read time: ~5 min

Setting the Scene

I was the build engineer (fancy title for "the person Jenkins calls at 2 AM") at a company with 35 development teams. We had a Jenkins cluster — 1 controller, 14 permanent agents, and an autoscaling fleet of EC2 spot instances. Over five years, we'd accumulated 214 Jenkinsfiles. Each one was a unique snowflake. Some used declarative pipeline syntax, some used scripted, and about 30 used a shared library that three people understood and two of them had left the company.

The mandate: migrate to GitHub Actions. Deadline: end of quarter. Simple, right?

What Happened

Week 1 — I exported every Jenkinsfile and categorized them. The breakdown: 80 were simple build-test-deploy pipelines. 60 used the shared library for Docker builds with custom caching. 40 had complex conditional logic (when blocks, input steps, parallel stages with fan-out/fan-in). 20 were release pipelines with manual approval gates. 14 were... things. One generated a PDF report by SSHing into a server and running LaTeX.

Week 2-3 — I wrote a GitHub Actions workflow template for the 80 simple pipelines. Find-and-replace on the Jenkinsfile patterns, map agent { docker } to runs-on: ubuntu-latest with container steps, swap sh for run. Got 65 of them converted in two weeks. The remaining 15 of the "simple" ones had hardcoded references to Jenkins agent paths like /home/jenkins/tools/maven-3.8.6/bin/mvn.

Week 4-5 — The shared library pipelines. The library had 2,300 lines of Groovy that did Docker layer caching by mounting the Jenkins agent's Docker socket and maintaining a local registry. GitHub Actions has no persistent agent, no docker socket to mount, and its docker/build-push-action caching uses GitHub's cache API. I had to rewrite the caching strategy entirely. Spent two weeks building a reusable composite action that used docker buildx with GitHub Container Registry for layer caching. It was better than the Jenkins version but it was a rewrite, not a migration.

Week 6-7 — The conditional pipelines. Jenkins input steps (manual approval gates) have no GitHub Actions equivalent. We used environments with required reviewers, which worked but changed the UX. Teams complained that they couldn't approve from Slack anymore because the Jenkins Slack integration had a button and GitHub's didn't. I wrote a GitHub App that posted approval links to Slack. That was three days I didn't plan for.

Week 8 — The LaTeX PDF pipeline. I stared at it for 20 minutes, then wrote a GitHub Actions workflow from scratch that used a texlive container. It took 45 minutes and was 30 lines instead of 200. Sometimes the best migration is a rewrite.

Week 9-10 — Testing. Every team had to validate their migrated pipeline. 28 of the 35 teams found at least one issue. Most common: environment variable differences (JENKINS_HOME vs GITHUB_WORKSPACE), secret injection patterns (credentials() vs secrets.*), and artifact handling (archiveArtifacts vs actions/upload-artifact).

The Moment of Truth

Week 5, realizing that the shared library — which was supposed to save time — was actually the biggest migration obstacle. It had encoded Jenkins-specific assumptions into every pipeline that used it. The abstraction layer that made Jenkins "easy" made leaving Jenkins hard. Vendor lock-in isn't just cloud providers; it's your own tooling choices.

The Aftermath

We finished in 12 weeks, not 10. GitHub Actions runners were faster than our Jenkins agents because we weren't managing agent state. Build times dropped 25% on average. But the real win was that every pipeline was now visible in the repo alongside the code, not hidden in a Jenkins shared library that three people could debug. Six months later, a new engineer could understand and modify any pipeline. That had never been true with Jenkins.

The Lessons

  1. Standardize before you migrate: The migration would have been half the effort if our Jenkinsfiles had followed a consistent pattern. Standardize the old system first, then migrate the standard.
  2. Migration is a chance to simplify: The LaTeX pipeline went from 200 lines to 30. Don't port complexity — question it. If nobody can explain why a pipeline step exists, maybe it shouldn't.
  3. Don't port bad patterns to new platforms: The Docker socket mounting hack was clever in Jenkins and would have been a security nightmare in GitHub Actions. Take the migration as permission to do it right.

What I'd Do Differently

I'd spend week 1 killing the shared library. Replace it with simple, inlined pipeline steps in every Jenkinsfile. Yes, it's duplication. But duplication you can see is better than abstraction you can't debug. Once every Jenkinsfile is self-contained, the migration becomes mechanical. I'd also build a pipeline compatibility test harness — run the same commit through both Jenkins and GitHub Actions and diff the outputs.

The Quote

"The shared library that made Jenkins easy also made leaving Jenkins impossible."

Cross-References