The Quest Begins (The "Why")
Honestly, I used to stare at my CI/CD config files like they were ancient runes. I’d push a commit, wait for the green check, and then—bam—something would break in staging because the pipeline skipped a test or deployed the wrong artifact. It felt like I was constantly fighting a hydra: cut one head, two more grew. I kept thinking, “There has to be a better way.” After a particularly painful midnight deployment that took down our checkout flow for twenty minutes (yeah, I still hear the pings), I decided to treat CI/CD not as a checklist but as a quest. My goal? Get a pipeline that runs fast, fails early, and tells me exactly what went wrong—no guesswork, no heroics required.
The Revelation (The Insight)
The big “aha!” moment came when I stopped treating each CI system as a black box and started looking at the pipeline as a series of pure functions: checkout → build → test → publish → deploy. If each step is idempotent and fails fast, the whole chain becomes reliable. I also realized that most of my headaches came from hidden state—environment variables leaking between jobs, caches that never got invalidated, or Docker layers that were rebuilt every run. Once I isolated those, the pipelines started behaving like a well‑rehearsed orchestra instead of a jam session.
Here’s the secret sauce I now swear by:
- Cache intelligently – only cache dependencies, never build artifacts.
- Use matrix builds sparingly – they’re great for version testing, but they explode runtime if you’re not careful.
- Fail fast with explicit exit codes – don’t let a step silently continue; make the pipeline stop the moment something’s wrong.
- Publish artifacts as the single source of truth – the deploy job should only pull from a vetted artifact, never rebuild.
When I applied these rules across GitHub Actions, GitLab CI, and Jenkins, the pipelines went from “sometimes works” to “works every time, and I can trust it.” It felt like finally beating the final boss in Zelda—the relief was real, and the loot (peace of mind) was worth every grind.
Wielding the Power (Code & Examples)
GitHub Actions – The Struggle
Before, my workflow looked like this:
name: CI
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install deps
run: npm ci
- name: Build
run: npm run build
- name: Test
run: npm test
- name: Deploy
if: github.ref == 'refs/heads/main'
run: |
aws s3 sync ./dist s3://my-bucket/
The problem? If npm test flaked, the job would still push the dist folder because the Deploy step only checked the branch, not the test outcome. Plus, every run rebuilt node_modules from scratch—slow and wasteful.
GitHub Actions – The Victory
name: CI/CD
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build-test:
runs-on: ubuntu-latest
cache:
paths:
- ~/.npm # cache only the npm cache, not node_modules
steps:
- uses: actions/checkout@v3
- name: Cache Node modules
uses: actions/cache@v3
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
- name: Install deps
run: npm ci
- name: Build
run: npm run build
- name: Test
run: npm test # any non‑zero exit fails the job instantly
- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: dist
path: dist
deploy:
needs: build-test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v3
- name: Download artifact
uses: actions/download-artifact@v3
with:
name: dist
path: dist
- name: Deploy to S3
run: |
aws s3 sync ./dist s3://my-bucket/
What changed? We cache only the npm cache, we fail fast on npm test, and we publish the built dist as an artifact. The deploy job can’t run unless the build-test job succeeded, and it pulls the exact artifact that passed tests—no guesswork, no rebuilds.
GitLab CI – The Struggle
My old .gitlab-ci.yml:
stages:
- build
- test
- deploy
build:
stage: build
script:
- npm ci
- npm run build
test:
stage: test
script:
- npm test
deploy:
stage: deploy
script:
- aws s3 sync ./dist s3://my-bucket/
only:
- main
Again, if the test job failed, the deploy stage would still run because GitLab only skips stages when the previous stage fails, not when a job within a stage fails. Plus, each job rebuilt node_modules.
GitLab CI – The Victory
stages:
- build
- test
- deploy
variables:
NODE_CACHE: "$CI_PROJECT_DIR/.npm"
cache:
key: "${CI_JOB_NAME}"
paths:
- $NODE_CACHE
build:
stage: build
script:
- npm ci
- npm run build
artifacts:
paths:
- dist/
expire_in: 1 hour
test:
stage: test
script:
- npm test
dependencies:
- build
deploy:
stage: deploy
script:
- aws s3 sync ./dist s3://my-bucket/
only:
- main
dependencies:
- test
Here, artifacts pass the dist folder from build to test and then to deploy. If test fails, the pipeline stops before deploy. The cache is limited to the npm directory, keeping builds snappy.
Jenkins – The Struggle
A classic Jenkinsfile I inherited:
pipeline {
agent any
stages {
stage('Build') {
steps {
sh 'npm ci'
sh 'npm run build'
}
}
stage('Test') {
steps {
sh 'npm test'
}
}
stage('Deploy') {
when { branch 'main' }
steps {
sh 'aws s3 sync ./dist s3://my-bucket/'
}
}
}
}
If the Test stage failed, Jenkins would still mark the build as unstable but would continue to the Deploy stage unless I explicitly added a catchError. Plus, each stage checked out the repo again, wasting time.
Jenkins – The Victory
pipeline {
agent any
options {
timeout(time: 20, unit: 'MINUTES')
timestamps()
}
environment {
NODE_CACHE = "${WORKSPACE}/.npm"
}
stages {
stage('Checkout') {
steps {
checkout scm
cache(buffer: true, key: "npm-${env.JOB_NAME}-${env.BUILD_NUMBER}", path: "${NODE_CACHE}")
}
}
stage('Build') {
steps {
sh 'npm ci'
sh 'npm run build'
archiveArtifacts artifacts: 'dist/**', fingerprint: true
}
}
stage('Test') {
steps {
sh 'npm test'
}
post {
failure {
error 'Tests failed – aborting pipeline'
}
}
}
stage('Deploy') {
when { branch 'main' }
steps {
// retrieve the exact artifact that passed tests
unarchive mapping: ['dist/**': '.']
sh 'aws s3 sync ./dist s3://my-bucket/'
}
}
}
}
Key changes: we checkout once, cache only the npm directory, archive the build output, and fail fast with an explicit error step when tests fail. The deploy stage consumes the archived artifact, guaranteeing it’s the same code that passed tests.
Why This New Power Matters
Now, when I push a change, I get reliable feedback in under five minutes. If something’s broken, I know exactly where—no more sifting through logs wondering if the failure was due to a flaky test or a missing environment variable. My team can merge with confidence, and our release frequency went from “once a week, if we’re lucky” to “multiple times a day.” The pipeline isn’t a mystical gatekeeper anymore; it’s a trusty sidekick that shouts “hey, fix this!” before the problem reaches production.
Give it a try: take one of your existing pipelines, isolate the cache, publish an artifact, and make every stage depend on the previous one’s success. Watch the noise drop and the signal rise. You’ll feel like you’ve leveled up your dev superpower—no cape required.
Your turn: Pick a repo you’ve been avoiding because its CI is flaky, apply the “cache → build → test → publish → deploy” pattern, and share the results in the comments. Let’s see whose pipeline goes from “meh” to “ Jedi‑level ” first! 🚀
Top comments (0)