Building a CI/CD Pipeline

A Practical Comparison of Jenkins, GitHub Actions, and CircleCI

Continuous Integration and Continuous Deployment (CI/CD) pipelines are now a baseline expectation in professional software development. They automate the repetitive work of building, testing, and deploying code — eliminating the "works on my machine" problem, tightening feedback loops, and reducing the blast radius of any single change.

But the tooling landscape is crowded, and the right choice depends heavily on your team's size, infrastructure preferences, and existing workflow. This guide walks through three of the most widely used options — Jenkins, GitHub Actions, and CircleCI — covering practical setup, real configuration examples, and honest trade-offs for each.

Jenkins: Maximum Flexibility, Maximum Ownership

What It Is

Jenkins has been the workhorse of CI/CD since 2011. It's a self-hosted, open-source automation server with a plugin ecosystem of over 1,800 integrations. Nearly anything you can imagine doing in a pipeline — deploying to Kubernetes, sending Slack alerts, parsing test reports, triggering other jobs — has a Jenkins plugin for it.

The trade-off is that Jenkins requires you to own the infrastructure. You provision the server, manage upgrades, configure agents, and monitor it. For teams with dedicated DevOps capacity or strict data residency requirements, that's acceptable. For smaller teams, it can be a tax.

Installation

Jenkins requires a JDK (Java 11 or 17 recommended) and runs on any major operating system. The quickest way to get started on a Debian/Ubuntu server:

# Install Java
sudo apt update && sudo apt install -y openjdk-17-jdk

# Add the Jenkins package repository and install
curl -fsSL https://pkg.jenkins.io/debian/jenkins.io-2023.key | sudo tee \
  /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian binary/" | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt update && sudo apt install -y jenkins

# Start and enable the service
sudo systemctl start jenkins && sudo systemctl enable jenkins

Jenkins will be available at http://your-server:8080. Retrieve the initial admin password from /var/lib/jenkins/secrets/initialAdminPassword to complete setup.

Defining a Pipeline

Jenkins supports two pipeline syntaxes: Declarative (structured, recommended for most teams) and Scripted (Groovy-based, more powerful but harder to read). Below is a production-style Declarative pipeline for a Maven Java project, including environment variables, parallel test execution, and conditional deployment:

pipeline {
    agent any

    environment {
        DOCKER_IMAGE = "myorg/myapp:${BUILD_NUMBER}"
    }

    stages {
        stage('Build') {
            steps {
                sh 'mvn clean package -DskipTests'
                archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
            }
        }

        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    steps { sh 'mvn test' }
                }
                stage('Integration Tests') {
                    steps { sh 'mvn verify -Pintegration' }
                }
            }
            post {
                always {
                    junit 'target/surefire-reports/*.xml'
                }
            }
        }

        stage('Build Docker Image') {
            steps {
                sh "docker build -t ${DOCKER_IMAGE} ."
                sh "docker push ${DOCKER_IMAGE}"
            }
        }

        stage('Deploy') {
            when {
                branch 'main'
            }
            steps {
                sh "./deploy.sh ${DOCKER_IMAGE}"
            }
        }
    }

    post {
        failure {
            slackSend channel: '#deployments',
                      message: "Build FAILED: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
        }
        success {
            slackSend channel: '#deployments',
                      message: "Build succeeded: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
        }
    }
}

A few things worth noting here. The parallel block in the Test stage runs both test suites simultaneously, cutting total pipeline time. The when { branch 'main' } condition means the Deploy stage only fires on merges to main — feature branches build and test without triggering a deployment. The post block handles notifications regardless of how the pipeline ends.

Connecting to Source Control

Rather than configuring Jenkins to poll your repository (inefficient), set up a webhook so GitHub or GitLab pushes events directly to Jenkins:

1. Install the GitHub plugin in Jenkins (Manage Jenkins → Plugins).
2. In your GitHub repository, go to Settings → Webhooks → Add webhook.
3. Set the Payload URL to http://your-jenkins-server/github-webhook/ and choose "Just the push event."
4. In your Jenkins job, enable "GitHub hook trigger for GITScm polling" under Build Triggers.

Jenkins will now kick off a build within seconds of every push.

When Jenkins Makes Sense

Jenkins is the right call when you need deep customization, have existing infrastructure and DevOps expertise, operate in an air-gapped environment where SaaS tools aren't an option, or are running complex multi-project pipelines where the Scripted DSL's full Groovy power genuinely matters. It's overkill for a small team shipping a web app.

GitHub Actions: Zero-Infrastructure CI/CD for GitHub Teams

What It Is

GitHub Actions is GitHub's native CI/CD platform, launched in 2019. Pipelines are defined as YAML workflow files that live directly in your repository under .github/workflows/. Because GitHub hosts the runner infrastructure, there's nothing to provision or maintain — you commit a file, and your pipeline exists.

The GitHub Marketplace offers thousands of prebuilt Actions for common tasks: checking out code, setting up language runtimes, publishing packages, deploying to cloud providers, sending notifications. Most pipelines are assembled from these building blocks rather than written from scratch.

Workflow Structure

Every workflow file has three core sections: a trigger (on:), one or more jobs, and steps within each job. Here's a complete Node.js workflow with caching, test reporting, and deployment gating:

name: CI/CD

on:
  push:
    branches: [main, "release/**"]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [18, 20]   # Test against multiple Node versions

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: "npm"               # Cache node_modules between runs

      - name: Install dependencies
        run: npm ci

      - name: Lint
        run: npm run lint

      - name: Run tests
        run: npm test -- --coverage

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-node-${{ matrix.node-version }}
          path: coverage/

  deploy:
    runs-on: ubuntu-latest
    needs: test                      # Only run if test job passes
    if: github.ref == 'refs/heads/main'   # Only deploy from main

    environment: production          # Requires manual approval if configured

    steps:
      - uses: actions/checkout@v4

      - name: Deploy to production
        env:
          DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
        run: npm run deploy

The matrix strategy runs the test job twice in parallel — once for Node 18 and once for Node 20 — with a single job definition. The needs: test dependency on the deploy job ensures that deployment never happens unless all matrix variants pass. Secrets are stored in your repository's Settings → Secrets and accessed via ${{ secrets.SECRET_NAME }}, never exposed in logs.

Reusable Workflows

For larger organizations maintaining multiple repositories, duplicating workflow files quickly becomes a maintenance burden. GitHub Actions supports reusable workflows — a way to define a workflow once and call it from other repositories:

# .github/workflows/reusable-deploy.yml (in a central "platform" repo)
on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
      deploy-token:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - run: echo "Deploying to ${{ inputs.environment }}"
      # ... deployment steps

# In any other repository's workflow
jobs:
  call-deploy:
    uses: myorg/platform/.github/workflows/reusable-deploy.yml@main
    with:
      environment: production
    secrets:
      deploy-token: ${{ secrets.DEPLOY_TOKEN }}

When GitHub Actions Makes Sense

If your codebase already lives on GitHub, GitHub Actions is the most natural choice for most teams. Setup is minimal, the YAML syntax is readable, and tight integration with pull requests (surfacing test status directly on PRs, requiring checks to pass before merging) is genuinely useful. Its main constraints are that it's GitHub-only and the free tier's monthly minutes can run out on active projects.

CircleCI: Speed-Optimized Cloud-Native Pipelines

What It Is

CircleCI is a dedicated CI/CD platform with a reputation for fast build times and sophisticated caching. Unlike Jenkins (which you host) or GitHub Actions (which is tied to GitHub), CircleCI connects to GitHub, GitLab, or Bitbucket and can be run either on CircleCI's cloud infrastructure or on self-hosted runners. It occupies a middle ground: managed infrastructure with more portability than GitHub Actions.

CircleCI's configuration lives in .circleci/config.yml at the root of your repository. Its key differentiators are granular caching, an Orbs system (reusable configuration packages), and first-class support for complex job dependency graphs.

Defining a Pipeline

Here's a production-style CircleCI config for a Node.js application, demonstrating Docker executor selection, layer caching, workspace sharing between jobs, and conditional deployment:

version: 2.1

orbs:
  node: circleci/node@5.1.0   # Use the official Node.js orb

jobs:
  build-and-test:
    docker:
      - image: cimg/node:20.10
      - image: cimg/postgres:15.2    # Spin up a test database alongside the app
        environment:
          POSTGRES_USER: testuser
          POSTGRES_DB: testdb

    steps:
      - checkout

      - node/install-packages:       # Orb step with built-in caching
          pkg-manager: npm

      - run:
          name: Run linter
          command: npm run lint

      - run:
          name: Run tests
          command: npm test
          environment:
            DATABASE_URL: postgresql://testuser@localhost/testdb

      - store_test_results:
          path: test-results/        # Enables test insights in CircleCI dashboard

      - store_artifacts:
          path: coverage/

      - persist_to_workspace:        # Share build artifacts with the deploy job
          root: .
          paths: [dist/]

  deploy:
    docker:
      - image: cimg/node:20.10
    steps:
      - attach_workspace:            # Retrieve artifacts from build-and-test
          at: .

      - run:
          name: Deploy to production
          command: npm run deploy

workflows:
  ci-cd:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
          filters:
            branches:
              only: main             # Only deploy from main branch

A few things to highlight here. Spinning up a Postgres service container alongside the application container is a clean pattern for integration testing — CircleCI handles the networking automatically. The persist_to_workspace and attach_workspace steps pass build artifacts (the compiled dist/ folder) from the build job to the deploy job without redundant recompilation. store_test_results parses JUnit XML output and surfaces test trends in the CircleCI dashboard.

Advanced Caching

CircleCI's caching system is more explicit than GitHub Actions' built-in caching, which gives you finer control. You define cache keys based on file hashes, falling back to progressively less-specific keys:

- restore_cache:
    keys:
      - node-deps-v1-{{ checksum "package-lock.json" }}  # Exact match
      - node-deps-v1-                                     # Fallback: any recent cache

- run: npm ci

- save_cache:
    key: node-deps-v1-{{ checksum "package-lock.json" }}
    paths:
      - ~/.npm

If your package-lock.json hasn't changed, the exact-match key hits and npm ci completes in seconds. If it has changed, CircleCI falls back to the most recent partial match, and only the changed packages need to be downloaded.

When CircleCI Makes Sense

CircleCI is a strong choice when you need cross-platform VCS support (GitHub + GitLab + Bitbucket from one tool), want more granular caching control than GitHub Actions provides, or need self-hosted runners without the operational overhead of Jenkins. It's particularly well-suited to data-intensive or compute-heavy pipelines where build time is a genuine bottleneck.

Side-by-Side Comparison

	Jenkins	GitHub Actions	CircleCI
Hosting	Self-managed	GitHub-managed	Cloud or self-hosted
VCS Support	Any	GitHub only	GitHub, GitLab, Bitbucket
Config format	Groovy DSL	YAML	YAML
Setup effort	High	Minimal	Low
Customization	Unlimited	Medium	High
Plugin/Orb ecosystem	1,800+ plugins	Thousands of Actions	Orbs library
Parallelism	Yes (with agents)	Matrix + parallel jobs	Built-in
Caching	Plugin-dependent	Built-in	First-class, explicit
Cost	Free (infrastructure not included)	Free tier with limits	Free tier; paid plans
Best for	Enterprises, complex pipelines, air-gapped environments	GitHub-hosted projects, small to mid-size teams	Speed-sensitive workloads, multi-VCS organizations

Choosing the Right Tool

All three tools can build, test, and deploy virtually any application stack. The decision comes down to context.

If your team lives in GitHub and values low operational overhead, GitHub Actions is the obvious default. You get solid CI/CD for nearly any project with nothing to configure beyond a YAML file.

If your organization has complex infrastructure requirements, heterogeneous tooling, strict compliance constraints, or genuinely complicated pipeline logic, Jenkins remains the most powerful and customizable option — as long as someone is willing to own it.

If you want managed infrastructure with more flexibility than GitHub Actions and you need to support multiple Git hosting providers, CircleCI is worth a close look. Its caching and Orbs system produce reliably fast builds with relatively little configuration.

The pipelines you build with any of these tools are living infrastructure. They'll need to evolve as your system grows. Pick the tool your team will actually maintain — and start simple.