Testing Management in CI/CD: GitHub Actions vs GitLab CI vs Jenkins with Real Pipelines

Abstract

An automated test suite is only as good as the system that runs it: if tests execute only when a developer remembers to run them, the contract they protect is enforced only sometimes. This article compares three of the most widely used CI/CD testing management tools — GitHub Actions, GitLab CI/CD and Jenkins — by giving them identical work: executing the same real API test suite (Jest + Supertest, 8 integration tests, 100% coverage, from my previous article) on every push. Instead of comparing marketing pages, we compare the three real pipeline files, side by side, in one public repository. The comparison covers configuration style, hosting model, dependency caching, artifacts and ecosystem, and closes with a practical decision guide. Full demo repo with all three pipelines: https://github.com/GianfrancoArocutipa/ci-testing-comparative.

The experiment: one suite, three pipelines

Comparing CI tools is usually apples to oranges — different projects, different jobs. To make it fair, this repository gives all three tools the exact same job:

1. Check out the code.
2. Install dependencies with npm ci.
3. Run npx jest --coverage --ci (8 API tests against an Express app).
4. Keep the coverage report as an artifact.
5. Fail the build if any test fails — blocking the merge.

The repo contains all three configurations at once, which is itself a real-world pattern: teams migrating between CI providers often run them in parallel for a while.

.github/workflows/ci.yml   → GitHub Actions
.gitlab-ci.yml             → GitLab CI/CD
Jenkinsfile                → Jenkins

Pipeline 1 — GitHub Actions

Configuration lives in .github/workflows/, one YAML file per workflow, triggered by events (push, pull request, schedule, manual dispatch):

name: CI — API Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest        # GitHub-hosted runner, zero setup
    strategy:
      matrix:
        node-version: [20, 22]    # same suite on two Node versions

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: npm              # dependency cache in one line
      - run: npm ci
      - run: npx jest --coverage --ci
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: coverage-node-${{ matrix.node-version }}
          path: coverage/

What stands out: the Marketplace. actions/checkout, actions/setup-node and actions/upload-artifact are reusable building blocks maintained by others — caching node_modules, something that takes real thought elsewhere, is literally cache: npm. The matrix is also the most elegant of the three: two lines and the suite runs on Node 20 and 22 in parallel. Runners are hosted by GitHub with a generous free tier for public repos, and failed checks block the pull request natively.

Pipeline 2 — GitLab CI/CD

One single file at the repo root, organized around stages and a Docker-first philosophy — every job declares the container image it runs in:

image: node:22          # every job runs inside this Docker image

stages:
  - test

cache:                   # cache keyed to the lockfile
  key:
    files:
      - package-lock.json
  paths:
    - .npm/

api-tests:
  stage: test
  script:
    - npm ci --cache .npm --prefer-offline
    - npx jest --coverage --ci
  coverage: '/All files[^|]*\|\s*([\d\.]+)/'   # GitLab parses coverage % from the log
  artifacts:
    when: always
    paths:
      - coverage/
    expire_in: 1 week

What stands out: the coverage: keyword — a regex that extracts the coverage percentage straight from the job log and displays it on every merge request, with no plugin. Artifacts have first-class expiration policies. The Docker-first model makes environments extremely reproducible: "it runs in node:22" is the whole story. The trade-off is that caching is more manual than GitHub's one-liner, and the reusable-component catalog is smaller than GitHub's Marketplace.

Pipeline 3 — Jenkins

The veteran. Configuration is a Jenkinsfile in Groovy, and the fundamental difference is the hosting model: you run the server yourself.

pipeline {
    agent {
        docker { image 'node:22' }   // requires Docker on the Jenkins node
    }

    stages {
        stage('Install') {
            steps { sh 'npm ci' }
        }
        stage('Test') {
            steps { sh 'npx jest --coverage --ci' }
        }
    }

    post {
        always {
            archiveArtifacts artifacts: 'coverage/**', allowEmptyArchive: true
        }
        failure {
            echo 'API contract broken — merge should be blocked.'
        }
    }
}

What stands out: total control. Jenkins runs on your own infrastructure — a hard requirement in banks, hospitals or any environment where code cannot leave the network — and its ~1,900 plugins integrate with practically everything ever built. The post block gives fine-grained control over what happens on success, failure or always. The price is operational: you install it, update it, secure it and scale it. There is no free hosted tier because there is no hosted anything; caching, PR blocking and secrets management all require configuration that the other two give you out of the box.

Side-by-side comparison

	GitHub Actions	GitLab CI/CD	Jenkins
Config	YAML per workflow	Single YAML	Groovy Jenkinsfile
Hosting	GitHub-hosted (or self)	GitLab-hosted (or self)	Self-hosted only
Setup effort	None	None	Server install + maintenance
Dependency cache	cache: npm (1 line)	Manual cache: block	Plugin / manual
Test matrix	Native, 2 lines	parallel:matrix	Plugin / scripted
Coverage on PR/MR	Via actions/artifacts	Native coverage: regex	Plugin (Cobertura, etc.)
Reusability	Marketplace (20k+ actions)	CI/CD components catalog	~1,900 plugins
Free tier	2,000 min/mo (∞ public)	400 compute min/mo	Free software, your hardware
Best fit	Code already on GitHub	GitLab shops, Docker-first teams	On-premise / regulated environments

(Free-tier numbers as of their current published plans — check each vendor's pricing page.)

Other tools in this space follow the same patterns: CircleCI and Travis CI resemble the hosted-YAML model, Bitbucket Pipelines is GitLab-style inside Atlassian, TeamCity is Jenkins-like with a commercial polish, and Tekton brings the Jenkins philosophy natively into Kubernetes.

So which one should you use?

The honest answer from running the same suite on all three: the tool matters less than the gate. All three ran the 8 tests, produced the coverage report and failed correctly when I broke a status code on purpose. The decision is about context, not capability: if your code lives on GitHub, Actions is the frictionless default — the matrix and marketplace saved the most lines in this experiment. If your organization runs GitLab, its CI is deeply integrated and the MR coverage display is genuinely nice. If your constraints say nothing leaves our network, Jenkins remains the standard answer, and its plugin ecosystem still covers cases the hosted tools don't.

Limitations worth knowing

This comparison covers the testing-management core (run suite, cache, artifacts, block merge) on a small Node project; it does not measure large-monorepo performance, deployment features (CD), or enterprise concerns like compliance controls and audit logs, where the tools differ much more. Free-tier limits and pricing also change frequently — verify them before committing a team.

Conclusion

We gave three CI/CD tools identical work — a real API test suite — and compared the actual configuration files instead of feature lists. GitHub Actions won on ergonomics, GitLab CI on integrated reporting, Jenkins on control and sovereignty. Whatever you choose, the destination is the same and non-negotiable: your tests must run on every push, automatically, and a red build must block the merge. Everything else is syntax.

Demo repository (all three pipelines): https://github.com/GianfrancoArocutipa/ci-testing-comparative(#)