GitHub Actions vs. Jenkins vs. CircleCI: How FreeAgent Cut Its Test Infrastructure Spend by 90%

GitHub Actions vs. Jenkins vs. CircleCI: How FreeAgent Cut Its Test Infrastructure Spend by 90%

FreeAgent, the accounting software company, moved its CI/CD workflows off Jenkins and onto GitHub Actions, and reported a 90% reduction in its monthly infrastructure spend for running tests. Nylas made the same Jenkins-to-Actions move and cut its deployment time by 30%, then started seeing whole weeks with zero build failures. Neither team changed what they were building. They changed the machine that builds it, and the difference showed up on the invoice and in the engineers' week.

Then there's the other side of choosing a CI platform, the one nobody puts in a benchmark. In January 2023, CircleCI told every customer to rotate every secret stored on the platform, after attackers got access to customer secrets. Open-source project conda-forge, which had stopped using CircleCI but never removed its tokens, ended up with an outage of about five days while it rotated credentials across all its CI services. A CI tool isn't just a build runner. It's a place where your most sensitive keys live, which makes this choice a security decision as much as a speed decision.

This guide compares GitHub Actions, Jenkins, and CircleCI honestly: what each one actually is, where the real numbers come from, what each one costs you beyond the invoice, and how to pick one without cargo-culting whatever your last employer used. If CI/CD concepts are still new, start with our CI/CD Pipelines roadmap, then come back here.

This post originally appeared on the Ciphemic Academia blog.

The Short Version

• GitHub Actions is CI/CD built into GitHub itself, configured in YAML files inside your repository, with a huge marketplace of reusable actions.
• Jenkins is a self-hosted, open-source automation server with a massive plugin ecosystem and total control, plus all the maintenance that comes with running it yourself.
• CircleCI is a dedicated, managed CI/CD service known for fast pipelines and strong configuration tooling, separate from wherever your code lives.

If you want one default: if your code is on GitHub, learn GitHub Actions first. It has the lowest setup cost, and the concepts (triggers, jobs, steps, runners, secrets) carry straight over to the other two. Reach for Jenkins when you need full control, self-hosting, or you're inheriting it. Reach for CircleCI when you want a dedicated managed CI service and accept that your secrets live with a third party.

What All Three Are Actually Trying to Solve

Before the differences, the shared job, since this is what you're really learning:

• Running your tests automatically on every push or pull request, so broken code gets caught before it merges
• Building your application the same way every time, removing "works on my machine" from the conversation
• Deploying through a repeatable process instead of a person remembering the right commands in the right order
• Handling secrets such as API keys and cloud credentials without hardcoding them in the repository
• Giving fast, visible feedback so a developer knows within minutes whether their change is safe

If these ideas are new, our post on CI/CD pipelines vs. manual deployment covers why this automation exists at all. Learn these ideas in any one tool and most of the skill transfers to the others.

GitHub Actions

What it actually is: a CI/CD system built directly into GitHub. You describe a workflow in a YAML file stored in .github/workflows/, and GitHub runs it on its own hosted machines (called runners) whenever an event you choose happens, such as a push or a pull request. You can also run jobs on your own self-hosted runners.

A small example (test a Node.js project on every push):

name: CI
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm test

Where it shines:

• Zero extra infrastructure: if your code already lives on GitHub, there's nothing to install, host, or sign up for separately
• Readable configuration: FreeAgent's engineers specifically cited that YAML workflows were easier for their developers to learn than Jenkins pipelines written in Groovy
• Huge reusable ecosystem: the Actions marketplace covers most common tasks without writing scripts from scratch, and GitHub reports its platform runs roughly 71 million jobs a day
• Tight GitHub integration: pull request checks, status badges, code scanning, and deployments all connect natively
• Free for public repositories: open-source projects get CI at no cost, which is why so many of them use it

Where it struggles:

• Tied to GitHub: your pipelines only make sense if your code lives there, which makes switching platforms later a real migration
• Third-party actions are code you run with your secrets: every action you pull from the marketplace executes inside your workflow, so pinning versions and reviewing what you use matters
• Pricing has been in flux: GitHub cut hosted-runner prices by up to 39% in January 2026, and announced then postponed a per-minute fee for self-hosted runners, so check the current pricing page before budgeting

Who this suits: teams and learners with code on GitHub, open-source projects, and anyone who wants the shortest path from zero to a working pipeline.

Jenkins

What it actually is: an open-source automation server you install and run yourself, on your own servers or cloud machines. Pipelines are typically written as a Jenkinsfile in Groovy, and Jenkins extends through plugins for nearly everything: source control systems, build tools, cloud providers, notifications.

A small example (the same Node.js pipeline, as a Jenkinsfile):

pipeline {
  agent any
  stages {
    stage('Install') {
      steps { sh 'npm ci' }
    }
    stage('Test') {
      steps { sh 'npm test' }
    }
  }
}

Where it shines:

• Total control: you decide where it runs, how it scales, what network it can reach, and who can access it, which matters in regulated or air-gapped environments
• Works with almost anything: it isn't tied to any one code host, and its plugin ecosystem is among the largest in the CI world
• No per-minute platform fee: the software is free, so your cost is the machines and the people who run them
• Deep customization: complex, unusual build flows that hosted services struggle with are often possible in Jenkins

Where it struggles:

• You own the maintenance: FreeAgent's write-up describes manually spotting problem build agents from logs and removing them by hand, and keeping a wiki of manual changes in case they needed to roll back
• Plugins are a double-edged sword: every plugin is something to update, secure, and keep compatible with the others
• Groovy pipelines have a steeper learning curve than plain YAML for many developers
• Infrastructure cost is real: FreeAgent's 90% drop in test infrastructure spend after leaving Jenkins is a reminder that always-on servers and build agents cost money, even when the software is free

Who this suits: organizations needing self-hosting or full control, teams with existing Jenkins investment and the people to run it, and anyone working in environments where SaaS CI isn't allowed.

CircleCI

What it actually is: a managed CI/CD platform that connects to your code host (such as GitHub or Bitbucket) and runs your pipelines on its infrastructure. You configure it in a YAML file at .circleci/config.yml, and reusable packages of configuration called orbs let you share setup across projects.

A small example (the same pipeline, as a CircleCI config):

version: 2.1

jobs:
  test:
    docker:
      - image: cimg/node:20.11
    steps:
      - checkout
      - run: npm ci
      - run: npm test

workflows:
  main:
    jobs:
      - test

Where it shines:

• Built for pipeline speed: features like caching, parallelism, and test splitting are core to the product, not afterthoughts
• Dedicated CI focus: it does CI/CD and little else, with tooling like SSH access into failed builds for debugging
• Reusable configuration: orbs let teams share tested pipeline building blocks instead of copy-pasting YAML
• Works across code hosts: useful when your repositories aren't all in one place

Where it struggles:

• Your secrets live with a third party: the January 2023 incident showed what that means in practice, with customers told to rotate every secret, including cloud credentials and tokens stored on the platform
• Another vendor and another bill: unlike Actions, it's a separate service to sign up for, secure, and budget
• Smaller ecosystem than Actions: fewer ready-made integrations than GitHub's marketplace for GitHub-centric teams
• Lingering trust questions after the incident: CircleCI published a detailed incident report and says customers can safely build on the platform, but each team has to decide for itself how much it weighs that history

Who this suits: teams that want a dedicated managed CI service, care about pipeline speed features, and have a clear policy for how secrets are stored and rotated.

Side-by-Side Comparison

	GitHub Actions	Jenkins	CircleCI
Hosting	Managed by GitHub (self-hosted runners optional)	Self-hosted by you	Managed by CircleCI
Config format	YAML in .github/workflows/	Jenkinsfile (Groovy)	YAML in .circleci/config.yml
Setup effort	Lowest, if code is on GitHub	Highest, you install and run it	Low to moderate
Ecosystem	Actions marketplace	Very large plugin ecosystem	Orbs
Maintenance burden	Low	High	Low
Cost model	Free for public repos, usage-based for private	Free software, you pay for servers and people	Paid plans beyond a free tier
Secrets live	With GitHub	On your own infrastructure	With CircleCI
Best for	GitHub-based teams, open source	Control, on-prem, regulated environments	Teams wanting dedicated managed CI

How This Fits a Career Path

• DevOps or platform engineer: you'll likely meet all three, so learn the shared concepts first; our CI/CD Pipelines roadmap is built around those fundamentals
• Backend or full-stack engineer: GitHub Actions is usually the first one you'll touch, since it sits right next to your code
• Engineer at a large or regulated company: expect Jenkins, either as legacy you maintain or as a deliberate self-hosting choice
• DevSecOps roles: how secrets are stored, scoped, and rotated across CI tools is central, and the CircleCI incident is a standard case study; see our DevSecOps career guide for how this fits into security work
• System design interviews: be ready to explain how you'd design a pipeline and justify the platform, including where secrets live and what happens if the CI provider is compromised

How to Choose Without Overthinking It

1. Start with where your code lives. On GitHub? Actions is the path of least resistance. Spread across hosts, or on-premises? Jenkins or CircleCI become more attractive.
2. Count the people, not just the dollars. Jenkins is free software, but someone has to patch it, scale its agents, and babysit its plugins. FreeAgent's savings came from removing exactly that overhead.
3. Decide where secrets are allowed to live. If your policy says credentials must never sit on a third-party SaaS, that rules out hosted CI for sensitive deployments and pushes you to self-hosted runners or Jenkins.
4. Use short-lived credentials where you can. Whatever platform you pick, prefer short-lived, narrowly scoped tokens over long-lived keys pasted into a settings page, so a platform incident has a smaller blast radius.
5. Don't wait for the perfect choice. The core ideas transfer, so building one real pipeline in any tool beats comparing all three for a month.

A note on honesty: FreeAgent's 90% and Nylas's 30% are real results from real teams, but they come from specific setups, with specific starting points (FreeAgent's earlier Jenkins setup had its own spot-instance and agent-management overhead). They show what's possible, not what you'll get. GitHub Actions pricing is also actively changing, so check current numbers on GitHub's own pricing page before you budget around anything in this post, including the figures above.

Common Mistakes When Learning CI/CD Tools

• Learning one tool's syntax without learning the concepts. Triggers, jobs, caching, secrets, and artifacts exist in all three. Learn those, and switching tools takes days instead of weeks.
• Leaving secrets behind in tools you've stopped using. conda-forge had deprecated CircleCI but never removed its tokens, which is a big part of why the 2023 incident hurt. When you retire a CI service, delete the secrets stored in it.
• Using unpinned third-party actions or plugins. Anything you pull in runs with access to your pipeline. Pin versions and review what you depend on.
• Putting long-lived cloud keys in CI settings. Short-lived credentials limit the damage when something goes wrong.
• Choosing Jenkins "because it's free" without budgeting for maintenance. Free software with a part-time operator is rarely free.
• Treating the CI platform choice as permanent. Both FreeAgent and Nylas migrated, and the pipeline logic you write is more portable than it feels.

Frequently Asked Questions

Should a beginner learn GitHub Actions, Jenkins, or CircleCI first?

GitHub Actions, for most beginners. If your projects are on GitHub, you can have a working pipeline in minutes with nothing to install, and the concepts transfer directly to the others.

Is Jenkins still worth learning?

Yes, especially for enterprise, on-premises, and regulated environments, where it remains common. You may well inherit a Jenkins setup in a real job even if you'd never choose it for a new project.

Did the CircleCI security incident make it unsafe to use?

CircleCI published a detailed incident report, revoked tokens, and states that customers can build safely on the platform. Whether that's enough is a risk decision for your team. The lasting lesson applies to every hosted CI tool: minimize what secrets live there, scope them tightly, and rotate them.

Is GitHub Actions really free?

It's free for public repositories, and private repositories get a monthly allowance of free minutes before usage-based billing applies. GitHub changed its pricing in 2026 and has postponed a planned fee for self-hosted runners, so check the current pricing page for exact numbers.

Can I use more than one CI tool?

Yes, and many organizations do, often because of acquisitions or gradual migrations. FreeAgent's own migration was a deliberate move from one to another, not an overnight swap.

How do interviewers evaluate this topic?

Rarely by asking which tool you prefer. They tend to ask you to design a pipeline: what runs on pull requests, what gates a deploy, where secrets live, and how you'd roll back. Explaining those decisions clearly matters more than any specific tool's syntax.

Pick One and Build a Real Pipeline

The honest lesson from FreeAgent's 90% and the 2023 CircleCI incident is the same: the CI platform you choose affects your costs, your engineers' time, and your security posture, not just your build speed. Pick one, build a real pipeline for a real project, and make sure you know exactly where your secrets live. Our free CI/CD Pipelines roadmap walks you through building one end to end, and you can explore the courses for the cloud and DevOps foundation underneath it.