DEV Community

Cover image for I priced the GitHub Actions workflows of 20 famous OSS projects. The results were ugly.
depmedicdev-byte
depmedicdev-byte

Posted on • Originally published at depmedicdev-byte.github.io

I priced the GitHub Actions workflows of 20 famous OSS projects. The results were ugly.

There is a class of "best practices for GitHub Actions" article that just lists the same five tips - cancel concurrent runs, set a timeout, cache your dependencies, pin actions to a SHA, narrow your matrix. Every one of those articles is correct, and almost none of them tell you how often even the best open source projects miss those exact things.

So I pulled the live workflow YAML out of .github/workflows for 20 well-known repositories, priced every job at GitHub's published per-minute rates, and ran a linter against the whole set. The data and the tools are public.

Headline numbers, all from public workflow YAML on main:

  • 20 repos
  • 229 workflows scanned
  • 388 priced jobs (159 more were jobs running on self-hosted or unknown runners and excluded from cost math)
  • 902 CI findings flagged by ci-doctor
  • About $51,000 of combined monthly spend assuming each job runs 30 times per day at 8 minutes per run

The interactive table is at https://depmedicdev-byte.github.io/benchmarks.html. A few of the per-repo numbers from the dataset:

Repo $/run Monthly @ 30/day Findings
denoland/deno $18.30 $16,473.60 84
facebook/react $16.19 $14,572.80 87
vercel/next.js (varies, see table) (varies) (varies)

(The full table has all 20 repos, sorted by monthly spend.)

What every project gets wrong

Across 902 findings, the distribution is heavy on a small number of issues:

Rule Count
missing-timeout 364
missing-cache 113
pinned-action-sha 91
missing-permissions 80
artifact-no-retention 73
matrix-overcommit 52
missing-concurrency 52
deprecated-action 39
fetch-depth-zero 22
expensive-runner 12
wide-trigger 4

missing-timeout is the single biggest one. Out of 388 priced jobs, 364 of them are running without a timeout-minutes. A stuck job in CI does not exit on its own. It runs until GitHub kills it at 360 minutes, which is six hours of paid runner time, on top of whatever the job was actually trying to do.

missing-cache is the second. Most projects have one cached language tool (npm, or pip, or whatever) and then forget to cache the rest. Every pnpm install or bundle install or cargo build that runs without a cache costs you 1-3 paid minutes per job, every run.

missing-concurrency only shows up 52 times but it is the highest-leverage one to fix. Without a concurrency: block, a developer who pushes three quick fixes in a row triggers three full CI runs back to back. With a concurrency: block, the first two cancel and only the third runs. On an active PR this can cut your monthly spend in half.

How the data was generated

Three small open-source tools, all on npm:

# Audit a workflow:
npx ci-doctor .github/workflows/ci.yml

# Estimate the cost of a workflow:
npx gha-budget .github/workflows/ci.yml --runs-per-day 30 --minutes 8

# Pin every action in a workflow to a SHA:
npx pin-actions
Enter fullscreen mode Exit fullscreen mode

The benchmark fetched workflow YAML for each repo via the GitHub REST API, then ran ci-doctor and gha-budget against every file. Each priced job was multiplied by 30 runs/day and 30 days/month. Self-hosted runners and runs-on values not in the GitHub price sheet were excluded from cost math but still scanned for findings. Code is in https://github.com/depmedicdev-byte if you want to reproduce it.

What you can do today

If you have a workflow you suspect is expensive, the fastest check is to paste the YAML into one of these:

Both run entirely in your browser. They share findings and totals via URL hash so you can send a teammate a link.

What I'd actually do

If I had to fix one thing per repo for maximum dollar impact, the order would be:

  1. Add concurrency: { group: ${{ github.workflow }}-${{ github.ref }}, cancel-in-progress: true } to every PR-triggered workflow.
  2. Add timeout-minutes: 15 (or whatever your real p99 is) to every job.
  3. Cache whatever your install step is downloading. actions/cache for ad-hoc dirs, the official setup actions for language tools.
  4. Pin every uses: to a full commit SHA. This is a security move more than a cost move, but the same pin-actions CLI handles it.

Those four changes alone, applied across the 20 projects in the dataset, would cut the combined monthly spend roughly in half. The longer set of patterns (60+) is in the Cut Your CI Bill cookbook but you do not need it to fix the top three.

Re-runs and updates

The dataset will be re-run monthly. If you want a specific repo added, open an issue on any of the tool repos and I will include it.

Top comments (0)