DEV Community: Joseph Vitko

What a GitHub Actions job costs on each runner: CI Bench v1 cost per job by workload

Joseph Vitko — Mon, 02 Nov 2026 22:43:27 +0000

I maintain CI Bench, an open benchmark of GitHub Actions runners. I'm also the founder of Vitko, which sells one of the products measured (Vitko Runners, private beta). This post is about cost per job.

A runner's list price per minute is not fully correlated with cost per job. Runners take different amounts of time and round differently. For example, GitHub rounds every job up to a whole minute, while Vitko Runners bills per second. So we measured the same twelve real open-source CI jobs on nine configurations and priced each run.

Cost per job against GitHub-hosted 2 vCPU, at 100,000 jobs a month:

RunsOn 88% lower (self-hosted in your own AWS account, so excluding setup and operating work)
Vitko Runners 84% lower
Ubicloud 83% lower
Blacksmith 52% lower.
A GitHub-hosted 4 vCPU runner cost 39% more per job and ran 1.6x as fast.
Self-hosted options spread fixed monthly costs over the volume so their figure moves with it, the calculator shows that.

Links to the benchmark results:

Corrections from any provider measured are welcome, please submit here.

GitHub Actions runners ranked by speed and cost per job (October 2026)

Joseph Vitko — Tue, 27 Oct 2026 07:00:39 +0000

I maintain CI Bench, an open benchmark of GitHub Actions runners.

Disclosure first: I'm the founder of Vitko, which sells one of the products it measures (Vitko Runners). That's why I made every workflow, raw job record and line of method public, so you can check the numbers or re-run them.

In my previous post I described how the benchmark works. This one is the results.

What we measured

Nine runner configurations, twelve real CI jobs from open-source projects (fd, flask, guava, pydantic, vite, vue, a monorepo and others), measured October 7 to 11, 2026:

GitHub-hosted: ubuntu-latest (2 vCPU) and a 4 vCPU larger runner
Managed services: Blacksmith, Ubicloud, Vitko Runners (and Vitko with warm starts and test splitting)
Self-hosted: RunsOn, plus two setups we ran ourselves on Google Cloud and on AWS (one VM per job)

Ranked by speed

Speed against ubuntu-latest, as the geometric mean of median job time over the 12 workloads:

Rank	Configuration	Speed vs ubuntu-latest
1	Vitko Runners (warm starts + test splitting)	2.3× faster
2	Vitko Runners	1.7× faster
3	GitHub-hosted larger runner (4 vCPU)	1.6× faster
4	Blacksmith	1.6× faster
5	Self-hosted on Google Cloud	1.5× faster
6	RunsOn	1.2× faster
7	Self-hosted on AWS (one VM per job)	1.2× faster
8	Ubicloud	1.1× faster
9	GitHub-hosted `ubuntu-latest`	baseline

The ranking hides a lot: Blacksmith was fastest on the burst workload, and the self-hosted Google Cloud setup tied for fastest on flask. The per-workload table is on the speed ranking page.

Ranked by cost per job

Price per minute isn't cost per job, because a faster runner finishes sooner. At 100,000 jobs a month, using list prices dated October 5:

Rank	Configuration	Cost per job vs ubuntu-latest
1	RunsOn (self-hosted, priced all-in)	88% lower
2	Vitko Runners	84% lower
3	Vitko Runners (warm starts + test splitting)	84% lower
4	Ubicloud	83% lower
5	Self-hosted on AWS	62% lower
6	Self-hosted on Google Cloud	59% lower
7	Blacksmith	52% lower
8	GitHub-hosted `ubuntu-latest`	baseline
9	GitHub-hosted larger runner (4 vCPU)	39% higher

Self-hosted costs depend on volume: RunsOn is 88% lower at 100,000 jobs a month, but 43% higher at 1,000, because fixed costs spread over fewer jobs. Details are on the cost ranking page.

What this doesn't tell you

Twelve workloads aren't your workload. Intervals cover sampling noise, not your repo. We didn't measure every provider (Depot, Namespace, WarpBuild and others aren't in v1), and prices change, so every price on the list prices page carries its source and the date it was read. The full method lists what's out of scope.

If you run one of these services and think a number is wrong, tell us and we'll fix it, or submit your own configuration for the next result set. The data is CC BY 4.0.

CI Bench v1: an open benchmark of GitHub Actions runners

Joseph Vitko — Sat, 24 Oct 2026 02:10:13 +0000

CI job times and costs are an increasingly large bottleneck for coding agents. Today we're releasing CI Bench v1, our open benchmark of Github Actions runners on 12 CI workloads for popular open-source projects.

The method, workflows, analysis and every raw job record are public in our GitHub repo, measured October 7 to 11, 2026.

Among the nine configurations measured, Vitko Runners with warm starts and test parallelization was the fastest and had the lowest managed cost per job; RunsOn on spot instances costs less per job at very high volume, while taking about twice as long.

Full disclosure: we sell Vitko Runners, now in private beta.