Disclosure: I have no affiliation with Blacksmith. Nobody asked me to write this and nobody paid for it. I installed their GitHub App like anyone else would.
I pointed the same GitHub Actions job at two runners, ubuntu-latest and Blacksmith's blacksmith-4vcpu-ubuntu-2404, ran each one 10 times, and timed the actual work with date, not the wall-clock job duration GitHub shows you. Blacksmith finished one script 3.17x faster and another 3.77x faster, and the ranges didn't overlap once across 10 trials on either side. Before any of that ran, though, the job just sat queued forever, because Blacksmith refuses to run on a personal repository at all.
What Blacksmith is
Blacksmith sells GitHub Actions runners on bare-metal hardware instead of GitHub's shared virtualized ones, Firecracker microVMs on gaming CPUs, with runs-on: blacksmith-... as the entire migration. It's real money: a $45M Series B led by Peak XV Partners at a $550M valuation in March 2026, on top of an earlier $3.5M seed from GV and Y Combinator (YC W24). For a company at that valuation, its dev.to footprint is almost nothing, one independent write-up on cutting CI costs with it, otherwise it only shows up as a line item in generic CI-runner comparison listicles alongside Depot and WarpBuild. No Show HN, no Launch HN, despite claiming 6,000+ companies running CI on it.
The org-only wall
The repo I wanted to benchmark, a small gzip-compression measurement harness, lives under my personal GitHub account. The Blacksmith job for it sat in queued indefinitely, no error, no explanation. Blacksmith's own quickstart docs mention "organization" 33 times and "personal repo" twice, and the second mention is the actual answer: Blacksmith is explicitly unavailable for personal repositories, org-only. I moved the repo to a fresh organization, installed Blacksmith's GitHub App there, and the job unqueued within seconds. Worth knowing before you evaluate this on a side project: the free 3,000 minutes/month and no-credit-card signup don't matter if your CI lives on a personal account.
Installing the app also opened a "Migration Wizard" pull request on its own, rewriting every runs-on: ubuntu-latest in the repo to the Blacksmith equivalent. I didn't ask for that PR, it just appeared after the app install, which is a genuinely convenient touch if you're migrating a repo with many workflows, and worth flagging if you don't want automated PRs against your default branch.
Running into an unavailable runner image, then a cancelled job
The Migration Wizard's PR pointed at blacksmith-4vcpu-ubuntu-2204. That tag queued forever too, a second time, different cause: the 22.04 image wasn't available on Blacksmith's infrastructure at all. Switching to blacksmith-4vcpu-ubuntu-2404 (24.04 LTS) fixed it. The run after that got further, both jobs started, but the Blacksmith job was cancelled mid-run after about five minutes with no steps executed, which looked like an account-verification hiccup rather than anything about the runner image. The next run completed cleanly, and the 10-trial run the numbers below are from is the one after that, once I'd added repeated trials to the workflow. None of these were show-stoppers, but if you're evaluating Blacksmith on a deadline, budget for at least one queued-forever mystery and one silently-cancelled run before you get a clean baseline.
The benchmark
Same workflow, two parallel jobs, one on ubuntu-latest, one on blacksmith-4vcpu-ubuntu-2404, running the same two Node scripts 10 times each per job. Each trial timed with date +%s%N around the node invocation, not parsed from bash's time builtin, so the numbers below are wall-clock milliseconds per run, plus one untimed warmup run discarded before the loop starts.
| script | github-hosted mean | github-hosted range | blacksmith mean | blacksmith range | speedup |
|---|---|---|---|---|---|
| client.js (main sweep) | 399.2ms | 368–468ms | 126.2ms | 115–166ms | 3.17x |
| crossover.js (fine sweep) | 278.3ms | 256–307ms | 73.7ms | 69–83ms | 3.77x |
Two things stand out beyond the raw speedup. First, the ranges don't overlap on either script, github-hosted's fastest trial (368ms) is still slower than Blacksmith's slowest (166ms) on client.js, so this isn't noise. Second, Blacksmith's own range is consistently tighter (51ms and 14ms wide) than github-hosted's (100ms and 51ms wide), meaning the speed is also more predictable run to run, not just faster on average.
The workflow and the raw run logs are public: github.com/Shed-engineering/gzip-threshold-json-api, workflow file at .github/workflows/blacksmith-benchmark.yml, trial script at .github/scripts/run-trials.sh.
What I didn't test
This is one small, CPU-light Node script run 10 times, not a real build. A workflow that's dominated by dependency installs, Docker layer builds, or a slow test suite could see a very different ratio, better or worse, than a two-second measurement loop does. I also didn't touch cost: everything here ran inside Blacksmith's free 3,000 minutes/month, so I never generated a bill and can't independently confirm their stated $0.004/min pricing myself, that's their pricing page, not something I measured. Separately, one dev.to article's user testimony reported 41% less run time, 45% lower per-run cost, and 2.4x faster execution after switching, that's a third party's real-world number, not mine or Blacksmith's, and I have no way to verify it either. And this was a single job type on one runner size (4vCPU); Blacksmith offers larger sizes I didn't try.
Blacksmith's own marketing claims 2-4x faster builds. My two numbers, 3.17x and 3.77x, both land inside that range, toward its upper half, for what that's worth from one small, controlled comparison rather than a real production pipeline.
Why I'm not switching, despite the numbers above
Worth saying plainly: I'm not moving my own CI to Blacksmith after writing this. Two reasons, neither of them about the performance, which is genuinely real. First, the org-only requirement means my actual GitHub Actions workloads, which live on personal repos, would need to move to an organization before Blacksmith would even queue a job, and that's a bigger change than swapping a runs-on line. Second, the GitHub App install grants write access to actions, workflows, contents, pull requests, and self-hosted runners on whatever it's installed on, which is a reasonable amount of access for what it does, but it's still a third-party app with that much reach into a repo, and my own workflows run a handful of short jobs a day, not enough volume for a 3-4x speedup to matter in wall-clock time I'd actually notice. If you're running a CI pipeline heavy enough that minutes add up, on an org already, the case above is a real one. Mine isn't that pipeline, plain GitHub-hosted runners are still fine for what I run.
Run it yourself
- Move (or create) the target repo under a GitHub organization, not a personal account, Blacksmith won't queue jobs otherwise.
- Install the Blacksmith GitHub App on that organization from blacksmith.sh.
- Change
runs-on: ubuntu-latesttoruns-on: blacksmith-4vcpu-ubuntu-.... A Migration Wizard PR may show up on its own after the app install, but check the image tag it proposes against Blacksmith's currently available images before merging it, the one it opened for me pointed at a tag that isn't available anymore. - If a job sits queued with no error, that's the first thing to check, not an account problem.
The whole benchmark workflow above (both jobs, the trial-loop script, 10 trials each) is copy-pasteable from the repo linked above if you want to point it at your own workload instead of taking these numbers on faith.
Top comments (1)
Rip to my company's AWS/Runner budget, but hey, at least the builds are blazing fast 😂.