Originally published on the Manzanita blog. I work on Manzanita, one of the runners compared below — so weigh the source. I've tried to make every number checkable: each one links to the public GitHub Actions run it came from, and I've kept in the cases where the dedicated runner was flat or slower.
Marketing pages say "faster." This is the measurement. I took five public iOS and macOS repositories, forked them, changed a single runs-on line, and ran their own GitHub Actions workflows on both Manzanita and GitHub-hosted macOS runners. Every number below links to the public run it came from, so you can check it yourself.
How I measured
-
Same workflow, one line changed. Only the runner label was swapped (for example
macos-26tomanzanita-standard). No application code was edited to make a job green. - Warm vs warm. Where a project uses a cache, the fair comparison is a repeat run where both sides restored the cache. Those are reported, with cold numbers noted separately.
- Execution and queue are separate. The tables are step and job execution time. A Manzanita job can also wait behind your own purchased concurrency; that queue time is called out, not hidden.
- Honest scope. Some projects pin an exact Xcode I don't publish, or couple non-Apple toolchains into their macOS jobs. I say so in each case rather than dropping the project.
The short version
| Project | Workload | GitHub-hosted | Manzanita | Result |
|---|---|---|---|---|
| argmax-oss-swift | iOS test job (whole) | 25m54s | 8m16s | 3.13× faster |
| TablePro | iOS test workflow (wall) | 10m23s | 3m27s | 3.0× faster |
| SwiftLM | Build & unit test | 20m31s | 8m22s | 2.45× faster |
| speech-swift | SwiftPM + Metal (cold) | 13m44s | 9m04s | 1.5× faster |
| skip | Apple build step | 5m46s | 1m13s | 4.74× faster |
The pattern: simulator-heavy iOS test jobs and clean compiles see the biggest wins, because a dedicated M4 Pro with 6 vCPU beats a shared 3–4 vCPU VM and the simulator is already warm. Now the detail, including where the picture is mixed.
argmax-oss-swift — 3.13× on iOS, verified
Argmax's open-source Swift SDK runs a macOS and an iOS lane. On the repeat (warm-cache) run the iOS job went from 25m54s to 8m16s; the simulator boot alone dropped from 2m45s to 17s and the test step from 20m06s to 6m28s. The macOS lane was a more modest 9m02s to 6m27s (1.40×). Whole run: 25m58s to 8m48s.
Scope note: the repo requests exact Xcode 26.4.1, which I don't publish; timing runs used the Xcode 26.6 image, so this is a like-for-like performance result, not an exact-version compatibility claim.
Verify: Manzanita run · GitHub-hosted baseline
TablePro — 3.81× on the iOS test step
TablePro is a native macOS/iOS database client. Its iOS tests ran 9m50s to 2m35s (3.81×), the iOS unit-test step 8m13s to 1m39s, and the whole iOS workflow 10m23s to 3m27s (3.0×).
Honest limitation: the native macOS matrix could not finish on the image at the time because the project's build script calls rustup, which GitHub's macos-26 image ships and this one did not yet. That's an image gap being closed, not a compute result, so only the iOS numbers are reported here.
Verify: Manzanita iOS run
SwiftLM — 2.45× on build, and where it was slower
SwiftLM is an MLX (Apple-Silicon ML) project that targets Xcode 16, so it ran on the manzanita-xcode16 channel. All eight retained jobs passed twice. The build-and-unit-test job went 20m31s to 8m22s (2.45×), and the larger inference jobs were 1.3× to 1.7× faster.
But three short jobs were slower on the dedicated runner — the SSD/draft memory guard (1m44s to 3m20s), vision integration, and fixtures — largely because this model-heavy repo churns through many large GitHub Actions cache keys and some missed on the repeat. I'm showing this because a benchmark that only reports its wins isn't a benchmark. On the standard Xcode 26 channel the unit tests were 2.70× faster but MLX Metal kernels failed to load, so the right migration for this project is the Xcode 16 channel, not the default.
Verify: Manzanita run · GitHub-hosted baseline
skip — a 4.74× build step, and an honest incomplete
Skip builds Swift apps for both Apple and Android. Its Apple build step ran 5m46s to 1m13s (4.74×), and Xcode selection dropped from 3m33s to 8s. No full-job speedup is claimed: the project's test target invokes Gradle and needs the Android SDK, which deliberately isn't on an Apple runner, so the full job stops at that cross-platform step. Useful as a build-step data point; not a complete end-to-end benchmark.
Verify: Manzanita run · GitHub-hosted baseline
speech-swift — 1.5× cold, against a cached baseline
soniqo's speech-swift is a SwiftPM + Metal workload. Even a cold Manzanita run (nothing cached) finished in 9m04s against a 13m44s GitHub-hosted run that hit its cache; the compile step alone was about 3.4× faster. With the cache warm the gap widens further.
Verify: Manzanita cold run · GitHub-hosted baseline
What this does and does not prove
It proves that on real, unmodified Apple workflows a dedicated M4 Pro runner is materially faster than a shared GitHub-hosted macOS VM — most on simulator and test-heavy jobs, less on already-short steps. It does not prove every job gets faster: short jobs dominated by cache I/O can be flat or slower, and a project that pins an Xcode I don't carry, or needs a non-Apple toolchain, won't be a clean win. Those are in the tables above on purpose.
The other half of the story is cost: a flat monthly price per runner instead of per-minute. The comparison page works through when that's cheaper.
Full disclosure: I work on Manzanita. Happy to answer questions or run a workflow you want to see.
Top comments (0)