CI job times and costs are an increasingly large bottleneck for coding agents. Today we're releasing CI Bench v1, our open benchmark of Github Actions runners on 12 CI workloads for popular open-source projects.
The method, workflows, analysis and every raw job record are public in our GitHub repo, measured October 7 to 11, 2026.
Among the nine configurations measured, Vitko Runners with warm starts and test parallelization was the fastest and had the lowest managed cost per job; RunsOn on spot instances costs less per job at very high volume, while taking about twice as long.
Full disclosure: we sell Vitko Runners, now in private beta.
Top comments (1)
실행 방법과 원시 작업 기록을 함께 공개한 점이 비교에 도움이 됩니다. 팀이 재현할 때는 cold·warm 시작을 분리하고 병렬화 수준도 같은 조건으로 맞추면, 러너 자체의 차이와 캐시 효과를 구분하기 좋겠네요.