- Book: System Design Pocket Guide: Interviews
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
In December 2025 GitHub wrote that the re-architected backend behind Actions handles 71 million jobs per day, and that developers used 11.5 billion Actions minutes in public and open source projects that year. Picture the interviewer saying: "Design that. You have 45 minutes."
You won't rebuild GitHub's internals in an interview, and you shouldn't pretend to know them. Design a CI service from first principles instead, and borrow from the parts GitHub documents publicly: how runners talk to the service, how the cache is scoped, what the limits are.
Minutes 0–8: requirements and the capacity math
Pin down the scope out loud. Functional: a push or pull request creates a run, a run expands into jobs, jobs run on machines that match their labels, users watch logs live, and builds can reuse a dependency cache. Non-functional: a job should start within seconds of being queued, a job must never be silently lost, and one customer's workload must not degrade anyone else's.
Then the numbers. The ones below are assumptions for the exercise, about a seventh of GitHub's published volume. Say them as assumptions.
jobs/day 10,000,000
avg jobs/s 10M / 86,400 ≈ 116
peak jobs/s x3 (assumed) ≈ 350
avg job length 300 s (assumed)
concurrent jobs 116 x 300 ≈ 35,000
peak concurrent 350 x 300 ≈ 105,000
log per job 200 KB (assumed)
log volume/day 200 KB x 10M = 2 TB
log ingest avg 2 TB / 86,400 s ≈ 23 MB/s
lines/s @ 100 B 2,000 x 116 ≈ 232,000
The concurrency rows come from Little's law, L = λW: arrival rate times time in system gives the average number in flight. That one line tells you where the design effort goes. Handing out 350 jobs a second is a modest scheduling problem. Keeping up to 105,000 machines busy and ingesting a few hundred thousand log lines a second are the hard parts.
Expect the follow-up: "What changes if the average job doubles to ten minutes?" The queue rate stays the same and the runner fleet doubles. Rerunning the math out loud is the point of the question.
Minutes 8–20: the job queue, leases and at-least-once delivery
First decision: do runners pull work or does the scheduler push it? GitHub's self-hosted runner pulls. Older GitHub Enterprise Server docs describe an HTTPS long poll held open for 50 seconds, then reopened, with no inbound connection needed. The current Actions Runner Controller docs describe the same HTTPS long poll. Pull works behind corporate firewalls and keeps the scheduler stateless about who is alive. Push gives the scheduler more control over placement, but it needs reachable runners and an accurate capacity map. For a multi-tenant service with customer-hosted machines, pull wins.
The runner leases the job: it holds it for a short window and keeps renewing. If the runner dies, the lease expires and the job goes back in the queue. This is the same model as an Amazon SQS visibility timeout: default 30 seconds, extended through a heartbeat with ChangeMessageVisibility.
A lease is a claim with an expiry: renew it or lose the job.
Postgres is enough at 350 claims a second, and it is easy to reason about on a whiteboard. SKIP LOCKED is the piece that makes it a queue. The Postgres docs say it "can be used to avoid lock contention with multiple consumers accessing a queue-like table."
-- claim: one queued job whose labels this runner has
UPDATE jobs
SET state = 'running', runner_id = $1,
attempt = attempt + 1,
lease_until = now() + interval '60 seconds'
WHERE id = (
SELECT id FROM jobs
WHERE state = 'queued' AND labels <@ $2
ORDER BY created_at
LIMIT 1
FOR UPDATE SKIP LOCKED
)
RETURNING id, attempt;
-- heartbeat: zero rows updated means you lost the job
UPDATE jobs
SET lease_until = now() + interval '60 seconds'
WHERE id = $1 AND runner_id = $2
AND attempt = $3 AND state = 'running';
-- reaper: expired leases go back, or fail after 3 tries
UPDATE jobs
SET state = CASE WHEN attempt >= 3
THEN 'failed' ELSE 'queued' END,
runner_id = NULL, lease_until = NULL
WHERE state = 'running' AND lease_until < now();
The attempt column is a fencing token. Every write a runner makes (heartbeat, status, artifact upload) carries it, and the server rejects writes from an older attempt.
Say this before the interviewer asks: leases give you at-least-once delivery, and SQS's own docs say a message can be delivered more than once. A runner can lose its network for 90 seconds, get reaped, and come back mid-step while a second runner starts the same job. So steps with side effects must be idempotent.
Some already are: npm publish fails if the name and version combination already exists. A deploy script that posts to Slack and appends a release row is not. Two pushes deploying at the same time is a separate problem with a user-facing fix: GitHub's concurrency groups make sure "only a single job or workflow using the same concurrency group will run at a time."
Expect the follow-up: "The partitioned runner comes back and tries to upload artifacts. What stops it from overwriting the new attempt's?" The fencing token in the WHERE clause.
Minutes 20–28: ephemeral runners and autoscaling
Untrusted code means one job per machine, then throw the machine away. GitHub recommends ephemeral runners for autoscaling: registered with --ephemeral, de-registered after one job. Its security guidance adds just-in-time runners that "perform at most one job before being automatically removed."
The scaling signal is queue depth per label, not CPU. GitHub's Actions Runner Controller works this way on Kubernetes: a listener holds a long poll, waits for a Job Available message, and raises the desired replica count of an ephemeral runner set, which deletes each runner when it finishes.
The trade-off is cold start against idle cost. If a VM takes 40 seconds to boot (assumed), every job pays 40 seconds of queue time unless a warm pool absorbs it. Little's law sizes the pool: 350 jobs a second times 40 seconds is 14,000 machines idling at peak. That is why you keep warm pools only for the busiest labels and let rare ones cold-start.
Expect the follow-up: "How do you keep p50 queue time under ten seconds without paying for 14,000 idle VMs?" Per-label pools sized from each label's own arrival rate, forecast a few minutes ahead from the queue.
Minutes 28–36: live logs to browsers
The runner batches lines and sends them to a log ingest tier tagged with (job_id, step, seq). The sequence number makes retries harmless: the server drops a chunk it has already seen. While the job runs, the tail lives on the log node that owns that job. When the job ends, compact the log to object storage and serve finished logs as static files.
Browsers subscribe with Server-Sent Events. Each frame carries an id, and on reconnect the browser sends it back in the Last-Event-ID header, so resume-from-line comes free. An example frame:
id: 4182
event: line
data: {"step":3,"text":"npm ci"}
Two details to mention. MDN notes that over HTTP/1.1 browsers cap SSE at 6 connections per browser and domain, so a user with eight build tabs open stalls. Serve the stream over HTTP/2. And fan-out cost is viewers times lines. Most jobs have zero viewers (assumed), so publish a job's lines only once someone subscribes.
Expect the follow-up: "A release build of a popular repo has 5,000 people watching. What breaks?" One log node writing to 5,000 sockets. Put a fan-out tier in front that subscribes once per job and multicasts, and batch lines into frames every few hundred milliseconds.
Minutes 36–42: the cache, keyed by what built it
A dependency cache entry is named by its inputs: OS, architecture, toolchain and a hash of the lockfile. GitHub's dependency caching docs use the same shape, OS plus hashFiles('**/package-lock.json'), and they add rules worth copying. You cannot change an existing entry, only create a new key. A new entry is saved only on a miss and only if the job succeeds. Entries untouched for 7 days are evicted, and the default limit is 10 GB per repository.
package cachekey
import (
"crypto/sha256"
"encoding/hex"
"os"
"runtime"
)
// Key names a cache entry by the inputs that built it.
// scope is the trust boundary, e.g. "refs/heads/main".
func Key(scope, tool, lockfile string) (string, error) {
b, err := os.ReadFile(lockfile)
if err != nil {
return "", err
}
sum := sha256.Sum256(b)
return scope + "/" + tool + "-" + runtime.GOOS + "-" +
runtime.GOARCH + "-" + hex.EncodeToString(sum[:]), nil
}
Store two layers. The key maps to a manifest, and the manifest points at blobs stored by the SHA-256 of their content, so identical tarballs across repos are stored once. Writes are first-writer-wins through a conditional insert on the key.
Name the entry by what built it, and never let anyone rewrite a drawer.
Expect the follow-up: "Two jobs miss the same key at the same moment. Who wins?" Both build, the first to commit the manifest wins, the second discards its upload.
Minutes 42–45: three failure modes to raise unprompted
Stuck leases. A heartbeat thread can stay alive while the step it guards hangs forever, so the lease never expires. Pair every lease with a hard job timeout. GitHub's limits page sets 6 hours of execution per job on hosted runners, 5 days on self-hosted, and cancels a self-hosted job that has waited 24 hours in the queue. The reaper also needs a maximum attempt count, or a job that crashes its runner loops forever.
Cache poisoning across forks. Cache keys are chosen by the client, so scope is the only real boundary. In 2024 research, Adnan Khan showed how untrusted code running in a default-branch context could write cache entries that a privileged release workflow later restored. GitHub's docs now give pull_request_target, issue_comment and workflow_run runs read-only access to the default branch's cache scope. In your design, write scope follows trust: a fork's pull request writes only to its own scope, and release builds skip the shared cache, as Khan recommends.
Noisy-neighbor runners. Shared hosts leak performance and state. GitHub's hosted runner docs say every hosted runner is a fresh VM, except single-CPU runners, which run in a container on a shared VM. Its security guidance warns that persistent self-hosted runners "can be persistently compromised by untrusted code," and that reusing hardware for JIT runners can expose information. Use a VM per job for untrusted code, cgroup CPU and I/O limits for the container tier, and per-tenant concurrency quotas so one organization's 256-job matrix cannot starve the rest. GitHub enforces both numbers: 256 jobs per matrix and per-plan concurrent job caps.
Close with a 90-second summary of the components, then ask the interviewer which one they want to push on.
If this was useful
The System Design Pocket Guide: Interviews walks through 15 real system designs, step by step. If taking a prompt like this one from requirements to capacity math to components and their trade-offs inside 45 minutes still feels shaky, that practice is what the book is for.
Top comments (0)