Command: ask Claude Code to pull CI timings to JSON with zero analysis, then run a second pass on medians. The author cut 41-min CI to 9-min using Claude Code's data-first pipeline profiling.
Key Takeaways
- Command: ask Claude Code to pull CI timings to JSON with zero analysis, then run a second pass on medians.
- The author cut 41-min CI to 9-min using Claude Code's data-first pipeline profiling.
The Problem: 41 Minutes Changes Team Behavior
A 41-minute CI pipeline doesn't just cost time. It changes how engineers work. Developers stopped pushing small fixes — they batched three days of work into one giant PR. When that PR went red, nobody knew which of fourteen changes broke it.
With six engineers pushing four times a day, that's 16 hours of pipeline time daily, plus queueing on a runner pool that could only handle six concurrent jobs. Some afternoons, "CI is slow" meant an hour of queue time on top of the 41 minutes.
The author had tried fixing it twice before. Both times: opened the CI config, found something that "looked slow," added a cache key, declared victory. Both times it got 3-4 minutes faster and drifted back within a month.
The root cause of those failures: never actually measuring where the 41 minutes went. That's not profiling — that's vibes.
The Technique: Data-First Pipeline Profiling with Claude Code
The fix that worked: treat the pipeline like a performance bug in an application. Get real timing data first. Don't touch a line of config until the data says where the time is.
Step 1: Extract Data with Zero Analysis
Every CI provider exposes per-step timings through its API. The author asked Claude Code to pull the last 50 successful runs on main and flatten them into a sortable file.
The critical prompt constraint:
"Pull the last 50 successful pipeline runs from the CI API. For each run, extract every job and step with its duration in seconds. Write it to
ci-timings.json. Do not analyze it yet, do not suggest fixes, and do not open the CI config. I only want the data."
That "do not suggest fixes yet" line is everything. If you ask an agent to fetch data and fix a problem in the same breath, it will start proposing fixes from the first thing it sees, and everything downstream becomes an argument for that first guess.
The resulting script was ~60 lines of Python using urllib to hit the CI API, extracting job name, stage, queue time, and duration into JSON rows.
Step 2: Let the Median Pick the Target
Second pass, on the file only:
"Read
ci-timings.json. Group by job name. For each job, report median duration, p90, and median queue time. Sort by median duration descending. Tell me what fraction of total wall-clock the top 3 jobs account for. No recommendations yet."
The output reordered the author's entire mental model:
| Job | Median | p90 | Median queue |
|---|---|---|---|
test:integration |
18m 40s | 26m 10s | 4m 02s |
build:docker |
9m 55s | 11m 30s | 0m 12s |
test:unit |
6m 20s | 7m 05s | 3m 40s |
lint |
2m 50s | 3m 00s | 3m 55s |
Two wrong assumptions revealed:
-
build:dockerwasn't the villain — it's the one people complain about because it's visible. Buttest:integrationwas 2x slower. -
Queue time was a hidden cost —
test:integrationandlintwere both waiting 4 minutes in queue, wasting runner slots.
Why It Works
This works because it applies the same discipline as profiling a slow endpoint: collect data, then analyze. Claude Code is excellent at both, but only when you enforce separation. The first pass builds a factual baseline. The second pass surfaces the median, not the average — medians resist the skew of one bad run.
How To Apply It
-
Write a prompt that forbids analysis. Tell Claude Code to extract CI data to
ci-timings.jsonand stop. No recommendations. - Run a second prompt on the file only. Ask for medians, p90s, and queue times sorted descending. Ask what fraction of wall-clock the top 3 jobs consume.
- Fix the top 3 only. The author's fixes were "boring" — parallelizing integration tests, caching Docker layers properly, and reducing queue contention — but they were targeted at real data.
Result: 41 minutes → 9 minutes in two afternoons.
A Note on Claude Code's Current Capabilities
This workflow is even more powerful with recent Claude Code models. With Opus 4.6 and the latest Claude Code, the agent can write the extraction script, run it, and produce the analysis table in one session — as long as you enforce the two-pass discipline. The separation of concerns is what makes it reliable, not the model's raw power.
Source: dev.to
Originally published on gentic.news
Top comments (0)