- Book: AI That Answers
- The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
You type "make the report job faster" and go refill your coffee. The agent comes back with a Map cache around a date formatter, a for loop where a reduce used to be, and a confident summary. The tests are green. Nobody measured anything, so nobody knows whether any of it helped.
That isn't laziness on the agent's part. Anthropic's Claude Code best practices say it plainly: "Claude stops when the work looks done. Without a check it can run, 'looks done' is the only signal available." For speed, the check is a timing, and the map of where to look is a CPU profile. Node writes one with a single flag.
The loop below goes profile, condense, brief, one attempt per branch, measure, land. Steps 1 to 3 are plain Node.js and hyperfine. From step 4 on I use Hermes IDE, which I build, so weigh that part accordingly.
Step 1: profile one workload you can rerun
Profile the job you care about, the same way every time. A script that exits on its own is easiest, because --cpu-prof "writes the CPU profile to disk before exit" (Node.js CLI docs).
// bench.mjs: one fixed workload, run the same way every time
import { buildReport } from './src/report.js';
import { makeOrders } from './test/fixtures.js';
const orders = makeOrders(200_000, { seed: 42 });
const report = buildReport(orders);
// Same input, same answer. A faster version must print this too.
console.log(report.totalCents);
buildReport and makeOrders stand in for your hot path and a fixture builder you already have. The last line gives every attempt a cheap correctness check on top of your tests.
Record the profile into a folder you keep out of git:
mkdir -p perf
node --cpu-prof --cpu-prof-dir=perf \
--cpu-prof-name=before.cpuprofile bench.mjs
Without --cpu-prof-name, Node names the file CPU.${yyyymmdd}.${hhmmss}.${pid}.${tid}.${seq}.cpuprofile. A fixed name means each run overwrites the last one. The profiler samples every 1000 microseconds by default (--cpu-prof-interval changes that), and the --cpu-prof flags have been stable since Node v22.4.0 and v20.16.0, per the same docs. Add perf/ to .gitignore. Profiles are evidence for this one job and don't belong in the repo.
Step 2: condense the profile into a top-20 table
Don't hand the agent the .cpuprofile file. It's JSON with every node of the call tree plus one entry per sample, and the agent pays for every token it reads. The same best-practices page warns that "Claude's context window fills up fast, and performance degrades as it fills." What the agent needs is where the time went.
The format is the Chrome DevTools Protocol's Profile type. nodes holds the call tree, each node with a callFrame. samples holds the "Ids of samples top nodes", and timeDeltas the "time intervals between adjacent samples in microseconds." The node on top of the stack when a sample fires is where the CPU was at that moment. Add up the deltas per top node and you get self time.
// top-self.mjs: top N functions by self time in a .cpuprofile
// usage: node top-self.mjs perf/before.cpuprofile 20
import { readFileSync } from 'node:fs';
const [file, n = '20'] = process.argv.slice(2);
const prof = JSON.parse(readFileSync(file, 'utf8'));
const nodes = new Map(prof.nodes.map((nd) => [nd.id, nd]));
const cwd = `${process.cwd()}/`;
const rows = new Map();
// timeDeltas[i]: microseconds between sample i-1 and sample i
prof.samples.forEach((id, i) => {
const { functionName, url, lineNumber } = nodes.get(id).callFrame;
const path = url.replace('file://', '').replace(cwd, '');
// lineNumber is 0-based in V8 call frames
const where = url ? `${path}:${lineNumber + 1}` : '-';
const name = functionName || '(anonymous)';
const key = `${name} ${where}`;
const row = rows.get(key) ?? { name, where, us: 0 };
row.us += prof.timeDeltas[i];
rows.set(key, row);
});
const total = prof.endTime - prof.startTime;
const top = [...rows.values()]
.sort((a, b) => b.us - a.us)
.slice(0, Number(n));
console.log('| self ms | self % | function | where |');
console.log('|---:|---:|---|---|');
for (const r of top) {
const ms = (r.us / 1000).toFixed(1);
const pct = ((100 * r.us) / total).toFixed(1);
console.log(`| ${ms} | ${pct} | ${r.name} | ${r.where} |`);
}
node top-self.mjs perf/before.cpuprofile 20 > perf/top.md
The script groups rows by function name and location, so a function reached from five different callers shows up once, with its total. The +1 is there because Runtime.CallFrame line numbers are 0-based, and you want the line your editor shows. Self time here is an estimate at the profiler's sampling resolution, which is plenty for ranking.
Rows with - in the where column have no script URL. A V8 profile also records the engine's own work, garbage collection included (JetBrains' V8 profiling docs), and that time is not in any one function you wrote. If those rows sit at the top, say so in the brief.
Step 3: write the brief with numbers in it
Take a baseline wall time first. hyperfine runs a command at least 10 times by default, aiming for about 3 seconds of measurement, and --warmup adds untimed runs before it starts:
hyperfine --warmup 3 --export-markdown perf/baseline.md \
'node bench.mjs'
hyperfine 2.0, released October 7, 2026, runs benchmarked commands without a shell by default (changelog). Keep pipes and && out of the command, or add -S to get the shell back.
Now the brief. It follows the pattern from the Claude Code docs: give the symptom, the likely location, and what "fixed" looks like.
bench.mjs is too slow. Make it faster without changing
what it prints.
Evidence (read both before editing):
- perf/top.md: top 20 functions by self time, from
`node --cpu-prof` on bench.mjs. Approximate, 1 ms samples.
- perf/baseline.md: hyperfine wall time for `node bench.mjs`.
Rules:
- Start from the top rows. For every change, name the row
it targets.
- No caching or memoization unless you can show the same
call repeating with the same input.
- Don't edit test/fixtures.js or bench.mjs.
- After each change, run npm test, then re-profile:
node --cpu-prof --cpu-prof-dir=perf \
--cpu-prof-name=after.cpuprofile bench.mjs
node top-self.mjs perf/after.cpuprofile 20
- Report the before and after rows for the function you
changed, and the hyperfine result.
- If a top row is outside our code, say so and stop there.
Each rule closes off a guess. "Name the row" makes every change traceable to the table. The memoization rule blocks the cache-everything reflex. makeOrders runs inside the profile too, so the fixture rule stops the agent from speeding up the benchmark instead of your code.
Step 4: one attempt per task, each on its own branch
Two attempts at the same brief are worth more than one, and they must not share a folder. First, commit bench.mjs, top-self.mjs and the .gitignore line. A worktree starts from a commit, so anything uncommitted won't be in it. In Hermes, a small recipe prepares each task's worktree. This one follows the format in the worktree recipe docs:
# .hermes/worktree.toml
setup = ["npm ci"]
copy = ["perf/*.md"]
done_when = ["npm test"]
copy brings over files git ignores, which is what perf/top.md and perf/baseline.md are, so every attempt starts with the same evidence. done_when is the default check for when a task counts as done, so a faster attempt that breaks the tests fails its done check. The first time Hermes sees the recipe, and whenever it changes, it shows you the commands, and nothing runs until you choose Run setup.
Press ⌘N (Ctrl+Shift+N on Windows and Linux), paste the brief, and pick Claude Code. The task starts on its own hermes/ branch in its own worktree, with the brief as the agent's first prompt. Press ⌘N again with the same brief and pick Codex, or the same agent with a narrower brief that only allows changes to the top row. Hermes lets you pick the agent, model, effort and account per task.
The ⌘N New task sheet in Hermes 2.3.4, shown with a demo task: Claude Code, the orders-api project, a new hermes/ worktree, Accept edits, model and effort chips, and the "Hermes will run" preview of the claude command.
Wait until both sessions show done before you measure anything. Two agents running test suites in neighbouring worktrees share your CPU, and a benchmark taken while they run picks up their load along with your code's.
Step 5: measure both, read by turn, land one
Measure every attempt in one hyperfine run, from your main checkout, with the machine otherwise quiet. Don't rely on the numbers the agents reported. Their runs happened at different times, under different load.
git worktree list
A=../path-to-attempt-a # paste the paths it printed
B=../path-to-attempt-b
hyperfine --warmup 3 'node bench.mjs' \
"node $A/bench.mjs" "node $B/bench.mjs"
diff <(node bench.mjs) <(node "$A/bench.mjs")
diff <(node bench.mjs) <(node "$B/bench.mjs")
The first command is the reference, and each later result shows its change from it (hyperfine README). Each diff prints nothing when that attempt prints the same total as your main checkout. Then profile the faster attempt and run top-self.mjs on it. The row the agent said it targeted should have shrunk. If the wall time dropped and that row didn't move, you don't yet know why it got faster, so read the change before you trust it.
Reading is where per-turn history pays off. Hermes saves every agent turn. The "Agent replies" strip under the session shows one chip per reply with the lines it changed, and a click shows that turn's changes or restores your files to that point. If turn 3 undid the gain from turn 2, you can go back to turn 2. ⌘G (Ctrl+Shift+G) opens the Review Desk, where you read the attempt by turn or by file. A line comment there goes back to the agent that wrote the line, so "this cache never hits, remove it" reaches the right session.
The Review Desk (⌘G) in By turn view: two turns listed, turn 1's diff of src/db.ts, the changed-files list, and a commit message drafted from the turns.
Land the attempt the numbers back. Land commits, opens a pull request, or merges locally into the branch the task started from, and it can be undone. The other attempt never lands. Its branch and worktree are plain git: while its session is open, or if you keep the worktree when you close the session, git worktree list still shows it.
What the profile won't tell your agent
A CPU profile shows where the CPU went. If bench.mjs spends most of its time waiting on a database or the network, the table will look thin, and it's the wrong tool for that job.
The benchmark is not production either. Seeded fixture orders can have a different shape from your real traffic, so a win here is a hypothesis for production. Sampling has limits too. At one sample per millisecond, very short functions may never land in a sample. Lower --cpu-prof-interval if the table looks too coarse. And when two attempts land within the spread hyperfine reports, call it a tie and keep the simpler diff.
None of this needs Hermes. Two git worktree add commands, a copy of perf/*.md into each new worktree, two terminals and the same brief run the same loop.
If this was useful
I build Hermes IDE. It's free, source-available under BSL 1.1, and runs on macOS, Windows and Linux. 2.3.4 is the latest release. It isn't affiliated with Nous Research, Anthropic, OpenAI or any agent vendor. Download it, and if the per-task worktrees save you a bad merge, star it on GitHub.
The brief in step 3 is a prompt built from evidence, with a stated output format. That's the habit AI That Answers teaches for your own LLM calls in TypeScript: prompts the model can't misread, structured output you can check, and what each token costs you.
It's book 1 of five. The whole AI in TypeScript series, from first LLM call to agents in production, is here.
Top comments (0)