I build Empryo, an AI coding agent for desktop and terminal. In my benchmark published on 16 August 2026, Empryo and pi both fixed all four bugs on Opus 5. The published round totals were $2.27 for Empryo and $3.10 for pi, summing the per-task costs.
I'm focusing on Opus because both agents passed every task in that slice, so the cost comparison doesn't trade away a successful fix. This isn't a claim about today's versions or every model. The wider report covers six models, and explicitly notes that pi can still be cheaper on inexpensive models.
Where the money went
The bugs came from hono and opencode. Hidden regression tests from the merged fixes graded the changes.
| Opus 5 task | Empryo | pi |
|---|---|---|
| Cookie mangling, hono | $0.468 | $0.269 |
| TrieRouter regexp, hono | $0.570 | $0.634 |
| Symlinked grep root, opencode | $0.762 | $1.19 |
| Revert by ID, opencode | $0.472 | $1.01 |
| Round total, rounded | $2.27 | $3.10 |
pi was cheaper on the cookie bug. Most of Empryo's net savings came from the two opencode tasks.
The published report uses a small number of attempts: mostly two per result, with some single runs. Both repositories are TypeScript. Empryo ran with 37 tools and its Genome code map; the tested pi configuration used four tools. This compares those configurations, not every possible pi setup.
The timing protocol also prebuilt Empryo's code index. Cold indexing was reported separately at 96-100 seconds for opencode and eight seconds for hono. Don't mistake an agent-working timer for a fresh-install timer. I'm keeping this post's comparison to cost and correctness.
Top comments (1)
The hono cookie vs opencode split explains the tool count tradeoff cleanly. When a task is localized to one file, carrying 37 tool definitions in the system prompt burns tokens on every turn without saving steps, which is why a lean 4-tool setup like pi runs cheaper. The code map only earns back its prompt weight once the repo is large enough that an unguided agent wastes multiple roundtrips grepping through symlinks and directories.