DEV Community: ProxySoul

Empryo vs pi on Opus 5: four bugs fixed, $2.27 vs $3.10

ProxySoul — Sun, 27 Sep 2026 16:55:13 +0000

I build Empryo, an AI coding agent for desktop and terminal. In my benchmark published on 16 August 2026, Empryo and pi both fixed all four bugs on Opus 5. The published round totals were $2.27 for Empryo and $3.10 for pi, summing the per-task costs.

I'm focusing on Opus because both agents passed every task in that slice, so the cost comparison doesn't trade away a successful fix. This isn't a claim about today's versions or every model. The wider report covers six models, and explicitly notes that pi can still be cheaper on inexpensive models.

Where the money went

The bugs came from hono and opencode. Hidden regression tests from the merged fixes graded the changes.

Opus 5 task	Empryo	pi
Cookie mangling, hono	$0.468	$0.269
TrieRouter regexp, hono	$0.570	$0.634
Symlinked grep root, opencode	$0.762	$1.19
Revert by ID, opencode	$0.472	$1.01
Round total, rounded	$2.27	$3.10

pi was cheaper on the cookie bug. Most of Empryo's net savings came from the two opencode tasks.

The published report uses a small number of attempts: mostly two per result, with some single runs. Both repositories are TypeScript. Empryo ran with 37 tools and its Genome code map; the tested pi configuration used four tools. This compares those configurations, not every possible pi setup.

The timing protocol also prebuilt Empryo's code index. Cold indexing was reported separately at 96-100 seconds for opencode and eight seconds for hono. Don't mistake an agent-working timer for a fresh-install timer. I'm keeping this post's comparison to cost and correctness.

Results, charts and limitations.

Public tasks, raw results and reproduction setup.

In Empryo, a bug fix should leave a guard behind

ProxySoul — Sun, 27 Sep 2026 13:28:27 +0000

i don't want to fix the same bug again next week because another agent wrote the same pattern. so this is the rule in Empryo, an AI coding agent i'm building: a fix should leave a guard behind!

i call this the immune system. here's the useful part if u want to build one for ur own project.

reproduce before fixing

a hunter gets a slice of the code and one kind of bug to look for. it has to reproduce the failure in a throwaway home directory, away from my real config. a convincing paragraph about a possible bug isn't enough.

an independent reviewer reruns that reproduction. if it doesn't hold up, the report gets rejected. the agent that found it doesn't get to approve its own finding.

prove the fix, then guard the pattern

the fixer has to show the failure on the old code and the passing case on the new code. another reviewer checks the diff and reruns the proof.

for patterns i can catch statically, the fix also gets a GritQL rule in Biome. two fixtures go with it:

bad code the rule must flag
good code the rule must leave alone

then i run the rule over the production code. if the pattern exists somewhere else, there's more work to do.

that's what i want from a fix. the next agent shouldn't need to remember a warning buried in an old conversation. lint should catch the pattern when it writes it again.

a lint rule still has limits! it catches the shape i taught it, not every possible version of the bug. the reproduction and real-app checks still matter.

keep the handoffs in files

a bug is a markdown file. its folder is its state: found, ready, fixed, rejected. an agent writes its evidence there before handing it off.

claiming a record uses mkdir claims/<id>. only the agent whose mkdir succeeds owns that claim. stale claims still need handling when a worker dies.

the record survives the session. and anything disputed comes back to human triage. i'm not letting a pile of confident reports decide what's true.

take the skill

the workflow is public in SoulStack's immune-system skill. u can use the approach with an agent that runs shell commands; it doesn't require switching coding agents.

Empryo is the app i'm building it around. its Genome maps symbols, callers and imports, which gives the hunters somewhere concrete to start.

start small: one reproduced bug, one reviewed fix, one guard u can prove. scale after that.

How i give my coding agent a map of the repo with Empryo

ProxySoul — Tue, 15 Sep 2026 19:54:13 +0000

i built Empryo solo because i was tired of explaining the same repo to my coding agent. where things live, what calls what, which files usually need to change together. then a new session starts and i'm doing it again.

Token usage bothered me too. A few prompts can involve a lot of reading, retries, and tool results. The prompt you type is only a small part of what the model processes.

So i started with the thing i wanted the agent to have before it opened another file: a useful map.

The Genome

The screenshot above is Empryo's Genome dock. It maps files, symbols, and imports, and updates as the code changes. The little agent moves around the graph while it works. yes, i gave it a home.

Behind that view, the agent gets a ranked text map. Connections between files matter. Recent reads and edits matter. Files that tend to change together in Git can provide another clue.

The graph helps choose where to look. The agent still needs to open the source, check its assumptions, and run the relevant tests.

One rename, several places to check

Take an illustrative task: rename a session method and update its callers.

auth/session.ts
  refreshSession()
       |
       +-- api/client.ts
       +-- workers/refresh.ts
       +-- tests/session.test.ts

A filename search might find the implementation immediately. The more interesting question is what depends on it.

An import relationship gives the agent a place to investigate. A test that often changes alongside the implementation is another lead. Neither proves that every caller has been found. Dynamic calls, generated code, and string references still need checking.

The workflow i want is simple:

Use the map to pick the likely implementation and dependents.
Read those files before changing the interface.
Make the change and update the map.
Search for remaining references and run the focused tests.
Review the diff for changes outside the task.

Keeping that map current matters. A map of yesterday's code can confidently send an agent to the wrong place.

Same idea, in the terminal

This is the TUI. The change view shows dependents and co-change hints beside the files. i want those clues visible to me too, so i can see what the agent is working from.

It also has themes. i spend enough time in a terminal to care how it feels to sit in one.

A cheaper model can still make a more expensive run

Empryo lets me route different tasks to different models. Exploration, editing, and review don't always need the same choice.

But switching models has a catch: cache reuse.

A worker on the same model can sometimes reuse an existing prompt prefix. Moving it to another model can mean paying to process that context again. A lower token price doesn't automatically mean a lower total.

This is why i care about cached input, fresh input, output, retries, and whether the change actually worked. A failed cheap attempt followed by a full retry is still two attempts.

Subscription usage is another separate measure. A token-price estimate is not the amount charged to a subscription, and cache percentage is not a percentage of money saved.

A small test i'd actually trust

If you're comparing coding agents, give each one the same commit and a small task with a checkable result. Keep the model and permissions comparable where possible.

Record:

Check	What to write down
Correctness	Did the relevant tests pass, and does the diff solve the task?
Exploration	Which files were read before the first edit?
Rework	How many retries or corrections were needed?
Usage	Fresh input, cached input, output, and the billing basis
Your time	How much explanation and review did it take?

Repeat it on a few tasks. One attractive screenshot can't tell you which setup works best for your repo.

That's the problem i'm working on with Empryo: better project awareness, with desktop and terminal interfaces that make the work easier to follow.

GitHub

How do you handle this today? A repo instructions file, your own index, or a lot of repeated explanations? i'm especially interested in what breaks once a project gets big.

AI disclosure: this article was drafted by an AI agent using my project documentation, screenshots, and the notes i supplied.