I expected an agentic coding assistant to use more tokens than a simpler tool. I didn't expect the difference to be this large.
For the exact same simple coding task, I measured:
| Pi | Aider | |
|---|---|---|
| LLM calls | 3 | 1 |
| JSON exchanged | ~760 KB | ~100 KB |
The task was very simple: a small PyQt project, two files, and a single instruction — "Make the cards width = total_width / 3." I already knew exactly which two files needed to be changed.
That made me curious about what was actually happening under the hood.
My API bill had gone above $400/month, despite trying various techniques to reduce token usage. My company doesn't pay for it, so I wanted to understand where all those tokens were going.
I therefore traced the exchanges between a coding harness and the LLM during a simple coding task.
To make the test as clean as possible, I used Pi with no add-ons. I explicitly added the two files that needed editing to its context, then compared the result with the exact same prompt in Aider.
Here is an overview of the Pi workflow:
- User sends prompt: "Make the cards width = total_width / 3" and tags A.py and B.py.
- LLM responds with read tool calls to fetch the contents of both files.
- Harness returns the full text of both files.
- LLM generates internal thoughts about UI design (adding color blue) and uses the edit tool on A.py and B.py to restructure the layout.
- Harness confirms: "Successfully replaced 2 block(s)".
- LLM uses the edit tool on B.py to add the new CSS classes for the updated components.
- Harness confirms: "Successfully replaced 2 block(s)".
- LLM sends a text response to the User summarizing the design changes (Modern layout, Visual Hierarchy, Selection states).
So you see: for a simple request where I already know what files to edit, there are 3 back and forth with the LLM.
The total size of the json files exchanged is 760kB. Not huge, but again this is a very trivial test.
The ah-ah moment came when I compared this with the exact same prompt with aider.
Here is the aider workflow:
-
Harness sends a massive, pre-assembled prompt containing:
- A "Repo Map" (summaries of all files in the project to provide global context).
- The full, raw text of the files the user "added" to the chat (A.py and B.py).
- Strict system instructions on how to format SEARCH/REPLACE blocks.
- The user's actual prompt: "Make the cards width = total_width / 3"
- LLM generates internal thoughts to figure out the Qt layout math.
- LLM responds with a text explanation of the planned changes, followed by three specific SEARCH/REPLACE blocks targeting A.py
Here we have a single LLM call. The first prompt is larger, but contains all the necessary context for the LLM to work.
The total size of the json files exchanged is only 100kB !
Of course, the experiment is excessively simple and fits a case where I already know which files need editing. But in my experience, this is actually the case for a majority of my daily tasks. I don't need an agent to "discover" them and burn tokens in a loop. I just need a tool that applies the edits efficiently.
Aider is fantastic for this, but I unfortunately it is not maintained anymore. So I ended up building my own lightweight harness around this exact single-prompt philosophy called Frugaast.
By switching part of my workflow away from agentic loops and just using single-shot edits for known files, my API bill dropped from $400+ to under $100 a month.
Has anyone else traced their AI editor's network calls? I'm curious if other popular tools like Cursor or Copilot are hiding massive token bloat under the hood.
Top comments (3)
My bill hit $380 last month before I started logging request sizes and saw where it was actually going. The harness confirmations were the surprise - they were getting appended to the context, so by round 3 I was paying for the original file content three times over. I've now split my workflow manually into "I know the file" vs "I don't" and it's cut costs but it's a constant judgment call I didn't really want to be making. I'd probably have built something similar to Frugaast if I'd measured this early enough instead of just feeling like something was off.
This is a strong argument for measuring cost per verified outcome rather than raw token volume or call count. Three calls can be worthwhile if they improve success rate, but repeated context, failed tool calls, and unnecessary edits should be charged to the same task outcome. I would log tokens, retries, touched files, test result, and whether the requested behavior actually passed. That makes the routing decision measurable instead of stylistic.
The known-files condition is the key boundary here. A useful router could ask two questions before choosing a harness: are the target files known, and is the acceptance test already expressible? If both are yes, a single assembled edit request should win. If either is no, exploration has value. I would measure cache-read and cache-write tokens too, not JSON bytes alone, because repeated large contexts can be cheap in bandwidth but still dominate the actual bill.