Does using the same model mean similar token usage across coding agents?
We compared Codex, SunnyCode Base and SunnyCode CodeMode on eight tasks from p-queue, Click and markdown-it. All used GPT-6 Astra with medium reasoning and Fast off. Each configuration ran each task three times, with shared primary acceptance checks and failed runs retained.
I develop SunnyCode, so this is a developer-run comparison, not an independent benchmark.
Results
Means across 24 runs per configuration, including failures:
| Configuration | Tokens | Estimated cost/run | Checks passed | Time |
|---|---|---|---|---|
| Codex | 725,185 | $1.440 | 21/24 | 222.6 s |
| SunnyCode Base | 281,080 | $0.785 | 23/24 | 297.1 s |
| SunnyCode CodeMode | 165,566 | $0.669 | 23/24 | 232.2 s |
The biggest gap was cached input
Cached input accounted for 96.5% of Codex’s additional recorded tokens relative to CodeMode. These totals include context processed across requests—not just generated code.
Because cached input is discounted, CodeMode’s 77.2% reduction in tokens translated to a smaller 53.5% reduction in estimated cost.
Codex still had the fastest overall mean completion time. Fewer tokens did not automatically mean faster execution.
Costs are API-equivalent estimates, not subscription bills, using frozen rates of $10/M uncached input, $1/M cached input and $50/M output.
What this does—and doesn’t—show
Agent configuration is worth measuring alongside model choice. Separate cached input, uncached input and output, and compare acceptance and time alongside cost.
But this is eight tasks, not a general ranking. System prompts differed, Base runs were collected later, and provider/cache conditions were not randomized. Supplemental checks also found defects beyond the primary acceptance results.
Top comments (0)