The latest articles on DEV Community by sunny (@z_jc_37f8254a1cb25bff22ed). https://dev.to/z_jc_37f8254a1cb25bff22ed https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4185272%2Fcfef467d-4037-4cf5-9ba6-dfa9dcb976fb.png https://dev.to/z_jc_37f8254a1cb25bff22ed en sunny Wed, 14 Oct 2026 18:16:29 +0000 https://dev.to/z_jc_37f8254a1cb25bff22ed/same-model-different-agents-comparing-token-usage-cost-and-task-success-h67 https://dev.to/z_jc_37f8254a1cb25bff22ed/same-model-different-agents-comparing-token-usage-cost-and-task-success-h67 <p>Does using the same model mean similar token usage across coding agents?</p> <p>We compared Codex, SunnyCode Base and SunnyCode CodeMode on eight tasks from p-queue, Click and markdown-it. All used GPT-6 Astra with medium reasoning and Fast off. Each configuration ran each task three times, with shared primary acceptance checks and failed runs retained.</p> <p>I develop SunnyCode, so this is a developer-run comparison, not an independent benchmark.</p> <h2> Results </h2> <p>Means across 24 runs per configuration, including failures:</p> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Configuration</th> <th>Tokens</th> <th>Estimated cost/run</th> <th>Checks passed</th> <th>Time</th> </tr> </thead> <tbody> <tr> <td>Codex</td> <td>725,185</td> <td>$1.440</td> <td>21/24</td> <td>222.6 s</td> </tr> <tr> <td>SunnyCode Base</td> <td>281,080</td> <td>$0.785</td> <td>23/24</td> <td>297.1 s</td> </tr> <tr> <td>SunnyCode CodeMode</td> <td>165,566</td> <td>$0.669</td> <td>23/24</td> <td>232.2 s</td> </tr> </tbody> </table></div> <h2> The biggest gap was cached input </h2> <p>Cached input accounted for 96.5% of Codex’s additional recorded tokens relative to CodeMode. These totals include context processed across requests—not just generated code.</p> <p>Because cached input is discounted, CodeMode’s 77.2% reduction in tokens translated to a smaller 53.5% reduction in estimated cost.</p> <p>Codex still had the fastest overall mean completion time. Fewer tokens did not automatically mean faster execution.</p> <p>Costs are API-equivalent estimates, not subscription bills, using frozen rates of $10/M uncached input, $1/M cached input and $50/M output.</p> <h2> What this does—and doesn’t—show </h2> <p>Agent configuration is worth measuring alongside model choice. Separate cached input, uncached input and output, and compare acceptance and time alongside cost.</p> <p>But this is eight tasks, not a general ranking. System prompts differed, Base runs were collected later, and provider/cache conditions were not randomized. Supplemental checks also found defects beyond the primary acceptance results.</p> <p><a href="https://www.sunnycode.pro/blog/reducing-ai-coding-token-usage/" rel="noopener noreferrer">Methodology, per-run results and source data</a></p> ai programming productivity showdev