<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Johmisking</title>
    <description>The latest articles on DEV Community by Johmisking (@jaehyun_cho_0dff271e0d2e5).</description>
    <link>https://dev.to/jaehyun_cho_0dff271e0d2e5</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4152263%2F585d5c8d-9d4a-4345-a948-492267896aca.png</url>
      <title>DEV Community: Johmisking</title>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jaehyun_cho_0dff271e0d2e5"/>
    <language>en</language>
    <item>
      <title>Claude Fable 5.2 costs 40% less than 5.1 per bug fix, not 20%. Opus 5.5 still fixed the same bugs for half the price.</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Fri, 13 Nov 2026 06:00:51 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/claude-fable-52-costs-40-less-than-51-per-bug-fix-not-20-opus-55-still-fixed-the-same-bugs-24ip</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/claude-fable-52-costs-40-less-than-51-per-bug-fix-not-20-opus-55-still-fixed-the-same-bugs-24ip</guid>
      <description>&lt;p&gt;Claude Fable 5.2 came out this week at $8 / $40 per million tokens, 20% below Fable 5.1 and twice the price of Opus 5.5. I wanted two answers: is it cheaper per fixed bug too, and does it fix more bugs than Opus?&lt;/p&gt;

&lt;p&gt;I gave all three models the same 25 real bugs in Claude Code and scored every run with the maintainers' own tests.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Fable 5.2 vs 5.1: 40% cheaper per run&lt;/strong&gt; ($0.73 vs $1.22), twice the list-price cut. It wrote 27% fewer output tokens, read 67% less cache and finished in half the time (median 65 s vs 136 s).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fable 5.2 vs Opus 5.5: the same bugs.&lt;/strong&gt; Both fixed 37 of 50 runs cleanly. The difference is 0 points (95% interval -8 to +8).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Opus 5.5 was the cheapest fix:&lt;/strong&gt; $0.54 per clean fix, against $0.99 for Fable 5.2 and $1.60 for Fable 5.1.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqmlcv61p0i2sl4oesg4m.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqmlcv61p0i2sl4oesg4m.png" alt="Clean fixes and cost per run" width="800" height="307"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;p&gt;The 25 bugs are the ones from my earlier &lt;a href="https://tokensave.app/data/claude-haiku-vs-opus-real-bugs" rel="noopener noreferrer"&gt;Haiku vs Sonnet vs Opus test&lt;/a&gt;: real issues fixed upstream between August and October 2026 in 13 open-source projects (js-yaml, jsdiff, qs, PapaParse, markdown-it, humanize, tomlkit, more-itertools, zod, cachetools, Python-Markdown, pyparsing, marked). 15 are "easy" (the cause sits near the symptom) and 10 are "hard" (the report only shows the symptom).&lt;/p&gt;

&lt;p&gt;Each run starts at the commit before the fix, with the bug report cut down to symptom and expected result. Claude Code 2.1.322, headless, effort medium, tests allowed, no web, no installs. A run counts as a clean fix when every maintainer test for the fix passes and no other test breaks.&lt;/p&gt;

&lt;p&gt;Pre-registered on GitHub before the first run. One dated amendment: the first 21 runs used my whole 5-hour subscription window in 34 minutes, so I cut the plan from 175 to 125 runs (Fable 5.2 × 2, Fable 5.1 × 1, Opus 5.5 × 2 per bug) before scoring anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Fable 5.2&lt;/th&gt;
&lt;th&gt;Fable 5.1&lt;/th&gt;
&lt;th&gt;Opus 5.5&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Clean fixes&lt;/td&gt;
&lt;td&gt;37/50 (74%)&lt;/td&gt;
&lt;td&gt;19/25 (76%)&lt;/td&gt;
&lt;td&gt;37/50 (74%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cost per run&lt;/td&gt;
&lt;td&gt;$0.73&lt;/td&gt;
&lt;td&gt;$1.22&lt;/td&gt;
&lt;td&gt;$0.40&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cost per clean fix&lt;/td&gt;
&lt;td&gt;$0.99&lt;/td&gt;
&lt;td&gt;$1.60&lt;/td&gt;
&lt;td&gt;$0.54&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Output tokens per run&lt;/td&gt;
&lt;td&gt;5,893&lt;/td&gt;
&lt;td&gt;8,124&lt;/td&gt;
&lt;td&gt;6,125&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cache reads per run&lt;/td&gt;
&lt;td&gt;298K&lt;/td&gt;
&lt;td&gt;889K&lt;/td&gt;
&lt;td&gt;440K&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Turns per run&lt;/td&gt;
&lt;td&gt;14.6&lt;/td&gt;
&lt;td&gt;21.0&lt;/td&gt;
&lt;td&gt;17.4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median time&lt;/td&gt;
&lt;td&gt;65 s&lt;/td&gt;
&lt;td&gt;136 s&lt;/td&gt;
&lt;td&gt;67 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Costs are API list prices from Claude Code's own log. On the easy bugs Fable 5.2 cost $0.48 a run against $0.26 for Opus; on the hard ones, $1.11 against $0.61.&lt;/p&gt;

&lt;p&gt;All three models missed the same four bugs: a cachetools class-access case, a js-yaml BOM case, a tomlkit underscore case and a zod numeric-key case. None of them found what the report left out.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;p&gt;25 bugs, 2 runs per bug for Fable 5.2 and Opus, 1 for Fable 5.1, so Fable 5.1's fix rate is the least certain number here. One machine, one prompt style, effort medium only. These are bounded, single-issue bugs; harder multi-file work may separate the models where this didn't.&lt;/p&gt;

&lt;h2&gt;
  
  
  Data
&lt;/h2&gt;

&lt;p&gt;All 135 runs (the 125 in the plan plus 10 finished before the amendment) with the full logs, diffs and scores, CC BY 4.0:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Data page: &lt;a href="https://tokensave.app/data/claude-fable-5-2-real-bugs" rel="noopener noreferrer"&gt;https://tokensave.app/data/claude-fable-5-2-real-bugs&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;GitHub (pre-registration, amendment, scripts): &lt;a href="https://github.com/johnisking/token-cost-by-language/tree/main/claude-fable-5-2-real-bugs" rel="noopener noreferrer"&gt;https://github.com/johnisking/token-cost-by-language/tree/main/claude-fable-5-2-real-bugs&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>claude</category>
      <category>ai</category>
      <category>programming</category>
      <category>productivity</category>
    </item>
    <item>
      <title>I made a free browser tool that takes an AI-generated 3D model from 98k to 8k triangles and re-bakes the textures (no upload)</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Fri, 13 Nov 2026 04:01:04 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/i-made-a-free-browser-tool-that-takes-an-ai-generated-3d-model-from-98k-to-8k-triangles-and-gln</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/i-made-a-free-browser-tool-that-takes-an-ai-generated-3d-model-from-98k-to-8k-triangles-and-gln</guid>
      <description>&lt;p&gt;AI 3D generators give you a model in a minute. Then you open it in an engine and it is 100k triangles with a messy UV layout. I wanted a quick way to make such a model game-ready without opening Blender, so I built one that runs in the browser.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tool:&lt;/strong&gt; &lt;a href="https://gameshipkit.com/tools/ai-3d-model-fixer/?utm_source=devto" rel="noopener noreferrer"&gt;AI 3D Model Fixer&lt;/a&gt; (free, no sign-up, the file never leaves your computer)&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Drop a GLB, glTF, OBJ, FBX or STL file.&lt;/li&gt;
&lt;li&gt;Pick a triangle target and turn on "New UV layout".&lt;/li&gt;
&lt;li&gt;Download a GLB with fewer triangles, a fresh UV layout and re-baked textures.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Numbers from one real test
&lt;/h2&gt;

&lt;p&gt;I generated a turret with TRELLIS.2 on Hugging Face and ran it through the tool.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;98,596 triangles went down to 7,746&lt;/li&gt;
&lt;li&gt;4.7 MB went down to 1.5 MB&lt;/li&gt;
&lt;li&gt;It took about 3 seconds on a desktop GPU&lt;/li&gt;
&lt;li&gt;The result imported cleanly into Godot 4.7.1 with tangents and albedo, normal, roughness and metallic textures&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That is one model on one machine. Yours will differ.&lt;/p&gt;

&lt;h2&gt;
  
  
  How it works
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Reduction:&lt;/strong&gt; meshoptimizer simplifies the mesh.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;New UVs:&lt;/strong&gt; xatlas (compiled to WASM) unwraps the reduced mesh.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Re-bake:&lt;/strong&gt; for every pixel of the new layout the tool finds the nearest point on the original dense mesh and copies colour, roughness, metallic and the surface normal. The lost detail comes back as a tangent-space normal map.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The bug that taught me the most
&lt;/h2&gt;

&lt;p&gt;My first version passed every test on clean sample models. Then I tried a real AI model and it came out with zero triangles.&lt;/p&gt;

&lt;p&gt;The cause: AI models are cut into thousands of small UV islands. Each island has its own copy of the vertices along its border. My code counted connected parts by vertex index, so it saw 3,635 "floating bits" and deleted them as debris. The whole model was debris.&lt;/p&gt;

&lt;p&gt;The fix was to weld vertices by position first and only then look for loose parts. If you write mesh tools, test with ugly real input early. Clean samples hide this.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;The "New UV layout" step is built for static models. I have not tested it on rigged or animated ones.&lt;/li&gt;
&lt;li&gt;Very thin parts can break at low triangle counts.&lt;/li&gt;
&lt;li&gt;I checked Godot only. Unity and Unreal are not verified yet.&lt;/li&gt;
&lt;li&gt;The tool itself does not use AI. It is plain geometry code.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It is free with a daily limit. A paid tier lifts the limit.&lt;/p&gt;

&lt;p&gt;If you try it on a model from Meshy, Tripo or another generator and it breaks, tell me what happened. That is the feedback I need most.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;This post was written with AI assistance. The numbers come from my own test.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>gamedev</category>
      <category>performance</category>
      <category>tools</category>
    </item>
    <item>
      <title>Claude Code moved its Explore subagent off Haiku in 2.1.198. Two env vars get it back for 60% less.</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Sun, 08 Nov 2026 05:26:55 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/claude-code-moved-its-explore-subagent-off-haiku-in-21198-two-env-vars-get-it-back-for-60-less-2i8i</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/claude-code-moved-its-explore-subagent-off-haiku-in-21198-two-env-vars-get-it-back-for-60-less-2i8i</guid>
      <description>&lt;p&gt;In February a post here said Claude Code's built-in Explore subagent is hard-coded to Haiku. I measured something different last week: on 2.1.318, an Opus session's Explore calls all ran on Opus.&lt;/p&gt;

&lt;p&gt;Both were right. I ran 205 headless sessions across 26 Claude Code versions to find out when it changed.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Up to 2.1.197:&lt;/strong&gt; Explore ran on Haiku in every run.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;From 2.1.198:&lt;/strong&gt; Explore runs on whatever model your main session uses.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost on Opus:&lt;/strong&gt; a delegated code search went from $0.18 to $0.37 per run (+105%).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The fix:&lt;/strong&gt; &lt;code&gt;CLAUDE_CODE_SUBAGENT_MODEL=haiku&lt;/code&gt; plus &lt;code&gt;CLAUDE_CODE_SUBAGENT_MODEL_FORCE=1&lt;/code&gt;. Explore went back to Haiku in 36 of 36 runs, cost dropped 60%, and it found the same answers.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnaxm2zi2po33gggds1cf.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnaxm2zi2po33gggds1cf.png" alt="Explore subagent model by version" width="800" height="420"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;p&gt;Each run asks Claude Code a "find every place" question in an open-source repo (undici, fastify, cachetools, Python-Markdown, anyio, zod) and tells it to use the Explore subagent. Headless &lt;code&gt;claude -p&lt;/code&gt;, read-only tools, stream-json logs. The subagent's model comes from its own messages and the per-model cost split in the log.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Version history:&lt;/strong&gt; 18 versions from 2.1.0 to 2.1.318, then a bisection over the npm releases in between. Main model fixed to Opus 4.5, because the API refuses Opus 5.5 below 2.1.280.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Settings test:&lt;/strong&gt; 2.1.318 with Opus 5.5 on 12 questions × 3 runs, under three settings.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Pre-registered on GitHub before the runs, with two dated amendments (one is my own bisection bug, explained below).&lt;/p&gt;

&lt;h2&gt;
  
  
  The settings
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;Explore ran on&lt;/th&gt;
&lt;th&gt;Found&lt;/th&gt;
&lt;th&gt;Cost/run&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Default&lt;/td&gt;
&lt;td&gt;Opus 5.5&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;td&gt;$0.38&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;CLAUDE_CODE_SUBAGENT_MODEL=claude-haiku-5-5&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Opus 5.5&lt;/td&gt;
&lt;td&gt;99%&lt;/td&gt;
&lt;td&gt;$0.37&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;CLAUDE_CODE_SUBAGENT_MODEL=haiku&lt;/code&gt; + &lt;code&gt;CLAUDE_CODE_SUBAGENT_MODEL_FORCE=1&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Haiku 5.5&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;td&gt;$0.15&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Setting the subagent model alone did nothing for Explore. That matches the &lt;a href="https://code.claude.com/docs/en/sub-agents" rel="noopener noreferrer"&gt;sub-agents docs&lt;/a&gt;: &lt;code&gt;CLAUDE_CODE_SUBAGENT_MODEL&lt;/code&gt; is only a default, and the FORCE flag (2.1.257+) is what applies it to every subagent. The docs also mention a project subagent named &lt;code&gt;Explore&lt;/code&gt; with &lt;code&gt;model: haiku&lt;/code&gt;. I didn't test that route.&lt;/p&gt;

&lt;p&gt;To use it, put both in your shell or in &lt;code&gt;settings.json&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"env"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"CLAUDE_CODE_SUBAGENT_MODEL"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"haiku"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"CLAUDE_CODE_SUBAGENT_MODEL_FORCE"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"1"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Note that FORCE applies to every subagent, not only Explore.&lt;/p&gt;

&lt;h2&gt;
  
  
  A mistake I made on the way
&lt;/h2&gt;

&lt;p&gt;My first bisection said the switch was at 2.1.176. The script checked for Haiku with a regex that could never match the log format, so every probe looked like "not Haiku". When I read the raw logs, 2.1.176 through 2.1.186 were all Haiku. I fixed the check, re-ran the bisection and logged it as an amendment. The wrong-step probes are in the data too.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;p&gt;Two questions per version in the history, so it shows which model ran, not how good it was. One machine, one account, one prompt. Defaults can change again.&lt;/p&gt;

&lt;h2&gt;
  
  
  Data
&lt;/h2&gt;

&lt;p&gt;All 205 runs (model, cost split, recall, every answer) are public under CC BY 4.0:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Data page: &lt;a href="https://tokensave.app/data/claude-subagent-model-by-version" rel="noopener noreferrer"&gt;https://tokensave.app/data/claude-subagent-model-by-version&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;GitHub (pre-registration, scripts): &lt;a href="https://github.com/johnisking/token-cost-by-language/tree/main/claude-subagent-model-by-version" rel="noopener noreferrer"&gt;https://github.com/johnisking/token-cost-by-language/tree/main/claude-subagent-model-by-version&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Previous round: &lt;a href="https://dev.to/jaehyun_cho_0dff271e0d2e5/i-told-claude-code-to-use-subagents-216-times-it-found-nothing-more-and-cost-up-to-221-more-14ea"&gt;telling Claude Code to use subagents found nothing more and cost up to 221% more&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>claude</category>
      <category>ai</category>
      <category>productivity</category>
      <category>devops</category>
    </item>
    <item>
      <title>I told Claude Code to use subagents 216 times. It found nothing more and cost up to 221% more.</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Fri, 06 Nov 2026 15:29:44 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/i-told-claude-code-to-use-subagents-216-times-it-found-nothing-more-and-cost-up-to-221-more-14ea</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/i-told-claude-code-to-use-subagents-216-times-it-found-nothing-more-and-cost-up-to-221-more-14ea</guid>
      <description>&lt;p&gt;Claude Code 2.1.318 added a new knob: &lt;strong&gt;subagent effort&lt;/strong&gt; (&lt;code&gt;CLAUDE_CODE_SUBAGENT_EFFORT&lt;/code&gt;). The pitch is easy to believe: hand the searching to an Explore subagent, give it more effort, get a better answer.&lt;/p&gt;

&lt;p&gt;So I measured it. 288 runs later: the answers were the same, the bill was not.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Found:&lt;/strong&gt; 97% to 100% of the right answers in every condition. No condition moved it by as much as 2 points.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost per run:&lt;/strong&gt; Haiku 5.5 +74% to +116%. Opus 5.5 +143% to +221%.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Time per run:&lt;/strong&gt; roughly 2x to 3x.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Subagent model:&lt;/strong&gt; always the same as the main agent. Opus calls an Opus subagent, so you pay Opus prices twice.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhkzdv9adie7vjshbebr7.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhkzdv9adie7vjshbebr7.png" alt="Recall flat, cost up" width="800" height="420"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;p&gt;12 "find every place" questions in 6 open-source repos (undici, fastify, cachetools, Python-Markdown, anyio, zod), at fixed commits. Things like "find every place in lib/ where a RequestAbortedError is created, with the enclosing function."&lt;/p&gt;

&lt;p&gt;The answer key came from a regex over the source plus a parser for the enclosing function or class: 151 items. Claude Code 2.1.318, headless, read-only tools, Haiku 5.5 and Opus 5.5, 3 runs each. 12 × 2 × 4 × 3 = 288.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Condition&lt;/th&gt;
&lt;th&gt;What changed&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;No instruction&lt;/td&gt;
&lt;td&gt;The question only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Subagents, effort low / medium / high&lt;/td&gt;
&lt;td&gt;+ "Use subagents (the Explore agent) for the searching, in parallel where it helps, and write the final answer yourself." and &lt;code&gt;CLAUDE_CODE_SUBAGENT_EFFORT&lt;/code&gt; set to low, medium or high&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;I pre-registered the questions, the answer key (sha256) and a 15-point threshold on GitHub before the first run.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Condition&lt;/th&gt;
&lt;th&gt;Haiku found&lt;/th&gt;
&lt;th&gt;Haiku cost/run&lt;/th&gt;
&lt;th&gt;Opus found&lt;/th&gt;
&lt;th&gt;Opus cost/run&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;No instruction&lt;/td&gt;
&lt;td&gt;98%&lt;/td&gt;
&lt;td&gt;$0.0056&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;td&gt;$0.16&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Subagents, low&lt;/td&gt;
&lt;td&gt;97%&lt;/td&gt;
&lt;td&gt;$0.0098 (+74%)&lt;/td&gt;
&lt;td&gt;99%&lt;/td&gt;
&lt;td&gt;$0.39 (+143%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Subagents, medium&lt;/td&gt;
&lt;td&gt;98%&lt;/td&gt;
&lt;td&gt;$0.011 (+87%)&lt;/td&gt;
&lt;td&gt;99%&lt;/td&gt;
&lt;td&gt;$0.51 (+221%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Subagents, high&lt;/td&gt;
&lt;td&gt;99%&lt;/td&gt;
&lt;td&gt;$0.012 (+116%)&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;td&gt;$0.48 (+204%)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Higher subagent effort didn't buy better searches either. For Opus, medium cost more than high.&lt;/p&gt;

&lt;p&gt;And with no instruction, Claude Code never reached for a subagent on its own: 0 of 72 runs here, 0 of 504 in my previous dataset.&lt;/p&gt;

&lt;h2&gt;
  
  
  The surprise: headless runs answer late
&lt;/h2&gt;

&lt;p&gt;In 2.1.318 the Explore agent runs in the background. In headless mode (&lt;code&gt;claude -p&lt;/code&gt;) that means one run can emit &lt;strong&gt;several &lt;code&gt;result&lt;/code&gt; events&lt;/strong&gt;:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;"The search agent is running in the background. I'll write up the list once its results come back."&lt;/li&gt;
&lt;li&gt;The actual answer.&lt;/li&gt;
&lt;li&gt;Sometimes: "Nothing to act on: the lookup is finished [...] so I've stopped the autonomous loop."&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In 210 of 216 subagent runs, the first &lt;code&gt;result&lt;/code&gt; held no answer. A script that reads only the first one gets &lt;strong&gt;0% to 9%&lt;/strong&gt; of the answers. A script that reads only the last one can also miss, because the last one is sometimes the "stopped the loop" note. 4 Haiku runs gave the answer and then never exited until my 20-minute timeout.&lt;/p&gt;

&lt;p&gt;My own scorer read the last &lt;code&gt;result&lt;/code&gt; at first, which is how I found this. I changed it to join all &lt;code&gt;result&lt;/code&gt; events and logged the change in the pre-registration. Both readings are in the data.&lt;/p&gt;

&lt;p&gt;If you call Claude Code from CI or a script and ask it to use subagents, read every &lt;code&gt;result&lt;/code&gt; event, not just one.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to do
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Don't add "use subagents" for ordinary code search.&lt;/strong&gt; Here it only added cost and time.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;If you do use them on Opus, know the subagent is Opus too.&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Leave subagent effort at the default&lt;/strong&gt; unless you've measured a gain on your own tasks.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;p&gt;Both models were near the ceiling with no instruction, so there was little room to gain. Mid-sized repos; a huge monorepo might be different. One prompt wording, one Claude Code version, 3 runs per cell.&lt;/p&gt;

&lt;h2&gt;
  
  
  Data
&lt;/h2&gt;

&lt;p&gt;All 288 runs (recall under three readings, cost, time, subagent calls and models, and every answer text) are public under CC BY 4.0:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Data page: &lt;a href="https://tokensave.app/data/claude-subagent-effort" rel="noopener noreferrer"&gt;https://tokensave.app/data/claude-subagent-effort&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;GitHub (pre-registration, answer key, scorer): &lt;a href="https://github.com/johnisking/token-cost-by-language/tree/main/claude-subagent-effort" rel="noopener noreferrer"&gt;https://github.com/johnisking/token-cost-by-language/tree/main/claude-subagent-effort&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Have you seen subagents actually help on a search? Tell me the task and I'll test it next round.&lt;/p&gt;

</description>
      <category>claude</category>
      <category>ai</category>
      <category>productivity</category>
      <category>programming</category>
    </item>
    <item>
      <title>Stop telling Claude to "handle edge cases". I ran it 504 times.</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Tue, 03 Nov 2026 12:28:28 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/stop-telling-claude-to-handle-edge-cases-i-ran-it-504-times-4k1d</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/stop-telling-claude-to-handle-edge-cases-i-ran-it-504-times-4k1d</guid>
      <description>&lt;p&gt;You've probably written this into a Claude Code prompt:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Make sure you handle all the edge cases."&lt;br&gt;
"Write the tests first."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Or you bumped effort to high. So did I. So I measured whether it does anything. 504 runs later: it mostly makes the bill bigger.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Haiku 5.5:&lt;/strong&gt; "handle edge cases" + "read the existing tests first" raised cost per run by &lt;strong&gt;134%&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Opus 5.5:&lt;/strong&gt; the edge-case line alone raised cost per run by &lt;strong&gt;60%&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;effort high:&lt;/strong&gt; +60% for Haiku, +22% for Opus&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Share of the cases the bug report left out that the fix actually handled:&lt;/strong&gt; no condition moved it by more than &lt;strong&gt;4 points&lt;/strong&gt;. Effort high made it slightly &lt;em&gt;worse&lt;/em&gt; for both models.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fau90wo8drcs3scuxa2yy.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fau90wo8drcs3scuxa2yy.png" alt="Pass rate flat, cost up" width="800" height="420"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;p&gt;21 real bugs fixed upstream between August and October 2026 in 12 open-source projects: cachetools, zod, marked, js-yaml, anyio, undici, fastify, p-queue, python-dateutil, Python-Markdown, qs, tomlkit. Claude Code 2.1.295, headless, Haiku 5.5 and Opus 5.5, four conditions, 3 runs each. 21 × 2 × 4 × 3 = 504.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Condition&lt;/th&gt;
&lt;th&gt;What changed&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Baseline&lt;/td&gt;
&lt;td&gt;Bug report cut to symptom, repro, expected result&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Effort high&lt;/td&gt;
&lt;td&gt;Same report, &lt;code&gt;--effort high&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Edge-case prompt&lt;/td&gt;
&lt;td&gt;+ "Before changing code, list the inputs and variants this report does not mention but the fix must also handle. Write tests for them first, then fix."&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Edge-case + read tests&lt;/td&gt;
&lt;td&gt;+ "Read the existing tests for the code you change before you start."&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Scoring used the maintainers' own tests for each fix. &lt;strong&gt;Before the first run&lt;/strong&gt; I labelled every test case as "shown in the report" or "left out" and pre-registered it on GitHub with sha256 hashes, so I couldn't move the goalposts later. The bar for calling something an effect: 20 points or more.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;p&gt;Share of the left-out cases passed (mean per bug):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Condition&lt;/th&gt;
&lt;th&gt;Haiku 5.5&lt;/th&gt;
&lt;th&gt;Opus 5.5&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Baseline&lt;/td&gt;
&lt;td&gt;84%&lt;/td&gt;
&lt;td&gt;88%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Effort high&lt;/td&gt;
&lt;td&gt;82%&lt;/td&gt;
&lt;td&gt;86%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Edge-case prompt&lt;/td&gt;
&lt;td&gt;84%&lt;/td&gt;
&lt;td&gt;91%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Edge-case + read tests&lt;/td&gt;
&lt;td&gt;86%&lt;/td&gt;
&lt;td&gt;84%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;API cost per run:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Condition&lt;/th&gt;
&lt;th&gt;Haiku&lt;/th&gt;
&lt;th&gt;Opus&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Baseline&lt;/td&gt;
&lt;td&gt;$0.050&lt;/td&gt;
&lt;td&gt;$0.51&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Effort high&lt;/td&gt;
&lt;td&gt;$0.081 (+60%)&lt;/td&gt;
&lt;td&gt;$0.62 (+22%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Edge-case prompt&lt;/td&gt;
&lt;td&gt;$0.10 (+108%)&lt;/td&gt;
&lt;td&gt;$0.82 (+60%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Edge-case + read tests&lt;/td&gt;
&lt;td&gt;$0.12 (+134%)&lt;/td&gt;
&lt;td&gt;$0.81 (+58%)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Median time per run went from 121s to 299s for Haiku and 120s to 271s for Opus. The models did exactly what they were told: listed variants, wrote tests. It just didn't turn into better fixes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why it didn't work: the bug decided, not the prompt
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Easy bugs were handled anyway.&lt;/strong&gt; On 12 of 21 bugs both models passed every left-out case in every condition. Nothing to gain.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hard bugs stayed missed.&lt;/strong&gt; On a js-yaml bug (a BOM before &lt;em&gt;each&lt;/em&gt; document in a stream, not just the first), &lt;strong&gt;0 of 504 runs&lt;/strong&gt; handled it. Asking "list the variants the report doesn't mention" didn't make the model think of that variant.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;"Read the tests" sometimes hurt.&lt;/strong&gt; On a Python-Markdown backtick bug, Opus passed the left-out case in 2 of 3 runs at baseline and 0 of 3 with the read-tests line.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What to do instead
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Name the edge cases yourself.&lt;/strong&gt; Both models passed over 94% of the cases the report &lt;em&gt;did&lt;/em&gt; show. They don't fail at following instructions; they fail at imagining what you didn't say.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Leave effort at the default for bug fixing.&lt;/strong&gt; Here, high only added cost.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Default to Haiku.&lt;/strong&gt; Across 252 runs each: Haiku $0.088 per run, Opus $0.69, about 87% less. Opus was a few points ahead and broke other tests less often, so it still earns its price on hard, multi-place bugs in big codebases.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;p&gt;21 bugs, 3 runs per cell. A 20-point effect is ruled out; a few points is not. One prompt wording per condition; a different wording or agent might do better.&lt;/p&gt;

&lt;h2&gt;
  
  
  Data
&lt;/h2&gt;

&lt;p&gt;All 504 runs (scores, cost, time, tokens) are public under CC BY 4.0, with the pre-registration and the scoring code:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Dataset (DOI): &lt;a href="https://doi.org/10.5281/zenodo.23796306" rel="noopener noreferrer"&gt;https://doi.org/10.5281/zenodo.23796306&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Pre-registration: &lt;a href="https://github.com/johnisking/token-cost-by-language/blob/main/claude-report-edge-cases/PREREG.md" rel="noopener noreferrer"&gt;https://github.com/johnisking/token-cost-by-language/blob/main/claude-report-edge-cases/PREREG.md&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What do you add to your Claude prompts that actually works? Drop it in the comments and I'll test it in the next round.&lt;/p&gt;

</description>
      <category>claude</category>
      <category>ai</category>
      <category>programming</category>
      <category>testing</category>
    </item>
    <item>
      <title>I built 60+ free browser tools for game devs. Here is what I learned.</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Tue, 03 Nov 2026 08:02:29 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/i-built-60-free-browser-tools-for-game-devs-here-is-what-i-learned-3fd3</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/i-built-60-free-browser-tools-for-game-devs-here-is-what-i-learned-3fd3</guid>
      <description>&lt;p&gt;A year ago I could not write code. I did not even know what a game engine was.&lt;br&gt;
Today I run &lt;a href="https://gameshipkit.com" rel="noopener noreferrer"&gt;GameShipKit&lt;/a&gt;, a set of 63 small browser tools for game developers.&lt;br&gt;
I built all of it by describing what I wanted to an AI and testing the result. Here is what that taught me.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why small tools
&lt;/h2&gt;

&lt;p&gt;Every time I made a game I hit the same small jobs.&lt;br&gt;
Turn a drawing into a sprite. Try a hit-stop value. Make a fire effect.&lt;br&gt;
Each job is too small for a big program, but it still eats an hour.&lt;br&gt;
So I started making one tool for each job.&lt;/p&gt;

&lt;h2&gt;
  
  
  Four tools I use the most
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Image to Pixel Art
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgsszehllgcjv4hh4zebo.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgsszehllgcjv4hh4zebo.png" alt="Image to Pixel Art converter turning a mushroom drawing into a 64 px sprite" width="800" height="410"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;A plain resize turns art into blurry mush. This tool keeps the strongest colour of each pixel instead.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Drop a drawing, a 3D render or a photo.&lt;/li&gt;
&lt;li&gt;Pick the width, like 32 or 64 pixels.&lt;/li&gt;
&lt;li&gt;Lower the colour count and keep "Remove stray single pixels" on.&lt;/li&gt;
&lt;li&gt;Download the PNG.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://gameshipkit.com/tools/image-to-pixel-art/" rel="noopener noreferrer"&gt;Try Image to Pixel Art&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Hit Feel Designer
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxccjlmkm2rmumka42kbi.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxccjlmkm2rmumka42kbi.gif" alt="Hit Feel Designer hitting a test dummy with hit stop, shake and damage numbers" width="640" height="359"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Hit a test dummy and tune what makes a hit feel good.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Start from a preset like Arcade or Heavy brawler.&lt;/li&gt;
&lt;li&gt;Press J, K and L for light, heavy and critical hits.&lt;/li&gt;
&lt;li&gt;Move the "Juice amount" slider until it feels right.&lt;/li&gt;
&lt;li&gt;Turn "Juice" off to compare. The difference is huge.&lt;/li&gt;
&lt;li&gt;Download the settings as JSON and the hit sounds as WAV.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://gameshipkit.com/tools/hit-feel-designer/" rel="noopener noreferrer"&gt;Try Hit Feel Designer&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Particle Effect Builder
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiym5bo7w90qkmzarepx3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiym5bo7w90qkmzarepx3.png" alt="Particle Effect Builder showing the fire preset" width="800" height="469"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Fire, smoke, explosions, sparks, coins and more, live in the browser.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Pick a preset.&lt;/li&gt;
&lt;li&gt;Change the particle shape, count and colours with sliders.&lt;/li&gt;
&lt;li&gt;Export a looping sprite sheet that works in any engine.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://gameshipkit.com/tools/particle-effect-builder/" rel="noopener noreferrer"&gt;Try Particle Effect Builder&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  4. 2D Character Rigging
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fc55pghklv0xprenfgk8x.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fc55pghklv0xprenfgk8x.gif" alt="A pixel art minotaur animated with bones from a single image" width="490" height="469"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Turn one character image into bone animations.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Drop one character image.&lt;/li&gt;
&lt;li&gt;The tool places a skeleton. Drag the joints to fix them.&lt;/li&gt;
&lt;li&gt;Pick a motion like idle, walk, attack or hurt.&lt;/li&gt;
&lt;li&gt;Export a sprite sheet or a GIF.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://gameshipkit.com/tools/character-rigging/" rel="noopener noreferrer"&gt;Try 2D Character Rigging&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 1: everything runs in the browser
&lt;/h2&gt;

&lt;p&gt;No sign-up and no upload. The files stay on your own computer.&lt;br&gt;
This made the tools simple to build and simple to trust.&lt;br&gt;
It also means there is no server cost, so the free tools can stay free.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 2: test the export in the real engine
&lt;/h2&gt;

&lt;p&gt;Many tools export straight to Godot 4 or Unity.&lt;br&gt;
At first I only checked that the file looked right. That was a mistake.&lt;br&gt;
When I finally opened every export in Godot 4.7 and Unity 6.3, I found real bugs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A generated Unity shader failed to compile. A loop variable had the same name as the shader input.&lt;/li&gt;
&lt;li&gt;Inventory icons were cut from the wrong place. The importer read the scaled texture size instead of the source size.&lt;/li&gt;
&lt;li&gt;A wave spawner spawned zero enemies. The prefab list was built too early and came up empty.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;None of these showed up in the browser. All of them would have broken someone's project.&lt;br&gt;
Now no export ships until it runs inside the engine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 3: AI writes the code, I check the behavior
&lt;/h2&gt;

&lt;p&gt;The AI is fast, but it is confident even when it is wrong.&lt;br&gt;
My job became writing clear requests and checking what actually happens.&lt;br&gt;
I cannot read every line, so I test every feature by using it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is next
&lt;/h2&gt;

&lt;p&gt;I am adding tools that people ask for.&lt;br&gt;
If there is a small game dev job that keeps eating your time, tell me in the comments.&lt;br&gt;
I might build it next.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://gameshipkit.com" rel="noopener noreferrer"&gt;https://gameshipkit.com&lt;/a&gt;&lt;/p&gt;

</description>
      <category>showdev</category>
      <category>gamedev</category>
      <category>godot</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Claude Haiku vs Opus on 25 real bugs: 52 vs 54 clean fixes, 95% cheaper</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Sun, 25 Oct 2026 01:21:25 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/claude-haiku-vs-opus-on-25-real-bugs-52-vs-54-clean-fixes-95-cheaper-cg9</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/claude-haiku-vs-opus-on-25-real-bugs-52-vs-54-clean-fixes-95-cheaper-cg9</guid>
      <description>&lt;p&gt;Last week I measured where Claude Haiku 5.5 falls behind Opus. The answer was one hard bug in the marked parser, where Sonnet and Opus caught an edge case the bug report left out and Haiku mostly did not. One bug is thin evidence, so this time I gave the three models 25 real bugs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Short version:&lt;/strong&gt; Haiku fixed as many bugs cleanly as Opus (52 and 54 of 75 runs) at about 95% less cost. Opus was ahead on the harder bugs, mostly because it broke other tests less often. And where a bug report left a case out, all three models missed it the same way.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setup
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;25 bugs&lt;/strong&gt; fixed upstream between August and October 2026 in 13 open-source projects: js-yaml, jsdiff, qs, PapaParse, markdown-it, marked, zod, humanize, tomlkit, more-itertools, cachetools, Python-Markdown, pyparsing.&lt;/li&gt;
&lt;li&gt;Each run started from &lt;strong&gt;the commit before the fix&lt;/strong&gt;, with dependencies installed and no git history, so the fix could not be looked up.&lt;/li&gt;
&lt;li&gt;The prompt was &lt;strong&gt;the bug report cut down to the symptom&lt;/strong&gt;: what happens, how to reproduce it, what should happen. Cause analysis and proposed fixes were removed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Claude Code 2.1.295&lt;/strong&gt;, headless, effort medium. The agent could read files, edit them and run the project's tests. No installs, no network.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;3 runs per bug and model&lt;/strong&gt;, 225 runs in total.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Scoring:&lt;/strong&gt; I kept only the run's changes to the library source (tests it wrote were thrown away) and ran &lt;strong&gt;the maintainers' own tests for the fix&lt;/strong&gt; against it, plus the project's tests to catch anything it broke.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The bugs came in two rounds. The &lt;strong&gt;easy round&lt;/strong&gt; (15 bugs) had cause and symptom close together. The &lt;strong&gt;hard round&lt;/strong&gt; (10 bugs) used bugs where the report only shows a symptom, often far from the cause.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgy4jdnjsr7su65ehoblx.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgy4jdnjsr7su65ehoblx.png" alt="One square per run: 25 bugs by Haiku, Sonnet and Opus" width="800" height="833"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;"Clean fix" means every maintainer test case for the fix passes and no other test breaks.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Haiku 5.5&lt;/th&gt;
&lt;th&gt;Sonnet 5.5&lt;/th&gt;
&lt;th&gt;Opus 5.5&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Clean fixes, easy round&lt;/td&gt;
&lt;td&gt;40/45&lt;/td&gt;
&lt;td&gt;35/45&lt;/td&gt;
&lt;td&gt;38/45&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Clean fixes, hard round&lt;/td&gt;
&lt;td&gt;12/30&lt;/td&gt;
&lt;td&gt;14/30&lt;/td&gt;
&lt;td&gt;16/30&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Clean fixes, all&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;52/75&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;49/75&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;54/75&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Runs that broke other tests (hard)&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cost per run (API prices)&lt;/td&gt;
&lt;td&gt;$0.019&lt;/td&gt;
&lt;td&gt;$0.14&lt;/td&gt;
&lt;td&gt;$0.38&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;All 75 runs together&lt;/td&gt;
&lt;td&gt;$1.41&lt;/td&gt;
&lt;td&gt;$10.83&lt;/td&gt;
&lt;td&gt;$28.53&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;On 22 of the 25 bugs, Haiku and Opus had exactly the same number of clean fixes. The three where they differed:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;marked, a link inside link text:&lt;/strong&gt; Opus fixed it cleanly 2 of 3 times, Haiku 0 of 3. Every Haiku run broke the spec test for that exact example.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;markdown-it, an unclosed link label hiding a code span:&lt;/strong&gt; Opus 3 of 3, Haiku 1 of 3. Two Haiku runs fixed the bug but broke two neighbouring tests.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;js-yaml, a block scalar with a whitespace-only body:&lt;/strong&gt; Haiku 3 of 3, Opus 1 of 3, Sonnet 0 of 3.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One more bug is worth a mention: a &lt;strong&gt;zod&lt;/strong&gt; bug that needed numeric record keys handled in several places of the JSON Schema output. No run fixed all of it, but Opus passed 14 of the 24 test cases over its three runs, Haiku 9 and Sonnet 5.&lt;/p&gt;

&lt;h2&gt;
  
  
  The part no model got
&lt;/h2&gt;

&lt;p&gt;On three hard bugs, every run of every model fixed the example in the report and missed the same case the report did not mention:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;cachetools:&lt;/strong&gt; the report showed one way of calling a cached method through its class. The maintainers also tested every other variant of the decorator. No run handled those.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;js-yaml:&lt;/strong&gt; the report was about a BOM at the start of a file. The maintainers also tested a BOM before each document in a multi-document stream. No run handled it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Python-Markdown:&lt;/strong&gt; every run fixed the reported HTML comment and broke 3 to 5 of the maintainers' tests for comments after other inline HTML.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Last time, Sonnet and Opus were better at finding a case like this. Across 25 bugs that did not hold. Paying for a bigger model did not buy the missing case; reading the maintainers' tests would have.&lt;/p&gt;

&lt;h2&gt;
  
  
  Is the difference real?
&lt;/h2&gt;

&lt;p&gt;Probably not detectable at this size, and I want to be careful here.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Haiku vs Opus, all bugs:&lt;/strong&gt; Fisher's exact test on clean fixes p = 0.86. Resampling whole bugs, the 95% interval for Haiku minus Opus is −11 to +5 percentage points. So there is no evidence that Opus fixes more bugs, but a 10-point gap cannot be ruled out.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hard round only:&lt;/strong&gt; the interval is −33 to 0 points. Opus may really be better on bugs like these; 10 bugs are not enough to say.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sonnet&lt;/strong&gt; passed a smaller share of test cases than Opus overall (paired permutation test p = 0.006), the only clearly significant difference.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;To show that two models are equal within ±10 points would take several hundred runs per model, on different bugs. I will keep adding bugs with the same method.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I take from it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;For bugs where the cause is near the symptom, Haiku is enough.&lt;/strong&gt; Same clean-fix count as Opus, about 95% cheaper.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;For bugs that touch several places, Opus is the safer pick.&lt;/strong&gt; It broke other tests in half as many hard-round runs as Haiku (5 against 10).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No model finds what the report leaves out.&lt;/strong&gt; In the hard round, 47% to 60% of each model's runs were not clean fixes. That is where review time should go, whichever model you use.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Limitations
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;3 runs per bug and model, 25 bugs. Treat the model differences as hints.&lt;/li&gt;
&lt;li&gt;Unlike last week's runs, every run here could run the project's tests.&lt;/li&gt;
&lt;li&gt;I wrote the tomlkit underscore report myself from the upstream issue, which had no reproduction. Every run of every model passed the same 20 of its 29 cases.&lt;/li&gt;
&lt;li&gt;On Windows, zod's CLAUDE.md (a symlink) did not extract, so zod runs had no project instructions.&lt;/li&gt;
&lt;li&gt;Timing tests were not counted as broken, because they fail at random under load.&lt;/li&gt;
&lt;li&gt;The projects are public, so the models may have seen similar code in training. All fixes are from August to October 2026.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Data (225 runs: cases passed, tests broken, time and cost per run; CSV and JSON, CC BY 4.0): &lt;a href="https://tokensave.app/data/claude-haiku-vs-opus-real-bugs" rel="noopener noreferrer"&gt;https://tokensave.app/data/claude-haiku-vs-opus-real-bugs&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;DOI: &lt;a href="https://doi.org/10.5281/zenodo.23619927" rel="noopener noreferrer"&gt;10.5281/zenodo.23619927&lt;/a&gt; (also on Hugging Face, Kaggle and GitHub)&lt;/p&gt;

&lt;p&gt;Related: Let's Data Science interviewed me about the earlier 45-run study in this series, where Haiku, Sonnet and Opus built the same game: &lt;a href="https://letsdatascience.com/news/tokensave-tells-lds-what-45-claude-coding-runs-reveal-0d3335f3" rel="noopener noreferrer"&gt;TokenSave tells LDS what 45 Claude coding runs reveal&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Disclosure: I run TokenSave, which publishes these measurements. No support or credits from Anthropic.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>claude</category>
      <category>llm</category>
      <category>programming</category>
    </item>
    <item>
      <title>What Claude Code sends before you type: 58 versions measured, from July 2025 to this week</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Sat, 24 Oct 2026 12:18:09 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/what-claude-code-sends-before-you-type-58-versions-measured-from-july-2025-to-this-week-282k</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/what-claude-code-sends-before-you-type-58-versions-measured-from-july-2025-to-this-week-282k</guid>
      <description>&lt;p&gt;Every Claude Code request starts with the same fixed block: the system prompt and the definitions of every built-in tool. You pay for it before your own words are counted, and it is a big reason why short sessions and subagents burn through limits faster than you'd expect. I wanted to know how that block has changed over time, so I installed old versions and measured it.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I measured
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;58 Claude Code versions installed from npm, one folder each, from 1.0.60 (July 2025) to 2.1.310 (October 23, 2026)&lt;/li&gt;
&lt;li&gt;Each one got the one-letter prompt "a" with &lt;code&gt;claude -p&lt;/code&gt;, model Claude Haiku 5.5, an empty folder and a fresh config folder (no CLAUDE.md, plugins or settings)&lt;/li&gt;
&lt;li&gt;I recorded the input tokens of the first model request: uncached + cache read + cache write&lt;/li&gt;
&lt;li&gt;MCP off, 3 runs per version. A second pass with my 13 claude.ai connectors left on, 1 run per version&lt;/li&gt;
&lt;li&gt;One Windows laptop. Earlier versions measured on October 21, the 12 newest on October 24 with the same script and login&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Result
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Version&lt;/th&gt;
&lt;th&gt;Released&lt;/th&gt;
&lt;th&gt;Tokens before you type&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1.0.60&lt;/td&gt;
&lt;td&gt;2025-07-24&lt;/td&gt;
&lt;td&gt;19,498&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.0.0&lt;/td&gt;
&lt;td&gt;2025-09-29&lt;/td&gt;
&lt;td&gt;19,420&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.0&lt;/td&gt;
&lt;td&gt;2026-01-07&lt;/td&gt;
&lt;td&gt;23,957&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.100&lt;/td&gt;
&lt;td&gt;2026-04-10&lt;/td&gt;
&lt;td&gt;33,305&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.150 (peak)&lt;/td&gt;
&lt;td&gt;2026-05-23&lt;/td&gt;
&lt;td&gt;36,309&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.250&lt;/td&gt;
&lt;td&gt;2026-08-27&lt;/td&gt;
&lt;td&gt;28,451&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.292&lt;/td&gt;
&lt;td&gt;2026-10-06&lt;/td&gt;
&lt;td&gt;29,701&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.293&lt;/td&gt;
&lt;td&gt;2026-10-07&lt;/td&gt;
&lt;td&gt;18,055&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.301&lt;/td&gt;
&lt;td&gt;2026-10-15&lt;/td&gt;
&lt;td&gt;18,171&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.307&lt;/td&gt;
&lt;td&gt;2026-10-21&lt;/td&gt;
&lt;td&gt;18,697&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.1.310&lt;/td&gt;
&lt;td&gt;2026-10-23&lt;/td&gt;
&lt;td&gt;18,841&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Three things stand out:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Ten months of growth.&lt;/strong&gt; From 19,498 to 36,309 tokens (+86%) between July 2025 and May 2026, while built-in tools went from 15 to 27.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One release cut it by 39%.&lt;/strong&gt; Version 2.1.293 on October 7 dropped from 29,701 to 18,055 tokens with the same number of tools. That puts it back where it was in mid-2025.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Since then it has been creeping back up.&lt;/strong&gt; Over the 15 releases after 2.1.293 it rose to 18,841 (+4%), in steps of 100 to 400 tokens. 2.1.310 added a 25th built-in tool (Monitor).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Every run of the same version gave exactly the same number, so these are not noisy averages.&lt;/p&gt;

&lt;h2&gt;
  
  
  The connector surprise
&lt;/h2&gt;

&lt;p&gt;Apps you connect on claude.ai also show up in Claude Code. In versions 2.1.83 to 2.1.140 (late March to mid May 2026), all of their tools were sent with every request. With my 13 connected apps, that was 420 to 426 extra tools and about 395,000 tokens for the prompt "a". Since 2.1.145 they are loaded only when needed, and in every version I measured this month the number is the same with connectors on or off.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this means in practice
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;After the first request, this block is usually read from the prompt cache at a tenth of the input price or less. It costs the most in short sessions, in subagents (each starts with its own copy), and after the cache expires.&lt;/li&gt;
&lt;li&gt;If your Claude Code is old, updating is the cheapest token saving you can make.&lt;/li&gt;
&lt;li&gt;Your own number will be higher: CLAUDE.md files, MCP servers, plugins and skills all add to it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;One model (Haiku 5.5), one prompt, one Windows machine. Other models and operating systems can get a different system prompt, so absolute numbers will differ; the trend is the point.&lt;/li&gt;
&lt;li&gt;Old versions were run against today's API and account, so features the server switches on per account may change what an old version sends.&lt;/li&gt;
&lt;li&gt;Versions before 1.0.60 would not install on Windows.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Update (October 25): Codex CLI over the same weeks
&lt;/h2&gt;

&lt;p&gt;I ran the same test on OpenAI's Codex CLI: all 18 releases from 0.155.0 (September 17) to 0.164.0 (October 23), the prompt "a" with &lt;code&gt;codex exec&lt;/code&gt;, gpt-6-luna, reasoning effort low, 3 runs each.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Sep 17&lt;/th&gt;
&lt;th&gt;Oct 23&lt;/th&gt;
&lt;th&gt;Change&lt;/th&gt;
&lt;th&gt;Releases where it changed&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Codex CLI&lt;/td&gt;
&lt;td&gt;13,147&lt;/td&gt;
&lt;td&gt;13,019&lt;/td&gt;
&lt;td&gt;−1%&lt;/td&gt;
&lt;td&gt;3 of 17&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code&lt;/td&gt;
&lt;td&gt;28,906&lt;/td&gt;
&lt;td&gt;18,841&lt;/td&gt;
&lt;td&gt;−35%&lt;/td&gt;
&lt;td&gt;12 of 21 measured steps&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;ul&gt;
&lt;li&gt;Codex barely moved: three small changes of 4 to 100 tokens.&lt;/li&gt;
&lt;li&gt;Don't compare the two totals directly. They use different tokenizers, and for the same code Claude's counts about 55% more tokens than OpenAI's. The point is how each changes from release to release.&lt;/li&gt;
&lt;li&gt;Older Codex versions could not be measured: with a ChatGPT plan login the server refuses every model they can use.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Codex CSV: &lt;a href="https://tokensave.app/data/codex-cli-baseline-tokens-by-version.csv" rel="noopener noreferrer"&gt;https://tokensave.app/data/codex-cli-baseline-tokens-by-version.csv&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Data
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Full table, per-version CSV, scripts and raw logs (CC BY 4.0, DOI 10.5281/zenodo.23527278): &lt;a href="https://tokensave.app/data/claude-code-baseline-tokens-by-version" rel="noopener noreferrer"&gt;https://tokensave.app/data/claude-code-baseline-tokens-by-version&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Related: what the rest of the harness costs (subagents +128%, minimal setup −64%): &lt;a href="https://tokensave.app/data/claude-code-harness-cost" rel="noopener noreferrer"&gt;https://tokensave.app/data/claude-code-harness-cost&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>claude</category>
      <category>ai</category>
      <category>programming</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Where Claude Haiku falls behind Opus: 163 coding runs with hidden tests</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Fri, 23 Oct 2026 16:09:04 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/i-tried-to-find-where-claude-haiku-stops-being-good-enough-for-coding-it-took-a-real-bug-to-find-17g7</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/i-tried-to-find-where-claude-haiku-stops-being-good-enough-for-coding-it-took-a-real-bug-to-find-17g7</guid>
      <description>&lt;p&gt;Most "which model should I use for coding" advice is a guess. I wanted a line: the point where the cheap model (Claude Haiku 5.5) starts falling behind and you should pay for Sonnet or Opus.&lt;/p&gt;

&lt;p&gt;So I ran 163 headless coding runs on one Windows laptop, across five agents, on tasks that get harder level by level, and scored every result with hidden tests.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Agents:&lt;/strong&gt; Claude Code with Haiku 5.5, Sonnet 5.5 and Opus 5.5; Codex with gpt-6-luna; Google's Antigravity CLI with Gemini 4 Argon. All at medium effort, each run in a fresh folder.&lt;/p&gt;

&lt;h2&gt;
  
  
  The levels
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Level&lt;/th&gt;
&lt;th&gt;Task&lt;/th&gt;
&lt;th&gt;Hidden checks&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;L2&lt;/td&gt;
&lt;td&gt;Five-level brick game with pause, level skip and saved progress&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;L3&lt;/td&gt;
&lt;td&gt;Multi-file expense tracker with CSV and summary modules&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;L4&lt;/td&gt;
&lt;td&gt;Budget app: 5 modules, 3 hash-routed views, budgets, reports, 50-step undo&lt;/td&gt;
&lt;td&gt;23&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;L5A&lt;/td&gt;
&lt;td&gt;An existing 1,032-line app with 6 planted bugs (symptoms only) + 3 new features&lt;/td&gt;
&lt;td&gt;38&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;L5B&lt;/td&gt;
&lt;td&gt;Spreadsheet engine: precedence, functions, cycle detection, $absolute copy, undo&lt;/td&gt;
&lt;td&gt;38&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;L6&lt;/td&gt;
&lt;td&gt;Three real bugs fixed in the marked Markdown parser in Sep to Oct 2026&lt;/td&gt;
&lt;td&gt;upstream test + full spec suite&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For L6, each run started from the commit before the real fix, got a user-style bug report, and was scored by the test the maintainers added with the fix plus marked's whole spec suite (about 1,900 checks). I copied back only the run's &lt;code&gt;src/&lt;/code&gt; folder, so editing tests could not help.&lt;/p&gt;

&lt;h2&gt;
  
  
  Levels 2 to 5: no line at all
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Agent&lt;/th&gt;
&lt;th&gt;L2&lt;/th&gt;
&lt;th&gt;L3&lt;/th&gt;
&lt;th&gt;L4&lt;/th&gt;
&lt;th&gt;L5A&lt;/th&gt;
&lt;th&gt;L5B&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Haiku 5.5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;4/4&lt;/td&gt;
&lt;td&gt;2/2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sonnet 5.5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;4/4&lt;/td&gt;
&lt;td&gt;2/2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Opus 5.5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;4/4&lt;/td&gt;
&lt;td&gt;2/2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Codex&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;4/5&lt;/td&gt;
&lt;td&gt;4/5&lt;/td&gt;
&lt;td&gt;3/4&lt;/td&gt;
&lt;td&gt;1/1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini 4 Argon&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;5/5&lt;/td&gt;
&lt;td&gt;2/2&lt;/td&gt;
&lt;td&gt;1/1&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;(runs with full marks / runs)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;With a clear spec, Haiku never failed.&lt;/strong&gt; It built the spreadsheet engine with cycle detection, and in L5A it fixed all six bugs from symptoms alone, added the features and kept all 23 old tests passing. The easy and medium real bugs in L6 were also solved by almost every run, Haiku and Sonnet in 14 to 20 seconds.&lt;/p&gt;

&lt;h2&gt;
  
  
  The hard bug: where the models split
&lt;/h2&gt;

&lt;p&gt;The hard L6 bug: a code span ending in a backslash, like &lt;code&gt;`C:\Users\`&lt;/code&gt;, stopped &lt;code&gt;**bold**&lt;/code&gt; later on the line from working. The maintainers' test for the fix has 7 cases, and the unfixed code already passes 3.&lt;/p&gt;

&lt;p&gt;No run fixed it completely. But the runs did not fail in the same way:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F64koc0efehq8goiu2qs0.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F64koc0efehq8goiu2qs0.png" alt="Test cases passed per run on the hard bug" width="800" height="440"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Tests&lt;/th&gt;
&lt;th&gt;Cases passed (of 7), per run&lt;/th&gt;
&lt;th&gt;Broke other tests&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Haiku 5.5&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;5, 5, 5, 5, 5&lt;/td&gt;
&lt;td&gt;1 run (15 tests)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Haiku 5.5&lt;/td&gt;
&lt;td&gt;allowed&lt;/td&gt;
&lt;td&gt;5, 6, 5, 5, 5&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sonnet 5.5&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;6, 6, 5, 6, 6&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sonnet 5.5&lt;/td&gt;
&lt;td&gt;allowed&lt;/td&gt;
&lt;td&gt;6, 6, 6, 5, 6&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Opus 5.5&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;6, 6, 6, 5, 0*&lt;/td&gt;
&lt;td&gt;1 run&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Opus 5.5&lt;/td&gt;
&lt;td&gt;allowed&lt;/td&gt;
&lt;td&gt;6, 6, 5, 5, 6&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Codex&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;3, 3, 4, 4, 4&lt;/td&gt;
&lt;td&gt;3 runs (2 tests each)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini 4 Argon&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;3, 3, 3&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;* That Opus run left an invalid regular expression in the source, so marked threw on load and every test failed.&lt;/p&gt;

&lt;p&gt;Every run fixed the examples in the bug report. What was left were code spans that themselves contain a backtick, such as &lt;code&gt;_&lt;/code&gt;&lt;code&gt;_&lt;/code&gt;&lt;code&gt;&lt;/code&gt;`, which the report did not mention. &lt;strong&gt;Sonnet passed 6 of the 7 cases in 8 of 10 runs, Opus in 6 of 10, Haiku in 1 of 10.&lt;/strong&gt; That one unreported edge case is the line I was looking for.&lt;/p&gt;

&lt;h2&gt;
  
  
  "They couldn't run the tests" was the obvious objection
&lt;/h2&gt;

&lt;p&gt;In the first round no agent could run marked's tests: no packages were installed, and Claude Code ran without shell access. A real developer would run the suite while fixing the bug, so I ran each Claude model 5 more times with the dependencies preinstalled and the prompt allowing &lt;code&gt;node&lt;/code&gt;, &lt;code&gt;npm test&lt;/code&gt; / &lt;code&gt;npm run&lt;/code&gt; and read-only git (installs and the network stayed off).&lt;/p&gt;

&lt;p&gt;Every one of those 15 runs ran the spec suite, and most wrote their own reproduction scripts. &lt;strong&gt;The number of cases they got right did not change.&lt;/strong&gt; The one thing that looked different was damage: without tests, 2 of 15 Claude runs broke something else (the Haiku run that broke 15 spec tests and the Opus run that stopped marked from loading); with tests, 0 of 15 did. With so few runs that gap could easily be chance, so I would not claim tests prevent it.&lt;/p&gt;

&lt;p&gt;So on this bug, running tests did not make any model think of the case the report left out. It may help avoid breaking other code, but these runs are too few to show that.&lt;/p&gt;

&lt;p&gt;I also tried Codex with tests allowed, but its Windows sandbox blocked esbuild from building marked in all 5 runs, so those runs are excluded. Gemini had used up its quota.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cost
&lt;/h2&gt;

&lt;p&gt;Claude Code reports an API-price cost for every run. Average per run:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Haiku 5.5&lt;/th&gt;
&lt;th&gt;Sonnet 5.5&lt;/th&gt;
&lt;th&gt;Opus 5.5&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;All levels&lt;/td&gt;
&lt;td&gt;$0.026&lt;/td&gt;
&lt;td&gt;$0.19&lt;/td&gt;
&lt;td&gt;$0.45&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hard bug, no tests&lt;/td&gt;
&lt;td&gt;$0.012&lt;/td&gt;
&lt;td&gt;$0.12&lt;/td&gt;
&lt;td&gt;$0.39&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hard bug, tests allowed&lt;/td&gt;
&lt;td&gt;$0.031&lt;/td&gt;
&lt;td&gt;$0.22&lt;/td&gt;
&lt;td&gt;$0.48&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Haiku cost about 94% less per run than Opus over all levels.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this means in practice
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Clear spec, Haiku is enough.&lt;/strong&gt; For building something from a clear spec, or fixing bugs whose symptoms are described, Haiku got the same full marks as Opus at about 94% less cost per run.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A real bug with an incomplete report, use Sonnet or Opus.&lt;/strong&gt; They caught the extra edge case far more often than Haiku (8 and 6 of 10 runs vs 1 of 10). On this bug Sonnet did best and was also cheaper than Opus ($0.12 to $0.22 vs $0.39 to $0.48 per run), but with 10 runs each the Sonnet vs Opus gap could be chance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Whatever the model, let it run the tests.&lt;/strong&gt; It did not find the missing case here, and no run with tests broke other code (2 of 15 without tests did, too few runs to be sure it is the tests).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Still review edge cases yourself.&lt;/strong&gt; No model fixed the bug completely, and the cases they all missed were the ones only the maintainers thought of.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  A note on Antigravity
&lt;/h2&gt;

&lt;p&gt;Antigravity's headless mode ends the whole turn the first time a command is denied. With read-only allow-rules, several Gemini runs read the code for minutes and then stopped on a &lt;code&gt;node -e&lt;/code&gt; check before editing (on Windows a quoted command needs an exact rule). With full permissions, Gemini solved the easy and medium bugs too, in 2 to 11 minutes; one hard run hit its 30-minute limit without an edit.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limitations
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;The split between models rests on one hard bug from one project, 10 runs per Claude model.&lt;/li&gt;
&lt;li&gt;Few runs per cell at levels 2 to 5 (1 to 5), and those prompts list every id and function name the tests use, which is easier than real work.&lt;/li&gt;
&lt;li&gt;The tests-allowed comparison covers the three Claude models only.&lt;/li&gt;
&lt;li&gt;The tools differ in more than the model (sandbox, permissions, how they stop on a denied command).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Data (163 runs, scores, test cases, time, tokens and cost per run, CC BY 4.0): &lt;a href="https://tokensave.app/data/ai-coding-limit-line" rel="noopener noreferrer"&gt;https://tokensave.app/data/ai-coding-limit-line&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Related: Let's Data Science interviewed me about the earlier 45-run study in this series, where Haiku, Sonnet and Opus built the same game: &lt;a href="https://letsdatascience.com/news/tokensave-tells-lds-what-45-claude-coding-runs-reveal-0d3335f3" rel="noopener noreferrer"&gt;TokenSave tells LDS what 45 Claude coding runs reveal&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Disclosure: I run TokenSave, which publishes these measurements. No funding or credits from Anthropic, OpenAI or Google.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>coding</category>
      <category>llm</category>
      <category>testing</category>
    </item>
    <item>
      <title>Same code, about 60% more output tokens on Claude: 80 runs across Claude Code, Codex and Antigravity in 5 languages</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Fri, 23 Oct 2026 03:05:51 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/same-code-about-60-more-output-tokens-on-claude-80-runs-across-claude-code-codex-and-2a7k</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/same-code-about-60-more-output-tokens-on-claude-80-runs-across-claude-code-codex-and-2a7k</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Update (Oct 23):&lt;/strong&gt; Antigravity now has 5 runs per language as well, so the dataset has 100 runs. Opus in Antigravity came out at 602 output tokens per 1,000 characters (was 596), and the gap to Gemini and GPT is still about 60%. 99 of 100 games worked. Current numbers: &lt;a href="https://tokensave.app/data/output-tokens-per-character-by-model" rel="noopener noreferrer"&gt;https://tokensave.app/data/output-tokens-per-character-by-model&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Yesterday I compared Google's new Antigravity CLI with Claude Code and Codex. One number kept bothering me: Claude Opus 5.5 wrote shorter games than Gemini but did not use proportionally fewer output tokens. So I divided output tokens by the size of the file each run wrote, across four tool and model pairs and five prompt languages.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I measured
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;The same brick-breaker task ("a single index.html with 2 levels, a score and 3 lives, keyboard and mouse"), prompted in English, Korean, Japanese, Spanish and Ukrainian&lt;/li&gt;
&lt;li&gt;Claude Code 2.1.295 + Opus 5.5 and Codex CLI 0.162 + gpt-6-luna: 5 runs per language (from my earlier prompt-language test, same versions)&lt;/li&gt;
&lt;li&gt;Antigravity CLI 1.4.5 with Gemini 4 Argon and with Claude Opus 5.5: 3 runs per language (24 new runs today plus yesterday's English runs)&lt;/li&gt;
&lt;li&gt;Medium effort everywhere, Windows 11, every run in an empty folder&lt;/li&gt;
&lt;li&gt;Every game played by a script in a headless browser: score, level 2, lives 3 to 0&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Result: output tokens per 1,000 characters of code
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Tool + model&lt;/th&gt;
&lt;th&gt;EN&lt;/th&gt;
&lt;th&gt;KO&lt;/th&gt;
&lt;th&gt;JA&lt;/th&gt;
&lt;th&gt;ES&lt;/th&gt;
&lt;th&gt;UK&lt;/th&gt;
&lt;th&gt;Average&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code + Opus 5.5&lt;/td&gt;
&lt;td&gt;554&lt;/td&gt;
&lt;td&gt;624&lt;/td&gt;
&lt;td&gt;581&lt;/td&gt;
&lt;td&gt;614&lt;/td&gt;
&lt;td&gt;584&lt;/td&gt;
&lt;td&gt;591&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Opus 5.5&lt;/td&gt;
&lt;td&gt;574&lt;/td&gt;
&lt;td&gt;590&lt;/td&gt;
&lt;td&gt;606&lt;/td&gt;
&lt;td&gt;612&lt;/td&gt;
&lt;td&gt;597&lt;/td&gt;
&lt;td&gt;596&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Gemini 4 Argon&lt;/td&gt;
&lt;td&gt;367&lt;/td&gt;
&lt;td&gt;380&lt;/td&gt;
&lt;td&gt;370&lt;/td&gt;
&lt;td&gt;364&lt;/td&gt;
&lt;td&gt;378&lt;/td&gt;
&lt;td&gt;372&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Codex + gpt-6-luna&lt;/td&gt;
&lt;td&gt;367&lt;/td&gt;
&lt;td&gt;386&lt;/td&gt;
&lt;td&gt;385&lt;/td&gt;
&lt;td&gt;366&lt;/td&gt;
&lt;td&gt;376&lt;/td&gt;
&lt;td&gt;376&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Opus needed about 60% more output tokens per character than Gemini, and about 57% more than GPT. The interesting part is that Opus came out almost the same in two very different tools (591 and 596). That points at the model's tokenizer, not the tool around it. The language of the prompt barely mattered.&lt;/p&gt;

&lt;p&gt;Output tokens are the expensive ones. On the API, Opus 5.5 charges five times more for output than for input, so this gap goes straight into the bill, on top of any difference in how much code each model decides to write.&lt;/p&gt;

&lt;p&gt;This also fits an earlier test of mine, where the same source code in 10 programming languages needed 43% to 59% more Claude tokens than GPT tokens.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prompt language: almost no effect on input
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Input tokens&lt;/th&gt;
&lt;th&gt;EN&lt;/th&gt;
&lt;th&gt;KO&lt;/th&gt;
&lt;th&gt;JA&lt;/th&gt;
&lt;th&gt;ES&lt;/th&gt;
&lt;th&gt;UK&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code + Opus 5.5&lt;/td&gt;
&lt;td&gt;96,462*&lt;/td&gt;
&lt;td&gt;78,638&lt;/td&gt;
&lt;td&gt;79,266&lt;/td&gt;
&lt;td&gt;79,030&lt;/td&gt;
&lt;td&gt;79,787&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Codex + gpt-6-luna&lt;/td&gt;
&lt;td&gt;51,674&lt;/td&gt;
&lt;td&gt;48,538&lt;/td&gt;
&lt;td&gt;48,660&lt;/td&gt;
&lt;td&gt;48,201&lt;/td&gt;
&lt;td&gt;48,816&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Gemini 4 Argon&lt;/td&gt;
&lt;td&gt;13,041&lt;/td&gt;
&lt;td&gt;13,289&lt;/td&gt;
&lt;td&gt;12,408&lt;/td&gt;
&lt;td&gt;12,629&lt;/td&gt;
&lt;td&gt;13,447&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Opus 5.5&lt;/td&gt;
&lt;td&gt;9,822&lt;/td&gt;
&lt;td&gt;11,316&lt;/td&gt;
&lt;td&gt;9,983&lt;/td&gt;
&lt;td&gt;9,945&lt;/td&gt;
&lt;td&gt;10,958&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;*Two of the five English Claude Code runs took an extra step.&lt;/p&gt;

&lt;p&gt;The prompt is a few dozen tokens; the rest of every request is the tool's own instructions. Which tool you use decides the input size, not which language you write in.&lt;/p&gt;

&lt;p&gt;One more detail: in Antigravity, Opus wrote longer games for Korean and Ukrainian prompts (about 31% and 20% more characters than in English), so its total output rose there. Per character it stayed the same.&lt;/p&gt;

&lt;h2&gt;
  
  
  Did the games work?
&lt;/h2&gt;

&lt;p&gt;79 of 80 did, and every game showed its text in the prompt language. The one failure is a Ukrainian Codex game from the earlier test.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;One task: a browser game in HTML, CSS and JavaScript. Other code may tokenize differently&lt;/li&gt;
&lt;li&gt;Output tokens include the short final reply and the tool-call wrapping, not only the code&lt;/li&gt;
&lt;li&gt;3 to 5 runs per cell, measured between 2026-10-18 and 10-23 with the same tool versions&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Update (October 24): checked with all three tokenizers
&lt;/h2&gt;

&lt;p&gt;I counted 76 of the game files with each company's own tokenizer, no matter which model wrote them (Claude via Claude Code, Gemini 4 Argon via the Google AI Studio counter, GPT with o200k).&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Code written by&lt;/th&gt;
&lt;th&gt;Claude&lt;/th&gt;
&lt;th&gt;Gemini&lt;/th&gt;
&lt;th&gt;GPT&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Opus 5.5 (Claude Code)&lt;/td&gt;
&lt;td&gt;512&lt;/td&gt;
&lt;td&gt;399&lt;/td&gt;
&lt;td&gt;334&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini 4 Argon&lt;/td&gt;
&lt;td&gt;451&lt;/td&gt;
&lt;td&gt;346&lt;/td&gt;
&lt;td&gt;290&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;gpt-6-luna (Codex)&lt;/td&gt;
&lt;td&gt;534&lt;/td&gt;
&lt;td&gt;407&lt;/td&gt;
&lt;td&gt;338&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Tokens per 1,000 characters. For the same code, Claude's tokenizer needs 55% more tokens than GPT's (every file between +47% and +65%) and 29% more than Gemini's.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Against GPT, the gap is the tokenizer: both models wrote code of about the same density and added a similar amount beyond the code (14% and 11%).&lt;/li&gt;
&lt;li&gt;Against Gemini, the tokenizer explains about half. The rest is that Gemini's own code needed 13% to 15% fewer tokens per character under all three tokenizers, and it added less beyond the code (7%).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Per-file counts: &lt;a href="https://tokensave.app/data/output-tokens-per-character-tokenizer-check.csv" rel="noopener noreferrer"&gt;https://tokensave.app/data/output-tokens-per-character-tokenizer-check.csv&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Data
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Full data, per-run CSV and method (CC BY 4.0, DOI 10.5281/zenodo.23585139): &lt;a href="https://tokensave.app/data/output-tokens-per-character-by-model" rel="noopener noreferrer"&gt;https://tokensave.app/data/output-tokens-per-character-by-model&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Yesterday's three-way comparison: &lt;a href="https://tokensave.app/data/antigravity-cli-vs-claude-code-vs-codex" rel="noopener noreferrer"&gt;https://tokensave.app/data/antigravity-cli-vs-claude-code-vs-codex&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>claude</category>
      <category>gemini</category>
      <category>llm</category>
    </item>
    <item>
      <title>Same model, 68% to 88% fewer tokens: Antigravity CLI vs Claude Code vs Codex, measured</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Thu, 22 Oct 2026 15:23:39 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/same-model-68-to-88-fewer-tokens-antigravity-cli-vs-claude-code-vs-codex-measured-36k1</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/same-model-68-to-88-fewer-tokens-antigravity-cli-vs-claude-code-vs-codex-measured-36k1</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Update (Oct 23):&lt;/strong&gt; I added more game runs, so every tool now has 5 (52 runs in total). The same Opus 5.5 now reads 71% to 89% fewer input tokens in Antigravity than in Claude Code (was 68% to 88%), and all 20 games worked. Current numbers: &lt;a href="https://tokensave.app/data/antigravity-cli-vs-claude-code-vs-codex" rel="noopener noreferrer"&gt;https://tokensave.app/data/antigravity-cli-vs-claude-code-vs-codex&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Google stopped serving Gemini CLI to personal accounts on June 18, 2026 and moved people to its new Antigravity CLI (&lt;code&gt;agy&lt;/code&gt;). Antigravity can run Claude models too, which makes a rare test possible: the same model, Claude Opus 5.5, in two different coding tools. So I ran Antigravity CLI, Claude Code and Codex on the same prompts, 44 runs in total, and checked every game they built in a headless browser.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I measured
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Windows 11, 2026-10-22. Every run in its own empty folder, one after another&lt;/li&gt;
&lt;li&gt;Claude Code 2.1.295 with Opus 5.5 (Max plan)&lt;/li&gt;
&lt;li&gt;Codex CLI 0.162 with gpt-6-luna (ChatGPT plan)&lt;/li&gt;
&lt;li&gt;Antigravity CLI 1.4.5 with Gemini 4 Argon and with Claude Opus 5.5 (personal Google account)&lt;/li&gt;
&lt;li&gt;Startup test: "Reply with the single word OK." with 0, 1, 10 and 30 dummy MCP tools, 2 runs each&lt;/li&gt;
&lt;li&gt;Task: a brick-breaker game in one index.html with 2 levels, a score and 3 lives, medium effort, 3 runs each&lt;/li&gt;
&lt;li&gt;Token counts are what each CLI reports in its JSON output&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Result 1: tokens before doing anything
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Tool&lt;/th&gt;
&lt;th&gt;Input tokens for "OK"&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code + Opus 5.5&lt;/td&gt;
&lt;td&gt;35,781&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Codex + gpt-6-luna&lt;/td&gt;
&lt;td&gt;14,818&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Gemini 4 Argon&lt;/td&gt;
&lt;td&gt;6,745&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Opus 5.5&lt;/td&gt;
&lt;td&gt;5,326 (+5,167 cache reads)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Most of this is each tool's own instructions and tool definitions, which go out again with every request.&lt;/p&gt;

&lt;h2&gt;
  
  
  Result 2: the same coding task
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Tool&lt;/th&gt;
&lt;th&gt;Input&lt;/th&gt;
&lt;th&gt;Output&lt;/th&gt;
&lt;th&gt;Seconds&lt;/th&gt;
&lt;th&gt;Lines&lt;/th&gt;
&lt;th&gt;Games working&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code + Opus 5.5&lt;/td&gt;
&lt;td&gt;78,671&lt;/td&gt;
&lt;td&gt;7,313&lt;/td&gt;
&lt;td&gt;58&lt;/td&gt;
&lt;td&gt;451&lt;/td&gt;
&lt;td&gt;3 of 3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Codex + gpt-6-luna&lt;/td&gt;
&lt;td&gt;55,680&lt;/td&gt;
&lt;td&gt;4,563&lt;/td&gt;
&lt;td&gt;68&lt;/td&gt;
&lt;td&gt;42 (minified)&lt;/td&gt;
&lt;td&gt;3 of 3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Gemini 4 Argon&lt;/td&gt;
&lt;td&gt;13,041 (+6,796 cache)&lt;/td&gt;
&lt;td&gt;6,373&lt;/td&gt;
&lt;td&gt;81&lt;/td&gt;
&lt;td&gt;608&lt;/td&gt;
&lt;td&gt;3 of 3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Opus 5.5&lt;/td&gt;
&lt;td&gt;9,822 (+15,381 cache)&lt;/td&gt;
&lt;td&gt;4,378&lt;/td&gt;
&lt;td&gt;42&lt;/td&gt;
&lt;td&gt;236&lt;/td&gt;
&lt;td&gt;3 of 3&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;With the same Opus 5.5, Antigravity read 68% to 88% fewer input tokens than Claude Code, wrote 40% fewer output tokens and finished 27% faster. Claude Code's games were longer (451 lines against 236), and Claude Code put each run at $0.37 at API prices. Antigravity and Codex show no price on a subscription.&lt;/p&gt;

&lt;p&gt;All 12 games worked: the score rises when bricks break, level 2 appears, lives go 3, 2, 1, 0. My automatic check could not read the score in two of the Codex games, so I checked those by hand; both scored.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why a range, not one number
&lt;/h2&gt;

&lt;p&gt;Antigravity's JSON reports &lt;code&gt;input_tokens&lt;/code&gt; and &lt;code&gt;cache_read_tokens&lt;/code&gt; separately, and its documentation does not say whether one contains the other. In the Opus runs the cache reads (15,381) were larger than the input (9,822), so there they cannot be inside it. I show both: the reported input as the low end (88% fewer) and input plus cache reads as the high end (68% fewer). Even the high end is far below Claude Code.&lt;/p&gt;

&lt;h2&gt;
  
  
  MCP tools barely matter any more
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;30 MCP tools added&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code&lt;/td&gt;
&lt;td&gt;+388 tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Codex&lt;/td&gt;
&lt;td&gt;+0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Gemini&lt;/td&gt;
&lt;td&gt;+582&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Antigravity + Opus&lt;/td&gt;
&lt;td&gt;+417&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;In Antigravity one tool already added about 280 to 460 tokens and the next 29 added little. All four setups now seem to load a tool's full definition only when it is used.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to take away
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;The coding tool around the model matters as much as the model. The same Opus 5.5 read a fraction of the input in Antigravity.&lt;/li&gt;
&lt;li&gt;Fewer tokens is not the same as a better result. Claude Code wrote longer games; all of them worked in every tool.&lt;/li&gt;
&lt;li&gt;If you were on Gemini CLI with a personal account, Antigravity CLI is the replacement, and it is lean.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;2 runs per startup test, 3 per game. One task, one machine, one day&lt;/li&gt;
&lt;li&gt;Each tool counts cache differently, so compare within a tool's own column with care&lt;/li&gt;
&lt;li&gt;I cannot see how Antigravity sets up Opus 5.5 behind its &lt;code&gt;claude-opus-5-5-medium&lt;/code&gt; name&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Data
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Full data, per-run CSV and method (CC BY 4.0, DOI 10.5281/zenodo.23569397): &lt;a href="https://tokensave.app/data/antigravity-cli-vs-claude-code-vs-codex" rel="noopener noreferrer"&gt;https://tokensave.app/data/antigravity-cli-vs-claude-code-vs-codex&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;The earlier Claude Code vs Codex test: &lt;a href="https://tokensave.app/data/claude-code-vs-codex-tokens" rel="noopener noreferrer"&gt;https://tokensave.app/data/claude-code-vs-codex-tokens&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>claude</category>
      <category>gemini</category>
      <category>productivity</category>
    </item>
    <item>
      <title>How many tokens does Blender MCP really use? 72 Claude Code runs, and a racing scene for $0.47</title>
      <dc:creator>Johmisking</dc:creator>
      <pubDate>Thu, 22 Oct 2026 12:17:50 +0000</pubDate>
      <link>https://dev.to/jaehyun_cho_0dff271e0d2e5/how-many-tokens-does-blender-mcp-really-use-72-claude-code-runs-and-a-racing-scene-for-047-5hl3</link>
      <guid>https://dev.to/jaehyun_cho_0dff271e0d2e5/how-many-tokens-does-blender-mcp-really-use-72-claude-code-runs-and-a-racing-scene-for-047-5hl3</guid>
      <description>&lt;p&gt;People say hooking Blender up to Claude through MCP eats tokens. I couldn't find a single measurement, so I took the most popular server, MCP for Blender, and ran it 72 times in Claude Code: describing scenes, building furniture, recolouring and checking with screenshots, two worst cases, and one showcase, a racing car with a chase camera. I also ran every task through the small proxy I built for browser MCP, now version 0.3, to see whether it helps here.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I measured
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Claude Code 2.1.295, Opus 5.5, effort medium, Windows 11 with a GTX 1650 (2026-10-22)&lt;/li&gt;
&lt;li&gt;Blender 5.2.2 LTS, MCP for Blender via &lt;code&gt;uvx mcp-for-blender&lt;/code&gt; (add-on 1.8, telemetry off)&lt;/li&gt;
&lt;li&gt;Only the Blender server attached; web, file and shell tools disabled, and the server's 3D-generation and asset-download tools too&lt;/li&gt;
&lt;li&gt;Scenes made by my own script, identical every run: a room with 13 objects, a city block with 580 objects, and an empty scene&lt;/li&gt;
&lt;li&gt;Every run checked afterwards by reading the saved .blend file: the table and four chairs exist, the chairs are really red, the animation is 120 frames, the camera follows the car. 72 of 72 passed.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Result 1: everyday tasks cost $0.16 to $0.23
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Task (5 runs each)&lt;/th&gt;
&lt;th&gt;Input tokens&lt;/th&gt;
&lt;th&gt;Cost&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Describe a small room&lt;/td&gt;
&lt;td&gt;69,606&lt;/td&gt;
&lt;td&gt;$0.16&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Describe a 580-object city block&lt;/td&gt;
&lt;td&gt;115,868&lt;/td&gt;
&lt;td&gt;$0.19&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Build a low-poly table and four chairs&lt;/td&gt;
&lt;td&gt;125,563&lt;/td&gt;
&lt;td&gt;$0.23&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Make the chairs red, then check with a screenshot&lt;/td&gt;
&lt;td&gt;124,649&lt;/td&gt;
&lt;td&gt;$0.20&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Look at the table from four sides&lt;/td&gt;
&lt;td&gt;112,118&lt;/td&gt;
&lt;td&gt;$0.21&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;(Cost at Opus 5.5 API prices, as reported by Claude Code.)&lt;/p&gt;

&lt;p&gt;The largest text result in these 25 runs was 2,347 characters. The current MCP for Blender is already careful: scene info lists 20 objects per call by default and screenshots default to 768 pixels. Most of each request (about 25,000 tokens) is Claude Code's own instructions, sent again at every step.&lt;/p&gt;

&lt;h2&gt;
  
  
  Result 2: a racing scene cost $0.47
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F55s3l2ap9awf2d9rh12g.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F55s3l2ap9awf2d9rh12g.jpg" alt="Six frames from the racing scenes" width="800" height="900"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I asked for a low-poly racing car of its own design, a circuit with a corner, a 5-second animation through the corner and a chase camera. Claude did it in about 10 requests and 93 seconds, for $0.47 on average ($0.42 to $0.56 over 5 runs). All 10 runs, with and without the proxy, produced a working animation. I rendered them afterwards in EEVEE; that took about 90 seconds per clip on the GTX 1650 and is not part of the measured cost.&lt;/p&gt;

&lt;h2&gt;
  
  
  Result 3: the one expensive case is a long script output
&lt;/h2&gt;

&lt;p&gt;I asked Claude to print every property of all 580 objects. The script returned 122,373 characters. Claude Code refused it as too large, so Claude ran it again in four chunks of about 32,000 characters each, and all four stayed in the context:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;242,782 input tokens, $0.87 per task, more than four times an everyday task&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Printing all 19,042 vertices of a sphere did not get that far: Claude Code refused it, and Claude reported that the vertices were printed and stopped there: $0.17.&lt;/p&gt;

&lt;h2&gt;
  
  
  The proxy: tokensave-mcp-diet 0.3
&lt;/h2&gt;

&lt;p&gt;It sits in front of the server and passes every tool through. For Blender, version 0.3 does two things:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Long text results&lt;/strong&gt; (over 8,000 characters) keep their first 2,000 and last 1,000 characters plus a summary of the hidden middle. On the property dump it read: "568 lines (132,418 characters) hidden ... 568 lines like: &lt;code&gt;name=Building ## | type=MESH | location=(#, #, #) ...&lt;/code&gt;". The full text stays in memory, and a new tool &lt;code&gt;result_detail&lt;/code&gt; returns any part by line range or search.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Smaller images by default&lt;/strong&gt;: tools with a &lt;code&gt;max_size&lt;/code&gt; get 512 pixels unless the model asks for more than 1,000.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Setup:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"mcpServers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"blender"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"command"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"npx"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"args"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"-y"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"github:johnisking/tokensave-mcp-diet"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"--"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"uvx"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"mcp-for-blender"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Task&lt;/th&gt;
&lt;th&gt;Cost, as shipped&lt;/th&gt;
&lt;th&gt;With the proxy&lt;/th&gt;
&lt;th&gt;Change&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Print every property of 580 objects (3 runs)&lt;/td&gt;
&lt;td&gt;$0.873&lt;/td&gt;
&lt;td&gt;$0.166&lt;/td&gt;
&lt;td&gt;−81%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Racing scene (5 runs)&lt;/td&gt;
&lt;td&gt;$0.471&lt;/td&gt;
&lt;td&gt;$0.445&lt;/td&gt;
&lt;td&gt;−6%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Everyday tasks (5 runs each)&lt;/td&gt;
&lt;td&gt;$0.16 to $0.23&lt;/td&gt;
&lt;td&gt;$0.15 to $0.21&lt;/td&gt;
&lt;td&gt;−4% to −8%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;To be honest about the rest: the proxy cut image tokens on the racing scene by 62%, but images were a small part of the input, so the scene cost only 6% less, and that is within run-to-run noise. The small-room description, where the proxy changed nothing at all, came out 4% cheaper too. So the everyday differences are noise. The proxy is a safety net for the long-output case, not a discount on every task.&lt;/p&gt;

&lt;p&gt;One bug I hit on the way: MCP for Blender also returns each result as &lt;code&gt;structuredContent&lt;/code&gt;, and Claude Code shows that instead of the text, so my first diet runs of the long-output task were not shortened at all. Version 0.3 now shortens those fields too, and I re-ran those tasks after the fix.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to take away
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Blender MCP in Claude Code is not a token hog for normal work. Cost follows the number of steps, and most of each step is Claude Code itself.&lt;/li&gt;
&lt;li&gt;Watch out for scripts that print a lot. One property dump cost more than four normal tasks. Ask for counts, summaries or a file instead of "print everything".&lt;/li&gt;
&lt;li&gt;If you use a client that passes large tool results straight through, the same dump would land in the context in one piece. That is where a proxy like this pays off most.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;One model, one machine, one client. 5 runs per task (3 for the two worst cases).&lt;/li&gt;
&lt;li&gt;Costs vary by run because Claude takes a different number of steps each time.&lt;/li&gt;
&lt;li&gt;Image tokens are estimated from image size (width × height / 750); the input totals are exact.&lt;/li&gt;
&lt;li&gt;MCP for Blender changes fast; this is the 2026-10 version.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Data
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Full data, method and per-run CSV (CC BY 4.0, DOI 10.5281/zenodo.23562484): &lt;a href="https://tokensave.app/data/blender-mcp-token-cost" rel="noopener noreferrer"&gt;https://tokensave.app/data/blender-mcp-token-cost&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;tokensave-mcp-diet (MIT): &lt;a href="https://github.com/johnisking/tokensave-mcp-diet" rel="noopener noreferrer"&gt;https://github.com/johnisking/tokensave-mcp-diet&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;The same test for browser MCP: &lt;a href="https://tokensave.app/data/browser-mcp-token-cost" rel="noopener noreferrer"&gt;https://tokensave.app/data/browser-mcp-token-cost&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>mcp</category>
      <category>blender</category>
      <category>claude</category>
      <category>ai</category>
    </item>
  </channel>
</rss>
