There is a trick going around for cutting Claude Code bills: render bulky context (system prompt, tool docs, logs, old history) as dense PNG images instead of text. An image is billed by its pixel size, not by how much text is inside it, so a page packed with small characters can cost far fewer tokens than the same text.
The open-source proxy pxpipe does exactly this and reports a 59-70% lower bill. Its own README is honest that the method is lossy. I wanted an independent number for both sides: how many tokens it really saves, and what the model can still read back.
The test
- Six documents, 10,600 to 23,200 characters: one of our articles in English, Korean and Japanese, a JavaScript file, a generated JSON API response and a generated 230-line server log
- Planted in each: five lines like "ticket T123 uses build hash 3f9a1c7e02bd and a timeout of 19043 ms". For the JSON and the log I also asked about five records that were already there
- Three ways to send it: plain text, pxpipe's own output (page images plus its text "factsheet" of exact identifiers), and the images alone
- Claude Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1 through Claude Code, 2 runs each, 144 calls, exact-match scoring
- System prompt: answer from the document only, null if unsure
Tokens: the saving is real
| Document | As text | As pxpipe images | Change |
|---|---|---|---|
| English prose | 5,870 | 2,100 | -64% |
| Korean prose | 9,483 | 2,142 | -77% |
| Japanese prose | 8,678 | 2,374 | -73% |
| Code | 7,586 | 1,906 | -75% |
| JSON | 8,274 | 2,022 | -76% |
| Server log | 13,053 | 2,935 | -78% |
(Opus 5.5; Haiku and Sonnet were the same within a few tokens. Fable's denser layout: -68% to -81%.)
The more tokens a language costs as text, the more an image saves. Korean and Japanese gained the most.
Exact values: this is where it breaks
As plain text, every model answered every item correctly. Through pxpipe:
| Model | Planted ids | Planted numbers | Other ids in JSON/log |
|---|---|---|---|
| Haiku 5.5 | 21/60 | 22/60 | 0/20 |
| Sonnet 5.5 | 49/60 | 40/60 | 0/20 |
| Opus 5.5 | 58/60 | 44/60 | 0/20 |
| Fable 5.1 | 41/60 | 28/60 | 0/20 |
Three things stood out:
- The ids came back through the factsheet, not the image. All planted ids happened to be in pxpipe's text list. The log had 235 ids and only 94 made the list. The ids that were left out were read 0 of 80 times, across all four models.
- Numbers were the weak spot. Opus 5.5, the best here, got 44 of 60. In Korean prose it got 3 of 10.
- Wrong answers looked right. Despite "null if unsure", models sometimes returned a plausible wrong value: 94392 read as 9392, a number that is not in the document, or (Fable 5.1) a real value from the factsheet attached to the wrong ticket.
So when is it worth it?
Use image context for what the model needs to understand, not for what it needs to copy. A long article to summarize, a design doc to review, old conversation history: fine. Ids, hashes, prices, line numbers, code you want edited: keep them as text.
I added a button for it
Because the saving is large for long non-English text, I added an experimental Text → image button to the free token counter at https://tokensave.app. It renders your text into PNG pages in the browser (nothing is uploaded), estimates the image tokens for the GPT, Claude or Gemini model you picked next to the text tokens, and lets you copy or download the pages. A 9,000-character Korean article came out at about 1,400 image tokens on Claude versus 8,229 as text.
Two caveats. The button uses a larger, plainer font than pxpipe's 5x8 pixel cells, so it saves somewhat less and I have not measured its read-back accuracy yet. And the warning above applies: do not use it for anything that has to be exact.
Limits of this test
Two runs per cell and six documents, so treat single cells as examples. I measured exact recall only; pxpipe's own tests report good results on gist and arithmetic questions, which I did not repeat. In normal use pxpipe images older history and the system prompt while recent turns stay text, so the saving on a full bill depends on your session.
Full tables, method and CSV (CC BY 4.0): https://tokensave.app/data/text-as-image-token-cost
Corpus, rendered pages, raw answers and scripts: https://github.com/johnisking/token-cost-by-language/tree/main/text-as-image
Top comments (5)
This is a genuinely useful benchmark because you measured both sides instead of just the headline saving — the detail that the ids came back through the factsheet and not the image (94 of 235 made the list, the rest read 0/80) is the whole story. It means the technique isn't really "the model reads text from pixels," it's "you built a lossy text extractor and shipped a PNG alongside it," and the PNG is doing less than the saving implies. The practical read for anyone tempted by this: it's viable for recall-tolerant context — prose, old conversation history, boilerplate tool docs — and actively dangerous for anything where an exact identifier or number has to survive (logs, JSON, config, financial figures). The "wrong answers looked right" finding is the real hazard: a 78% token cut that silently fabricates a build hash or a timeout value isn't a cost optimization, it's a correctness regression with a nice-looking invoice. Did you test any hybrid where structured fields stay as text and only prose gets rasterized? That seems like where the real win lives — rasterize what's cheap to lose, keep the exact tokens exact.
The "saving is real, exact values break" split is exactly the kind of measurement more token-trick posts skip. The fact that planted ids only survived through the factsheet, and ids left out of it scored 0/80, makes the lossiness concrete. For agent sessions, that suggests image compression is fine for prose context and dangerous for logs or anything where a wrong number ships. Did any model refuse more often instead of inventing, or did they mostly hallucinate confidently?
Good question, and I checked the data. Mostly they declined rather than invented. Out of 320 image answers per model: Haiku 56 right, 4 wrong, 260 "can't read". Sonnet 135 / 21 / 164. Opus 164 / 32 / 124. Fable 78 / 37 / 205. So Haiku almost never made a number up but also read the least, and Fable invented the most. The confident wrong answers are the dangerous part, which is why I'd keep logs and exact values as text.
Thanks for pulling the numbers, that's more useful than the headline result. The interesting bit for me is that the stronger models read more but also invented more: Opus gets the most right and still makes up 32. So "better model" doesn't fix it, it just moves the tradeoff. Agreed on keeping logs and exact values as text.
The detail that stopped me is that wrong answers looked right: 94392 read as 9392 despite "null if unsure". A drop in recall is easy to notice; a plausible wrong number is not.
Did any model return null for a number it would otherwise have got wrong, or was abstention close to zero across the 144 calls? That rate would tell people more than accuracy alone about whether this is safe for logs.
Thank you for publishing the raw answers and scripts; it makes the 0 of 80 for ids left out of the factsheet easy to check.