TOON vs JSON: A Reality Check — When It Saves Tokens and When It Doesn't

There's been a wave of articles lately about TOON (Token-Oriented Object Notation), many proclaiming it saves "50% tokens" or calling JSON "outdated" for LLM applications. I've spent some time analyzing the actual numbers.

The verdict? TOON is genuinely useful—but the marketing oversimplifies reality. Let me set the record straight so you can make informed decisions about when to use it.

The 50% Savings Claim: A Misleading Baseline

Here's the uncomfortable truth that many TOON articles gloss over: the "50% savings" figure is typically measured against formatted JSON (with pretty-printing, indentation, and whitespace). But real-world API calls and LLM prompts use minified JSON.

When you compare against minified JSON - which is what you're actually sending to your LLM—the picture changes dramatically:

Data Type	vs Minified JSON	Reality
Uniform arrays (user lists, logs)	20-35% savings	This is where TOON shines
API responses with arrays	0-15% savings	Modest improvement
Mixed nested structures	-5% to +10%	Results vary, test your data
Configuration objects	+10% to +20% MORE tokens	TOON hurts here
Deeply nested objects	+15% to +20% MORE tokens	TOON hurts here
Single flat objects	-5% to +5%	Negligible difference

Yes, you read that correctly: TOON can increase your token usage by 15-20% depending on your data structure.

How TOON Works (and Why Structure Matters)

TOON saves tokens through a clever trick: for uniform arrays, it declares field names once in a header, then streams values in CSV-like rows:

users[3]{id,name,role}:
1,Alice,admin
2,Bob,user
3,Carol,user

Compared to the equivalent minified JSON:

{"users":[{"id":1,"name":"Alice","role":"admin"},{"id":2,"name":"Bob","role":"user"},{"id":3,"name":"Carol","role":"user"}]}

The savings come from not repeating "id":, "name":, and "role": for every record. With 100 records, this adds up significantly.

But here's the catch: TOON still needs syntax for non-tabular data. For nested objects, it uses YAML-style indentation, which can actually be more verbose than minified JSON because:

1. Whitespace (indentation) consumes tokens
2. YAML-style key declarations still repeat
3. The format overhead for small objects exceeds JSON's braces

When TOON Actually Hurts You

Let me show you a real example. Consider this configuration object:

{"debug":true,"maxRetries":3,"timeout":5000,"features":{"darkMode":true,"beta":false}}

In TOON, this becomes:

debug: true
maxRetries: 3
timeout: 5000
features:
  darkMode: true
  beta: false

The minified JSON is actually more token-efficient because:

• No newlines between properties
• No indentation overhead
• Colons and braces are single tokens anyway

My benchmarks at json2toon.de consistently show configuration objects using 10-20% more tokens in TOON format.

The Complete Decision Matrix

Based on extensive testing, here's when to use each format:

Use TOON when:

• You have large arrays of uniform objects (100+ records with consistent fields)
• Your data is primarily tabular (user lists, product catalogs, log entries, analytics data)
• Every object in your array has the same fields
• You're processing paginated API results with uniform structure

Stick with minified JSON when:

• Your data has deep nesting (org charts, recursive trees)
• You're sending configuration objects or feature flags
• Your objects have varying fields (sparse data)
• You have mixed structures (some arrays, some nested objects)
• You're dealing with single flat objects

Consider plain CSV when:

• Your data is purely tabular with no hierarchy
• You don't need nested structures at all
• You want maximum token efficiency (CSV beats TOON by ~30% for flat tables)

What about YAML?

Despite some suggestions, YAML is typically the worst choice for token efficiency. It uses approximately 21% more tokens than minified JSON because:

• Whitespace carries meaning and can't be removed
• Keys still repeat for every array item
• No tabular optimization like TOON provides

The Hidden Gotchas

1. LLM Training Data Bias

Most LLMs were trained predominantly on JSON. When you use TOON, you may need to include format explanations in your prompts, which partially negates your token savings. Some developers report that smaller models (like GPT-3.5-Turbo) struggle to output valid TOON even when they understand TOON input.

2. No Standard Specification (Yet)

While there's an open spec on GitHub, TOON lacks the RFC-level standardization of JSON. There's no guarantee of consistent parsing across implementations.

3. Debugging Complexity

When something goes wrong in your LLM pipeline, debugging TOON is harder than debugging JSON. Most tools, loggers, and validators expect JSON.

4. Mixed Data Penalty

Real-world API responses often mix arrays with metadata. If your response looks like this:

{"total":150,"page":1,"users":[...]}

Only the users array benefits from TOON optimization. The metadata overhead remains, and TOON's format declarations for small objects may actually increase total tokens.

My Recommendation: Measure, Don't Assume

Don't blindly convert all your JSON to TOON based on marketing claims. Instead:

1. Use real data: Test with your actual payloads, not synthetic examples
2. Compare against minified JSON: Use the "Minify" button in converters to see accurate savings
3. Profile your specific structure: Uniform arrays will save tokens; config objects won't
4. Consider the full picture: Include any format explanation prompts in your token calculation
5. Test LLM comprehension: Make sure your target model actually understands TOON output

You can try this yourself at json2toon.de — paste your real production data and see the actual savings (or costs) for your specific use case.

Conclusion: It's Not JSON vs TOON - It's Using the Right Tool

TOON is a valuable addition to the developer toolkit. For uniform arrays of objects, it delivers genuine 20-35% token savings against minified JSON. That's real money at scale.

But it's not a universal JSON replacement. The "JSON is costing you money" narrative ignores that TOON costs you money too - sometimes more money - when applied to the wrong data structures.

The future isn't "TOON everywhere." It's intelligent format selection based on your actual data:

• Uniform arrays → TOON
• Flat tabular data → CSV
• Everything else → Minified JSON

Don't let hype drive your architecture decisions. Measure your specific use case, understand the tradeoffs, and choose accordingly.

---

Built json2toon.de to help developers make data-driven format decisions. Try it with your real payloads and see the actual numbers.

---

What's your experience been? Have you seen the promised savings, or did you run into the scenarios where TOON increased your token usage? Share your benchmarks in the comments—I'd love to see real-world data from the community.

Comments

[david duymelinck]

I still don't understand the hype around TOON. It is the baby of YAML and CSV. The only thing it does different is adding metadata to a tabular structure.

If the main goal is to keep the token count low, choose the most efficient format. For a large amount of data that is probably a binary format.
I agree not to put all your eggs in one basket.

[Raphael Stäbler]

I get that we want to think about token optimization and which format is best for the job. To that end, I'm very much in favor of new formats like TOON but as you said: choose the most efficient format for your use case. That's all there is to it.