Is JSON Outdated? The Reasons Why the New LLM-Era Format "TOON" Saves Tokens

JSON, You Talk Too Much: Exploring TOON for Token-Efficient LLM Communication

Introduction

JSON, buddy, you might be a bit too chatty.

As developers, JSON (JavaScript Object Notation) is like the air we breathe. It's everywhere—API responses, config files, logs, you name it. I've trusted JSON as my reliable companion for years.

But ever since I started working with LLMs (Large Language Models), something's been bugging me:
"Wait... are you talking way too much?"

[
  {"id": 1, "name": "user_a", "role": "admin"},
  {"id": 2, "name": "user_b", "role": "member"},
  {"id": 3, "name": "user_c", "role": "member"}
]

JSON is thorough. Very thorough. A good friend, really (or maybe not a "friend" at all, but you get the idea).

But that diligence—repeating "id":, "name":, "role": every single time—is eating away at those pay-per-token charges. You know the saying: "Every little bit adds up." In the LLM era, those "little bits" translate directly to money and speed.

Recently, I stumbled across TOON (Token-Oriented Object Notation). At first, I thought it might be related to cartoons (cute, right?), but the reality is far more pragmatic and ruthless. I like that kind of thing.

This article is my exploration of TOON and what it means for our relationship with JSON.

---

The "Bloat" Problem: JSON in the LLM Era

JSON is a fantastic format—readable by both humans and machines. That's undeniable.

But from a token efficiency perspective? JSON is wearing some pretty heavy clothes.

Here's where the "bloat" comes from:

1. Repeated keys: Every array element duplicates "name":, "role":, etc.
2. Excessive brackets: {} and [] everywhere
3. Quotation marks: All those "" add up fast

These features were welcomed as "readability" in traditional Web APIs. But for LLMs, they're just meaningless tokens that waste context window space.

How Are Tokens Actually Counted?

"Character count ≠ token count" is common knowledge among AI engineers, but have you ever really thought about how JSON gets tokenized?

Let's look at OpenAI's Tokenizer (cl100k_base):
https://platform.openai.com/tokenizer

Take a simple JSON snippet: {"id": 123}.

A human sees "id is 123" instantly, but the tokenizer breaks it down like this:

1. { (1 token)
2. " (1 token)
3. id (1 token)
4. ": (1 token)
5. (1 token)
6. 123 (1 token)
7. } (1 token)

Total: 7 tokens.

More than half the tokens are just structural symbols. Scale this to 10,000 data entries, and... yeah, it gets scary fast.

Compare that to a TOON-style |123|:

1. | (1 token)
2. 123 (1 token)
3. | (1 token)

Total: 3 tokens.

These "tiny" differences become mountains when you're dealing with RAG systems handling thousands of characters of context.

A Note for Non-English Users

For those working with non-English languages, this problem is even more acute. While English often maps to ~1 token per word, languages with different character systems can consume 1-2 tokens per character.

It's like wrapping premium ingredients in layers of thick packaging—wasteful and expensive.

TOON's approach of stripping structure to the bare minimum might be especially valuable for those of us dealing with token-inefficient languages.

---

What is TOON (Token-Oriented Object Notation)?

TOON is exactly what it sounds like: a data format (or rather, a notation concept) designed with token efficiency as the top priority.

It maintains JSON-compatible data structures while radically simplifying the representation.

Think of it as "JSON on a diet, keeping only the essential skeleton."

Seeing is Believing: JSON vs TOON

Actions speak louder than words. Let's compare the same data in both formats.

Standard JSON:

[
  {
    "id": 1,
    "name": "Alice",
    "role": "Admin",
    "status": "Active"
  },
  {
    "id": 2,
    "name": "Bob",
    "role": "User",
    "status": "Inactive"
  },
  {
    "id": 3,
    "name": "Charlie",
    "role": "User",
    "status": "Active"
  }
]

TOON version:

| id  | name    | role  | status   |
| --- | ------- | ----- | -------- |
| 1   | Alice   | Admin | Active   |
| 2   | Bob     | User  | Inactive |
| 3   | Charlie | User  | Active   |

Or even more stripped down (depending on implementation):

#keys: id, name, role, status
1, Alice, Admin, Active
2, Bob, User, Inactive
3, Charlie, User, Active

The difference is striking, isn't it?

All those { } " key: symbols vanish, leaving only the actual data. It's similar to CSV or Markdown tables, but the key insight is getting LLMs to recognize this as structured data—that's the TOON philosophy.

---

Implementation: Converting JSON to TOON in Python

"Okay, I get the concept, but how do I use it?" Here's a simple conversion script using only Python's standard library.

def json_to_toon(data: list[dict]) -> str:
    """
    Convert a list of dictionaries to TOON format (Markdown table style)
    """
    if not data:
        return ""

    # Get headers
    headers = list(data[0].keys())

    # Create header and separator lines
    header_line = "| " + " | ".join(headers) + " |"
    separator_line = "| " + " | ".join(["---"] * len(headers)) + " |"

    lines = [header_line, separator_line]

    # Create data rows
    for item in data:
        values = [str(item.get(h, "")) for h in headers]
        line = "| " + " | ".join(values) + " |"
        lines.append(line)

    return "\n".join(lines)

# Try it out
users = [
    {"id": 1, "name": "Alice", "role": "Admin"},
    {"id": 2, "name": "Bob", "role": "User"}
]

print(json_to_toon(users))

Output:

| id  | name  | role  |
| --- | ----- | ----- |
| 1   | Alice | Admin |
| 2   | Bob   | User  |

Embed this in your prompt with "Please analyze the following data," and you're good to go. For output parsing, specify the format and use regex or split('|').

The Reverse: Parsing TOON Back to JSON

"Okay, but I need to actually use this data programmatically." Absolutely. Here's a parser to convert TOON output back to Python dictionaries:

def toon_to_json(toon_text: str) -> list[dict]:
    """
    Convert TOON format (Markdown table style) to list of dictionaries
    """
    lines = toon_text.strip().split('\n')
    if len(lines) < 3:
        return []

    # Parse header row
    headers = [h.strip() for h in lines[0].strip('|').split('|')]

    result = []
    # Skip separator line (row 2), process from row 3
    for line in lines[2:]:
        if not line.strip():
            continue

        values = [v.strip() for v in line.strip('|').split('|')]

        if len(values) != len(headers):
            continue

        row_dict = {headers[i]: values[i] for i in range(len(headers))}
        result.append(row_dict)

    return result

# Test
toon_data = """
| id  | name  | role  |
| --- | ----- | ----- |
| 1   | Alice | Admin |
| 2   | Bob   | User  |
"""

print(toon_to_json(toon_data))
# Output: [{'id': '1', 'name': 'Alice', 'role': 'Admin'}, {'id': '2', 'name': 'Bob', 'role': 'User'}]

Of course, production code would need more robust handling (escaping pipes in values, etc.), but the basic logic is straightforward. No external dependencies needed—just copy-paste this utility function.

---

Why TOON? Why Now?

"Why not just use CSV?" Good question!

But TOON offers benefits beyond simple character reduction:

1. Budget-Friendly (API Cost Reduction)

LLM API pricing is typically pay-per-token, with output tokens often costing more than input. When passing large document lists to LLMs (especially in RAG systems), JSON quickly hits token limits. TOON can compress this by 30-60% in many cases.

With RAG systems handling increasingly large contexts, this is crucial knowledge to have.

%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#ffcc00', 'edgeLabelBackground':'#ffffff', 'tertiaryColor': '#f4f4f4'}}}%%
xychart-beta
    title "Token Consumption by Format (Approximate)"
    x-axis [JSON, YAML, TOON]
    y-axis "Token Count" 0 --> 500
    bar [450, 320, 180]

Translation: "Same budget, more data processing."

Cost Reduction Simulation

"30% reduction" sounds abstract. Let's talk dollars.

Imagine running a daily batch process on 10,000 user records with GPT-4o:

• JSON: 500 tokens/record × 10,000 = 5M tokens
• TOON: 300 tokens/record × 10,000 = 3M tokens

At GPT-4o's input price ($5.00 / 1M tokens):

• JSON: $25/day
• TOON: $15/day

$10 difference per day.

Sounds like lunch money? That's $300/month or $3,600/year. For startups, that's non-trivial server costs. And if your RAG document volume 10x's, so does this savings.

2. Speed Boost (Faster Inference)

LLMs generate tokens sequentially. Fewer tokens = faster response.

Writing "name": "Alice", versus just Alice has a measurable physical time difference. For real-time chatbots or agents, these milliseconds compound into UX impact. User churn rates are no joke—people don't wait as well as you'd think.

3. "Remember More" (Better Context Window Utilization)

LLM context windows have limits. Lighter data formats mean more room for prompt instructions and few-shot examples.

No more "context length exceeded" despair.

There's even research on "Lost in the Middle"—where LLMs struggle with information buried in the middle of long contexts. Reducing noise (unnecessary symbols) helps LLMs focus on what matters.

https://arxiv.org/abs/2307.03172

---

The Competition: What About YAML or CSV?

Valid question: "If JSON is bloated, why not YAML or CSV?"

They're solid options. Let's compare:

vs YAML

YAML has fewer brackets and is more readable, but it has a hidden enemy: indentation.

For LLMs, whitespace sequences consume tokens and are error-prone during generation. TOON's explicit delimiters (pipes) reduce structural collapse risks compared to YAML's sensitivity to spacing.

vs CSV

CSV has unbeatable token efficiency: 1,Alice,Admin. Can't get leaner.

But CSV has a weakness: headers are distant from values. With many columns, LLMs lose track ("Wait, what was column 3 again?"), leading to hallucinations.

TOON (Markdown tables) visually aligns columns, making structure clearer to LLMs—a sweet spot between CSV's efficiency and JSON's clarity.

---

Limitations and Trade-offs

Nothing's perfect. Technology choices always involve trade-offs:

• Weak with nesting:

  • JSON's strength—nested objects and arrays—is hard to represent in TOON's flat table format. Forcing it makes things worse. TOON works best for flat data lists.

• Type ambiguity:

  • JSON distinguishes true (bool) from "true" (string). TOON text tables make everything look like strings. Strict type requirements need separate schema definitions.

• No standard library:

  • No import json one-liner. You'll write parsers yourself or let LLMs handle it.

---

Real-World Use Cases: Where Should You Use TOON?

I'm not suggesting you convert all Web API responses to TOON tomorrow. That's madness (and where would you find the engineering hours?).

JSON's ecosystem is powerful and established.

TOON shines in the closed world of LLM communication:

• RAG (Retrieval-Augmented Generation):

  • Passing 100 document search results to an LLM. JSON overflows; TOON fits.

• AI Agent-to-Agent Communication:

  • Planner Agent passing task lists to Worker Agent. Machine-to-machine efficiency beats human-friendly verbosity. As we enter the "multi-agent era," TOON makes sense for inter-agent protocols where humans aren't in the loop.

• Structured Data Extraction:

  • Asking LLMs to "extract data from this text." Specifying TOON output format speeds generation while maintaining parseability.

---

Will It Catch On? Realistic Adoption Lines

After all this praise, would I recommend "Let's adopt TOON for team development tomorrow!"? Honestly, probably not. Not yet.

As I discussed in my previous article on prompt compression via logical structures, "AI-friendly formats are often human-unfriendly (high cognitive load)."

In team development, JSON's "verbose but universally readable" advantage is invaluable. "Universally readable" really means "engineers typically know JSON," but that's still a huge win.

The costs of documentation ("This is TOON format, parse accordingly"), onboarding new members, and debugging difficulties often lead to the conclusion: "Paying slightly more in tokens for JSON's safety is worth it."

Existing API ecosystems run on JSON. Overturning that isn't realistic.

Where TOON Realistically Shines

TOON's practical adoption line settles into "local optimization" scenarios:

1. Large-scale operations where API costs are mission-critical
2. Solo or small-team projects with fast decision-making
3. Internal LLM processing (user-invisible data passing)

Rather than "next-generation JSON replacement," think of TOON as "a tactical tool for specific bottlenecks." The art of modern engineering is knowing when to deploy the right tool for the right job—hybrid approaches included.

---

Conclusion

TOON won't replace JSON anytime soon—and that's okay. It's not trying to.

What TOON offers is a pragmatic solution for a specific problem: token efficiency in LLM communication. When you're paying per token, when speed matters, when context windows are tight, TOON gives you options.

Is it the right choice for every project? Absolutely not. But knowing it exists, understanding the trade-offs, and having it in your toolkit for the right moment? That's what makes a skilled engineer.

JSON, you're still my reliable friend. But when the bill gets too high, TOON might just be the budget-conscious alternative I need.

---

Resources

• TOON (Token-Oriented Object Notation) - X
• Format Tokenization Exploration
• Lost in the Middle: How Language Models Use Long Contexts