DEV Community

Davide Santangelo
Davide Santangelo

Posted on

Stop Wasting LLM Tokens: Introducing CTON (Compact Token-Oriented Notation)

#ruby #llm

If you are building applications with Large Language Models (LLMs), you are likely familiar with the "Token Tax."

Every character you send to an API like OpenAI or Anthropic costs money and eats into your context window. We’ve spent decades treating JSON as the gold standard for data serialization, and for good reason—it's readable and ubiquitous.

But JSON is chatty. It loves whitespace, it demands quotes around every key, and it uses heavy punctuation ({, }, [, ]) that adds up quickly when you are sending thousands of records in a prompt.

Enter CTON (Compact Token-Oriented Notation).

What is CTON?

CTON is an aggressively minified, shape-preserving data format designed specifically for LLM interactions. It strips away the syntactic sugar humans like (indentation, excessive quoting) but keeps the structural hints machines need.

It bridges the gap between a compressed binary format (which LLMs can't read) and standard JSON (which is too verbose).

The "Before and After"

Let's look at the difference immediately.

Standard JSON:

{
  "task": "planning",
  "urgent": true,
  "context": {
    "user": "Davide",
    "theme": "dark"
  }
}
Enter fullscreen mode Exit fullscreen mode

CTON:

task=planning,urgent=true,context(user=Davide,theme=dark)
Enter fullscreen mode Exit fullscreen mode

By removing the outer braces, removing quotes around safe keys/values, and using parentheses for nesting, we significantly reduce character count without losing the hierarchy.

The Killer Feature: Table Compression

The real magic of CTON (and where the provided Ruby gem shines) is how it handles arrays of objects.

In JSON, a list of items repeats the keys for every single entry. This is a massive waste of tokens.

JSON Array:

{
  "files": [
    { "name": "README.md", "size": 1024 },
    { "name": "lib/cton.rb", "size": 2048 },
    { "name": "spec/test.rb", "size": 512 }
  ]
}
Enter fullscreen mode Exit fullscreen mode

CTON automatically detects this pattern and converts it into a Header-Value Table:

CTON Array:

files[3]{name,size}=README.md,1024;lib/cton.rb,2048;spec/test.rb,512
Enter fullscreen mode Exit fullscreen mode

Notice what happened:

  1. files[3]: The array declares its length explicitly. This helps the LLM parser "plan" the generation or consumption.
  2. {name,size}: The keys are declared once.
  3. =val1,val2;val3...: The values are delimited by semicolons.

This approach can reclaim 50–60% of your token budget compared to raw JSON on large datasets.

Why not just use YAML?

YAML is great, but it relies heavily on significant whitespace (indentation). In the context of LLM prompts:

  1. Whitespace consumes tokens.
  2. It creates ambiguity if the LLM generates output with slightly off indentation.

CTON is deterministic. It uses explicit delimiters ((, ), =, ,), making it robust enough to round-trip data safely.

Using CTON in Ruby

I've released CTON as a Ruby gem. It follows a dump/load API similar to the standard JSON library.

Installation 

bundle add cton
Enter fullscreen mode Exit fullscreen mode

Usage 

require "cton"

payload = {
  "context" => "hiking",
  "location" => "Alps",
  "trails" => [
    { "id" => 1, "name" => "Blue Path", "dist" => 10 },
    { "id" => 2, "name" => "Red Path",  "dist" => 15 }
  ]
}

# Encode to CTON string
encoded = Cton.dump(payload)
# => "context=hiking,location=Alps\ntrails[2]{id,name,dist}=1,Blue Path,10;2,Red Path,15"

# Decode back to Hash
decoded = Cton.load(encoded)
Enter fullscreen mode Exit fullscreen mode

Under the Hood: Type Safety

One concern with minified formats is type safety. Does true become the boolean true or the string "true"?

The CTON encoder is smart about this. It follows strict guardrails (inspired by the TOON specification):

  • Literals: true, false, and null are preserved as native types.
  • Numbers: Integers and Floats are normalized.
  • Safety: If you have a string that looks like a boolean (e.g., "true"), CTON will auto-quote it (e.g., "\"true\"") to ensure it doesn't get parsed incorrectly on the way back.

When should you use this?

CTON is ideal for:

  1. System Prompts: Embedding large configuration hashes or user context into the system prompt.
  2. Few-Shot Examples: Providing examples of data structures to the LLM.
  3. Data Retrieval: When asking an LLM to "output the data in CTON format" to save generation time (and therefore latency).

Get Involved

CTON is open source and MIT licensed. The parser implementation is relatively small, consisting of a recursive descent parser (Decoder) and a smart Encoder.

Check out the repository, try it out in your next AI project, and let me know how many tokens you saved!

👉 GitHub Repository: https://github.com/davidesantangelo/cton

Have you experimented with custom data formats for LLMs? Let me know in the comments!

Top comments (0)