When building applications with large language models (LLMs), one of the most overlooked costs is how structured data is represented.
Most systems use JSON.
And JSON is inefficient for LLM input.
What is KODA?
KODA (Knowledge-Oriented Data Abstraction) is a schema-first data format designed to reduce token usage when sending structured data to LLMs.
It works by:
- Defining structure once (schema-first)
- Encoding values positionally
- Eliminating repeated keys found in JSON
KODA is optimized for:
- RAG pipelines
- Tool calling systems
- Agent workflows
- High-volume structured LLM input
The Problem with JSON in LLM Pipelines
JSON repeats field names for every record.
Example:
[
{"id": 1, "title": "Bug", "state": "open"},
{"id": 2, "title": "Fix", "state": "closed"}
]
Each object repeats:
- id
- title
- state
If you send 1000 records:
- those keys are repeated 1000 times
- tokens are wasted
- costs increase
- context window shrinks
KODA Equivalent
KODA/1
@META
schemas:issue
counts:issue=3
@SCHEMA
issue:id title state
@DATA:issue
1|Bug|open
2|Fix|closed
No repeated keys.
Only structure + values.
Token Reduction Benchmark
Measured using a gpt-4o-mini tokenizer on real datasets.
| Case | JSON Tokens | KODA Tokens | Reduction |
|---|---|---|---|
| Repetitive Logs | 3202 | 1233 | 61.5% |
| GitHub Issues | 4137 | 2576 | 37.7% |
| Small Dataset | 26 | 35 | -34.6% |
Key insight
KODA performs best on large, repetitive structured data.
For small datasets, schema overhead can outweigh benefits.
Why This Matters
In LLM systems:
- Tokens = cost
- Tokens = latency
- Tokens = context capacity
Reducing tokens by ~30–40%:
- lowers API costs
- increases usable context
- improves system efficiency
How KODA Works
KODA separates:
- Schema → defined once
- Data → streamed positionally
This removes structural redundancy.
Quick Python Example
from koda import Schema, Field, encode
schema = Schema("user", [
Field("id"),
Field("name"),
Field("email", optional=True),
Field("active", default="true")
])
data = [
{"id": 1, "name": "Alice", "email": "alice@example.com"},
{"id": 2, "name": "Bob"}
]
koda_str = encode(data, schema)
print(koda_str)
KODA vs JSON vs YAML vs TOON
| Format | Token Efficiency | Readability | Best Use Case |
|---|---|---|---|
| JSON | Low | High | APIs |
| YAML | Medium | Medium | Config files |
| TOON | High | Medium | LLM structured data |
| KODA | High | Low | LLM pipelines |
When to Use KODA
Use KODA if you are:
- sending large structured datasets to LLMs
- building RAG pipelines
- working with tool calls or agents
- optimizing token usage in production systems
When NOT to Use KODA
Do not use KODA for:
- small datasets (1–2 records)
- irregular or deeply nested JSON
- human-authored configuration files
JSON is better in those cases.
Design Principles
- Schema-first design
- Positional encoding
- Deterministic parsing
- No repeated keys
- Optimized for LLM input
Is KODA a JSON Replacement?
No.
KODA is a transport format for LLM pipelines.
Typical workflow:
JSON → KODA → LLM
FAQ
What is KODA?
KODA is a schema-first data format that reduces token usage for structured data in LLM systems.
Is KODA better than JSON?
For LLM input, yes. For general use, JSON is still better.
Does KODA always reduce tokens?
No. It works best on large structured datasets.
Where should I use KODA?
RAG pipelines, tool calls, and structured LLM input.
Try It
GitHub: https://github.com/Om7035/koda
pip install koda
Final Thoughts
If you're sending structured data to LLMs, you're likely wasting tokens.
KODA is a simple way to reduce that overhead.
It’s not a replacement for JSON it’s an optimization layer for LLM pipelines.
Feedback and contributions are welcome.
Top comments (0)