Everyone writes response_format={"type": "json_object"} and moves on. It's the obvious choice: your code needs structured data, JSON is structured data, done.
Then the extraction job runs on a hundred thousand rows and the bill arrives, and a surprising share of it went to curly braces, colons and the word subscription_status retyped ten thousand times.
Two words first, since the argument lives in them. A token is roughly ¾ of a word — models bill per million of them. And they bill at two different prices: input tokens (what you send) and output tokens (what the model writes back), with output typically 5–6× the price of input. Formatting you ask the model to produce lands on the expensive side.
The same 200 records, five formats
I generated 200 synthetic customer records — id, name, email, city, amount, signup date, status — and counted them with OpenAI's o200k_base tokenizer, the one behind the current GPT models. Same data every time, only the wrapper changed:
| Format | Tokens per row | vs pretty JSON |
|---|---|---|
JSON, indent=2
|
77.8 | — |
| JSON, compact | 52.8 | −32% |
| JSONL | 53.8 | −31% |
| CSV | 29.9 | −62% |
| TSV | 29.8 | −62% |
Pretty-printed JSON costs 2.6× what CSV costs for byte-identical information. Nothing about the data changed. The model did the same work. You just asked it to type more.
Where the tokens actually go
Two places, and both are avoidable.
Indentation is 25 tokens a row. That's the gap between pretty and compact JSON — 32% of the bill, spent on whitespace that no parser needs and no human will read, because this output goes straight into json.loads.
Key names are another 29 tokens a row. "customer_id":, "email_address":, "subscription_status": — the schema, re-typed for every single record. Across 200 rows that's 5,800 tokens spent restating a header you already knew (net 23 a row once CSV's own commas are counted back in).
The tempting fix is renaming fields to a, b, c. I measured that too: it takes compact JSON from 52.8 to 44.8 tokens a row. Eight tokens, in exchange for a payload nobody can debug. Dropping the keys entirely — one header line, then rows — saves 23 net. CSV is short-keys done properly.
What it costs in dollars
Take a bulk extraction job on gpt-5.6 at its list price of $15.00 per 1M output tokens, producing 1,000 rows per call, 100 calls a day.
- Pretty JSON: 77,765 output tokens per call → $1.17 a call, $117 a day
- CSV: 29,885 output tokens per call → $0.45 a call, $45 a day
That's $2,155 a month for the indentation and the repeated field names. The prompt is one sentence different.
Count yours before you argue about it
Don't take my row numbers — yours have different field names and different value lengths, and both move the ratio. Fifteen lines:
import json, csv, io, tiktoken
enc = tiktoken.get_encoding("o200k_base") # current GPT models
rows = json.load(open("sample_output.json")) # 100+ real rows
buf = io.StringIO()
w = csv.DictWriter(buf, fieldnames=list(rows[0]))
w.writeheader(); w.writerows(rows)
for name, text in [("json pretty", json.dumps(rows, indent=2)),
("json compact", json.dumps(rows, separators=(",", ":"))),
("csv", buf.getvalue())]:
n = len(enc.encode(text))
print(f"{name:14s} {n:7d} tokens {n / len(rows):6.1f}/row")
If the gap is under 20%, your values are long enough that the wrapper doesn't matter — keep JSON and go optimize something else. If it's the 60% I measured, your records are short and mostly schema.
What this doesn't fix
Three honest limits. This is a bulk trick: for a single object with five fields, the difference is pennies and JSON's error-tolerance is worth more. Tool calling and JSON mode force JSON on you — the model's structured-output path emits it and you can't ask for CSV there, so the win is limited to plain-text completions you parse yourself. And CSV needs real escaping: commas and newlines inside values will corrupt a naive split(","), so use csv.reader, or TSV, which measured identically and collides with far less.
Also, I counted with OpenAI's tokenizer. Anthropic and Google tokenize differently, so the ratio holds directionally but the per-row numbers won't transfer exactly — recount on the model you actually bill.
Format work shrinks the token count. It does nothing to the other multiplier, the price per token, and your bill is the product of the two. That's the gateway's half: altrouter.ai resells the same models 10–25% below the vendors' list prices — gpt-5.6 at $12.75 per 1M output against OpenAI's $15.00 — over the OpenAI-compatible API, so it's a base_url change. Its honest gap here: we don't expose a token-counting endpoint, so the measurement above still has to run locally against the vendor's own tokenizer.
The one number to take away
77.8 versus 29.9 tokens per row. Before you tune anything else about an extraction pipeline, look at what format you asked for — and whether anybody was ever going to read the indentation you're paying output price to generate.
Top comments (0)