Your model reads 1M tokens and writes back 8K — and the gap lands on your input bill

The first number you see on a model page is the context window: 200K, 400K, a million. It reads like capacity — "I can feed it my whole repo." True, and beside the point.

The number that decides your bill is the other one, printed smaller: how much the model is allowed to write in a single reply. It's usually 10 to 30 times smaller than the context window, and it quietly turns one job into a loop.

Three words first, because the whole argument lives in them. A token is roughly ¾ of a word — models bill per million of them, at one price for input (what you send) and a higher one for output (what the model writes). The context window is how many tokens one request may contain. And max output tokens — max_tokens in most APIs — is the ceiling on a single reply. When the reply hits it, the API doesn't crash: it stops mid-sentence and sets finish_reason: "length". You are billed in full for the truncated half.

The gap is real, and it changes with the route

Here is the same family of models, seen from two places — the numbers on the left are what one OpenAI-compatible gateway exposes, the ones on the right are Anthropic's own published limits:

Model / route	Context window	Max output	Ratio
Gemini 3 Pro, via gateway	1,000,000	32,000	31×
Claude Sonnet 5, via gateway	200,000	8,192	24×
Claude Sonnet 5, first-party API	1,000,000	128,000	8×

Same model name, a 15× difference in how much it will write per call. The cap is not a property of the model alone — it's a property of the model plus the road you reach it by, and nobody puts it in the marketing copy. Check it yourself: the Models API returns it, and any gateway's /v1/models should too.

What the cap actually costs

I ran the arithmetic on a real corpus: the 46 Russian posts in our blog's content/ directory, 138,108 tokens counted with o200k_base, the tokenizer behind the current GPT models. The job: produce an English version of all of it.

Reading it is free of drama — 138K fits in every context window in the table above, once. Writing it back is the problem. Output is roughly as long as input, so with an 8,192-token cap the job takes 17 calls; with 128,000 it takes 2.

Seventeen calls wouldn't matter if each one were cheap. But the obvious way to keep 17 chunks consistent is to hand the model the whole corpus every time — and now you've sent 138K tokens seventeen times.

import math
CORPUS = 138_108          # tokens of material you need rewritten

for cap in (8_192, 128_000):
    calls = math.ceil(CORPUS / cap)
    naive_input = calls * CORPUS      # full context resent every call
    print(f"cap {cap}: {calls} calls, {naive_input/1e6:.2f}M input tokens")

cap 8192: 17 calls, 2.35M input tokens against cap 128000: 2 calls, 0.28M. At GPT-5.6's official $2.50 per million input and $15 per million output, that's $7.94 versus $2.76 for byte-identical work. The output bill never moved — 138K tokens either way. The cap multiplied the input side, 8.4×.

The fix is boring and it works

Stop resending the whole thing. Send the chunk you're rewriting plus a short shared brief — a glossary, a style note, the previous chunk's last paragraph. Input then totals about one corpus, not seventeen: $2.42 for the same job, cheaper than the 128K version, because the big cap wasn't the point. Cheap context makes resending feel free; it isn't.

Two habits worth building the same week. Read finish_reason on every call and treat "length" as an error, not a shrug — a truncated answer that your code cheerfully parses is worse than a crash. And set max_tokens explicitly instead of inheriting a default: gateways often impose their own (ours truncates at 4,096 when you omit it), and "the answer got cut off" is the most common first bug in LLM code.

What this doesn't solve

Chunking costs you consistency — terminology drifts between chunk 4 and chunk 12, and a shared glossary only narrows the drift. Prompt caching pushes the resend price down a lot when the prefix is stable, which changes the arithmetic above but not its direction; I can't quote our own hit rates, because our usage records don't split cache tokens out yet. And nothing here helps when a single indivisible answer genuinely exceeds the cap — then the cap is a hard wall and you need a route with a bigger one.

For reference, the per-token prices I used are the vendors' official ones. We resell the same models 10–25% below those (GPT-5.6 at $2.13/$12.75 against the official $2.50/$15.00), which shaves the bill by a fixed percentage — it does not change the multiplier. A 15% discount on seventeen redundant calls is still seventeen redundant calls.

Go look up the output cap of whatever model you're calling right now. If you don't know it, you're not budgeting — you're guessing.