The model wrote 23 tokens and billed 150 — reasoning tokens are output you never see

On 22 September we sent Kimi K3 a short request with max_tokens: 16. The visible answer was 23 tokens long. The usage block said 150 completion tokens. 127 of them were text nobody ever saw.

That gap is the whole article. Reasoning models think before they answer, the thinking is billed at the output price, and in most APIs you never see it. A short visible answer can cost as much as a long one.

What a reasoning token is

A token is roughly ¾ of a word; you pay per million of them. Output tokens — what the model writes — usually cost 4–8× more than input tokens. A reasoning model (o3, GPT-5.x, Gemini 2.5+, Kimi K3, Claude with thinking on) writes a draft for itself first: it breaks the task down, checks a guess, discards it. Those draft tokens are reasoning tokens.

The provider generates them on the same hardware as the answer, so they are billed as output. Most APIs then hide the draft or return a short summary. You pay for the draft in full either way.

So the price of a call isn't "how long is the answer". It is answer plus thinking, and the thinking part depends on how hard the model decides the question is, not on how long you want the answer to be.

What we actually measured

Two real cases, both from our own gateway logs. The numbers come from the providers' usage fields:

Case	Output cap	Reasoning	Visible answer
Kimi K3, one request, 22.09.2026	16 (ignored)	127	23
Gemini 2.5 Pro / 3.x, health probe, 15 runs 01–15.10.2026	32	28–30	0

The Kimi row: at the official $15 per million output tokens, 150 tokens cost $0.00225. The 23 tokens you can see would cost $0.00035. That's 6.5× more than the text suggests. A million such calls a month is $2,250 instead of $345.

The Gemini row is our availability probe. It asks "Reply with exactly: " with a 32-token cap. In 15 runs the model spent 28–30 tokens thinking about a task that needs no thinking. It hit the cap (finish_reason: "length") and returned no text at all. That's 15 out of about 5,000 Gemini probes, so it's rare. But it's the purest case: the whole output was invisible.

Where it shows up in the response

You don't have to guess. With an OpenAI-compatible API, the count is in usage.completion_tokens_details.reasoning_tokens, and it is already included in completion_tokens. Divide the two and you know what share of your output bill went to thinking:

from openai import OpenAI

client = OpenAI(base_url="https://api.openai.com/v1", api_key="...")

for effort in ("low", "high"):
    r = client.chat.completions.create(
        model="o3",
        reasoning_effort=effort,
        messages=[{"role": "user", "content": "Is 1001 prime? One word."}],
    )
    u = r.usage
    hidden = (u.completion_tokens_details.reasoning_tokens or 0)
    visible = u.completion_tokens - hidden
    print(f"{effort}: billed {u.completion_tokens}, hidden {hidden}, "
          f"visible {visible} -> {r.choices[0].message.content!r}")

Anthropic does it differently. You turn thinking on with a token budget, and the response contains a separate thinking content block before the text block. The thinking tokens are counted inside usage.output_tokens. There is no separate number for them, so to split the bill you subtract the text you got from the total you paid for. If the API gives you a summary of the thinking, you are still billed for the full thinking, not for the summary.

{
  "model": "claude-...",
  "max_tokens": 4000,
  "thinking": { "type": "enabled", "budget_tokens": 1024 },
  "messages": [{ "role": "user", "content": "Is 1001 prime? One word." }]
}

Two knobs, and how they bite

reasoning_effort (OpenAI-style: low / medium / high) tells the model how much to think. It's the cheapest fix there is: classification, extraction and "reply with X" don't need high. Don't trust the default either: it differs by model and by provider, so set it explicitly.

budget_tokens (Anthropic, Gemini's thinking budget) is a hard ceiling on the draft. Set it too low and the model runs out mid-thought, like our Gemini probe did. You pay for a cap's worth of thinking and get nothing back.

There's a trap here we stepped into. Kimi K3 ignores max_tokens and only obeys max_completion_tokens. Its effort scale is low / high / max, not low / medium / high. If you send it OpenAI's medium, it doesn't recognize the value and thinks at max. On 22.09 that cost 2–3× more than high. No error, no warning — only a bigger number in usage. An unknown knob value doesn't fail loudly; it falls back to whatever the provider thinks is safe, and that's usually the expensive end.

A reader's fair objection: "less thinking means worse answers." Sometimes, yes. That's why you measure. Run your real prompts at two effort levels, compare quality and reasoning_tokens, and keep the lowest level that still passes.

What this doesn't solve

You can't predict reasoning tokens before the call. The same prompt can think 30 tokens one day and 300 the next, so budgets per request stay estimates. Turning effort down trades cost for accuracy on hard tasks, and only your own test set tells you where the line is. And some providers report the number badly or not at all. We had to ask two of our upstreams for stream_options.include_usage explicitly, or thinking tokens on streamed calls came back unreported.

On our side: the gateway we run passes completion_tokens_details.reasoning_tokens through and maps effort levels per model, which is how the Kimi medium bug got caught and fixed. But our own usage records still store only total completion tokens. Our dashboards can't show you the reasoning share yet; you have to read it from each response. And a 10–25% lower per-token price doesn't change the ratio: 127 hidden tokens out of 150 stays 85% at any price.

Log reasoning_tokens next to completion_tokens on your next hundred calls. If the hidden share is above half on simple tasks, lower the effort before you look for a cheaper model.