Extended Thinking in Claude: budget_tokens and What Replaced It

#ai #machinelearning #llm #programming

budget_tokens was the dial that decided how long Claude could reason before answering. Understanding what it did is still worth the time, because the constraint it made visible — reasoning and answer share one budget — did not go away when the parameter did.

The request shape

Extended thinking is enabled with a thinking object carrying a token budget:

{
  "model": "claude-sonnet-4-5",
  "max_tokens": 16000,
  "thinking": {
    "type": "enabled",
    "budget_tokens": 10000
  },
  "messages": [
    {"role": "user", "content": "Find the race condition in this scheduler."}
  ]
}

Two documented constraints govern the value. It has a floor of 1,024 tokens — below that the request is rejected — and it must be strictly less than max_tokens. Both are validated server-side, so a budget larger than the cap is a 400 rather than a silent clamp.

The second constraint is the one that carries meaning. It exists because thinking tokens are output tokens: they are generated by the model, billed at output rates, and counted against max_tokens like any other. The budget is not a separate allowance.

The arithmetic budget_tokens forces

Because both come out of the same cap, setting a budget is simultaneously setting a ceiling on the visible answer. The derivation is one line:

answer_ceiling  =  max_tokens - thinking_actually_used

with max_tokens = 16,000 and budget_tokens = 10,000:
  worst case (model uses the whole budget)   16,000 - 10,000  =  6,000
  typical case (model uses ~40% of budget)   16,000 -  4,000  = 12,000
  best case (model barely thinks)            16,000 -    500  = 15,500

The important asymmetry is that budget_tokens is a target, not a reservation. The model may use less than the budget — often much less on an easy question — and what it does not use is available for the answer. What it cannot do is exceed it. So the budget sets the worst case for your answer length, and the worst case is the one you must size for.

The failure this produces is distinctive: a response whose stop_reason is max_tokens containing a long thinking block and a visible answer that stops mid-sentence. The model did not lose the thread. It spent the budget reasoning and ran out of room to write. The fix is to raise max_tokens, not to lower the budget — lowering the budget also lowers the quality you turned thinking on to get.

Size the cap for both. A useful starting shape is max_tokens at the budget plus whatever the answer needs, plus headroom. A 10,000-token budget with a 12,000-token cap leaves 2,000 for the answer in the worst case, which is not enough for anything substantial.
Stream once the cap is large. A budget generous enough to be worth setting usually pushes max_tokens past the point where a buffered request risks a transport timeout.
Thinking is billed. It appears in usage.output_tokens and is charged at output rates whether or not you display it.

What comes back

Thinking appears as its own content block type, before the text block:

{
  "content": [
    {
      "type": "thinking",
      "thinking": "The scheduler takes the lock after reading the queue length…",
      "signature": "EqQBCgIYAhIM…"
    },
    {
      "type": "text",
      "text": "The race is between the length read and the lock acquisition…"
    }
  ],
  "stop_reason": "end_turn"
}

The signature field is a cryptographic marker that the block is genuine and unmodified. If you send a thinking block back as conversation history — which you must do in a tool loop, so the model can see its own reasoning from the previous step — the block has to be echoed verbatim, signature included. Editing the text and resending it is rejected.

In a streamed response the same content arrives as thinking_delta events followed by a signature_delta that closes the block. See the streaming event types. On models configured to omit reasoning text, the thinking block still appears in the stream with empty content — so a renderer that shows a “thinking” panel whenever the block type arrives will show an empty panel rather than nothing at all.

Thinking blocks in a tool loop

This is where thinking stops being a quality setting and starts being something your conversation-building code has to know about.

When a thinking-enabled model calls a tool, the assistant turn contains a thinking block and a tool_use block. To continue the loop you append that turn as history — and the thinking block must go back with it, unmodified, signature intact. It is not optional and it is not cosmetic: the model needs to see the reasoning that led to the call in order to interpret the result, and the signature is what proves the reasoning is the one it actually produced.

Three concrete failures come out of getting this wrong:

Dropping the block. The most common cause is history-building code that extracts text and tool_use blocks by name and discards everything else. The request is rejected, and the error names a block type the author never knowingly handled.
Editing the text. Redacting the reasoning before storing it, or round-tripping it through a formatter that normalises whitespace, breaks the signature. If you must not persist reasoning text, the answer is to not persist the turn, not to persist a modified one.
Reordering the blocks. The thinking block comes first in the array for a reason. Rebuilding the turn with the tool call first is a different message.

The budget interacts with the loop as well, and multiplicatively. Every iteration is a fresh generation with its own reasoning, so a ten-step agent run pays the thinking cost ten times, and each of those steps re-reads a context that now includes all the previous reasoning. This is the specific reason agent loops with thinking enabled cost markedly more than the same loop without it, and why a budget chosen by looking at a single-turn request tends to look very different after you watch it run for twenty turns.

Where the parameter was removed

This is the part that a page written a year ago would get wrong. budget_tokens is the extended-thinking parameter for the model generations it shipped with, and on the newest generations it is gone: sending {"type": "enabled", "budget_tokens": N} to a current frontier Claude model returns a 400 rather than being honoured or ignored.

What replaced it is adaptive thinking — the model decides how much to think per request — configured as {"type": "adaptive"}, with depth steered by a separate effort setting rather than a token count. The rationale is that a fixed token budget is a bad instrument: too low on the hard requests, wasted on the easy ones, and impossible to tune once for a mixed workload.

Which parameter a given model accepts is a per-model fact that changes with each release, and the two forms are not interchangeable — the wrong one is a 400, not a fallback. Anthropic’s extended thinking documentation and the adaptive thinking page carry the current per-model support. Pin your model version and you will at least be wrong predictably.

Choosing a setting today

If you are on a generation that takes budget_tokens, treat it as a worst-case reservation against your answer and size max_tokens accordingly. If you are on a generation that takes adaptive thinking, the question changes from “how many tokens” to “how much effort”, and the answer is workload-specific rather than universal: lower settings for routine extraction and classification, higher for the long multi-step problems that thinking exists for.

Either way the invariant holds. Reasoning and answer share one budget; if you do not leave room for both, the model runs out of room to speak.