Token-Efficient Tool Use in the Claude API

#ai #llm #machinelearning #programming

Tool use is expensive in a way that is invisible in your prompt: the tool definitions and the model’s tool-call output are both tokens, and in a loop they are paid for on every turn. Token-efficient tool use is a beta that reduces the output side of that.

Where the tokens actually go

There are three separate charges in a tool-calling turn, and they behave differently.

Tool definitions, as input, every single request. Your tools array is serialised into the prompt. Names, descriptions and the full JSON Schema of every parameter. Twelve tools with thorough descriptions is commonly a four-figure token count before the user has said anything, and it is re-sent on every turn of the loop because there is no server-side state.
The tool call, as output. Each tool_use block is generated: the name, and the arguments as structured output. This is billed at the output rate, which is several times the input rate.
The tool result, as input, forever after. Once a tool_result is in the conversation it is re-sent on every subsequent turn. A verbose tool that returns a 4,000-token API dump costs that on turn one and on every turn after it.

Token-efficient tool use targets the second of these. It changes how the model emits tool calls so that the same call costs fewer output tokens, without changing the shape you receive in your SDK.

The beta header and its scope

It is enabled with the anthropic-beta header value token-efficient-tools-2025-02-19:

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "anthropic-beta: token-efficient-tools-2025-02-19" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-3-7-sonnet-20250219",
    "max_tokens": 1024,
    "tools": [ { "name": "get_weather", "description": "...",
                 "input_schema": { "type": "object",
                   "properties": { "city": { "type": "string" } },
                   "required": ["city"] } } ],
    "messages": [ { "role": "user", "content": "Weather in Lisbon?" } ]
  }'

Anthropic’s documentation for the feature reports an average reduction of around 14% in output tokens, with the range varying by workload. Two scoping rules matter more than the number:

It is scoped to specific models. The beta was introduced for Claude 3.7 Sonnet. On a model that does not implement it, the header is inert — the request succeeds and behaves exactly as if you had not sent it.
Failure is silent. Because it changes an encoding rather than adding a request field, a wrong or unsupported beta name does not error. Nothing tells you it did not apply except the token count, which is why this feature in particular needs measuring rather than shipping. The general case is on beta headers in the Claude API.

Beta scope and availability change, and a beta scoped to one model generation is precisely the kind of thing that stops applying when you upgrade. Check the feature’s documentation page for the current model list before assuming a header carried over from an earlier integration is doing anything.

Measuring it on your own tools

A published average tells you whether to try it, not what it does for you — the saving depends on how many tools you declare and how large their arguments are. Getting your own number takes one afternoon and no benchmark harness:

Collect ten to twenty real requests from your logs that ended in stop_reason: "tool_use". Real ones, with your actual tool array — a synthetic single-tool example will understate the overhead badly.
Replay each one twice against the same pinned model ID: once with the beta header, once without. Keep temperature at 0 to reduce variation between the two runs, and keep every other field identical.
Record usage.output_tokens from each response. That field is the measurement; there is nothing else to instrument.
Compare the totals across the whole set rather than per request. Tool calls vary enough in argument length that a single pair tells you almost nothing.
Separately, call POST /v1/messages/count_tokens with your tools array and an empty user message to see the fixed input cost of your tool definitions alone. See the count_tokens endpoint. This is free and it is frequently the more shocking number.

How the overhead compounds in a loop

A single tool call looks cheap. The reason tool-using agents surprise people on the bill is that the fixed costs are paid again on every turn, and the conversation grows as it goes. The arithmetic below is an illustration with assumed inputs, not a measurement — substitute your own numbers from count_tokens and the shape holds:

Assumptions (yours will differ):
  tool definitions          1,800 input tokens   (10 tools, described properly)
  system prompt               400 input tokens
  user question               120 input tokens
  each tool result            600 input tokens
  each tool call               90 output tokens

Turn 1  input = 1800 + 400 + 120                        = 2,320
        output = 90
Turn 2  input = 2,320 + 90 + 600                        = 3,010
        output = 90
Turn 3  input = 3,010 + 90 + 600                        = 3,700
        output = 90
Turn 4  input = 3,700 + 90 + 600 (final text answer)    = 4,390
        output = 250

Total input  = 13,420 tokens for one four-step task
Total output =   520 tokens
Of the input, 1,800 x 4 = 7,200 tokens are tool definitions alone.

Two things fall out of that layout. The first is that output tokens — the thing this beta reduces — are 520 of roughly 14,000 tokens moved. A 14% saving on the output side is a saving on the small number. Even allowing for output being priced several times higher than input, it is not where the money is in this shape of workload.

The second is that the tool definitions, re-sent verbatim four times, are more than half the input. That is the number to attack, and the three levers in the next section attack it. None of this makes the beta not worth enabling — it is a header, it costs nothing — but it does explain why teams turn it on, see a 3% change in their bill, and conclude the feature does not work. It worked; it was applied to the wrong 4% of the total.

Bigger levers than the flag

A percentage off the output side is real, and for most tool-calling applications it is not the largest available saving. Three things usually beat it:

Cache the tool definitions. They are a long, stable prefix that is re-sent on every turn — the exact shape prompt caching exists for. A cache_control breakpoint after the tools array converts that repeated input cost to the cached-read rate for the life of the cache.
Declare fewer tools. Every tool costs input tokens on every request whether or not it is called. Routing to a task-specific subset of tools cuts the fixed overhead directly, and tends to improve tool selection accuracy at the same time.
Return less from the tool. Tool results are the charge that compounds, because they stay in context. Returning the three fields the model needs rather than the upstream API’s full response is usually the single largest reduction available in a multi-turn agent, and it requires no beta header at all.

The block shapes involved — what a tool_use block contains and how tool_result is structured — are covered in Claude’s tool_use content block.

A last point in the beta’s favour that the token figure understates: output tokens are generated one at a time, so fewer of them is less wall-clock time as well as less money. In an agent loop where the user is waiting through three or four tool calls, a reduction in output tokens shows up at every step. It is a small effect per call and it is the only latency saving on this page that requires no architectural change at all — which is a reasonable argument for enabling it on supported models and then going and doing the three things above, rather than treating it as a choice between them.