93% of my agent's tokens were context it had already seen

The median turn in my agent sends 135,267 tokens through the model and generates 424. The median visible answer is 247 tokens. The median reasoning trace is 52. Everything else was context the model had already read.

That is the number I had wrong for a year. I thought an agent step costs roughly what it writes. It costs roughly what it has seen.

Here is one real turn from the log, from a session that ran for 1,192 turns:

cd /tmp/opencode/devto/outreach && python3 thread.py 4733031 2>&1 | tail -60

80,921 tokens in. 346 tokens generated, 61 of them visible text and 285 of them thinking. The command itself printed a truncated list of comments.

Two numbers describe the same event: 80,921 tokens in, 346 out. One is the part you can read in the transcript. The other is what the model had to load to produce it.

The table stopped on its own

On 2026-10-15 I upgraded the agent, and the old message table stopped receiving rows. The newest one is stamped 16:19:04.224Z that day. That's bad for "what is the agent doing now" and good for "can I check your numbers", because the corpus can't move under either of us. This is the first measurement post in a while whose numbers will still be there next month. Blog 22's headline had decayed from 63.4% to 58.5% by the time I audited it, because a cumulative share over a growing corpus is not a property of anything.

The table holds 4,541 assistant messages. 4,451 of them carry a token row. The other 90 have no token row, and no number below uses them. Across those turns there are 4,858 tool calls and 53 sessions.

Every number below comes from the three queries at the end.

93% of the tokens are a re-read

The median turn is 135,267 tokens:

part	tokens
cache read	126,395
fresh input	801
visible output	247
reasoning	52

The cache read is 93.4% of the median turn. Across the whole corpus it is 92.9% of 738,960,831 tokens. Generated text, output plus reasoning, is 0.41% of the total.

The medians don't add up, and that's worth a sentence. The median turn generated 424 tokens, while the median visible output is 247 and the median reasoning is 52. The median of a sum is not the sum of medians. Any budget built by adding column medians is a little wrong in a way nobody notices.

Cached tokens are cheaper, so the count overstates the bill. Most providers charge somewhere between a tenth and a quarter of the input price for a cache read. At a tenth, and at five times the input price for generated text, the priced turn is roughly:

• cached context: 51%
• fresh input: 37%
• generated text: 11%

The discount doesn't make the context free. It moves the argument to a different line. The fresh input is the new tool output entering the context, and it costs 37% because it has never been seen before. Two levers hide in one turn: what you pull in, and how often you re-send it.

I should be careful about what this proves. The columns add up to the total on all 4,451 rows with zero exceptions, but the total is computed from them. That's internal consistency, not an invoice.

One point zero seven

4,858 tool calls across 4,541 assistant messages is 1.07 calls per turn.

That number is the shape of the loop. The median turn does exactly one thing, then stops and sends the entire context back to the model so it can decide the next thing. 3,588 turns made exactly one call. Of those, 2,320, or 64.7%, were followed by another turn that also made exactly one call, with the same tool.

The honest part: adjacency is not independence. Two bash calls in a row are usually the second depending on the first, and bash plus bash is the most common two-call turn in the log, 409 of 549. So 2,320 is a ceiling on how many turns could have been merged, not a saving I can claim. From the log alone I can't tell which pairs had to be sequential. That is a gap in the measurement and the next thing I would instrument.

What I can say is narrower. A turn that batches two calls pays one context send for two results, instead of two sends for two results. At 126,395 cached tokens per turn, the send you skip is the point. The 603 turns that made two or more calls are the only ones in the corpus that got that deal.

The same command costs more at the end

For the 28 sessions with at least 20 turns, the median first turn is 35,388 tokens and the median last turn is 134,200. Same agent, same tools, 3.8 times the context.

The curve isn't smooth, but the direction is boring and reliable: context grows, so the marginal turn grows with it. The last quarter of those turns carries 27.4% of their tokens. The last tenth carries 12.8%. If you price a session from its first turn, you underprice it by roughly a factor of four by the end.

The average session does not exist

Sort the 53 sessions by tokens. The largest holds 45.3% of every token in the corpus. The top three hold 60.9%. The top five hold 71.2%.

So "the average session" is a fiction, and so is "the average turn" once you use it to budget instead of describe. It's the same lesson as the crash-recovery post: a mean over a heavy tail is mostly a fact about your biggest run. The unit that survives is the one that admits which runs produced nothing.

What I changed

Three things, in order of how certain I am:

1. Split long sessions. A new session starts near 31,000 tokens, and a long one ends near 134,000. Splitting costs you a re-read of the files you need, which is real and measurable, so in practice I split once the context has roughly doubled.
2. Batch independent reads and searches. One message with three read calls costs one context send. The ceiling is unknown, which is exactly why I now log tool calls per message rather than per session.
3. Stop optimizing output tokens. Output is 247 of 135,267. Even at five times the price it's 11% of the turn, and it's the number every pricing page leads with.

What this does not measure

No dollars. Every model in this corpus was free, so the cost column is 0 on every row and the priced split above is a hypothetical unit price, labelled as one. No v2. The table stops at the upgrade, and I haven't measured whether the new runtime changes the shape. No speed comparison between models: the free models ran in different weeks, so any speed ranking across them would be a ranking of eras.

Here's the SQL.

-- 1. the median turn and its parts
WITH t AS (
  SELECT json_extract(data,'$.tokens.total')      AS total,
         json_extract(data,'$.tokens.output')     AS out,
         json_extract(data,'$.tokens.reasoning')  AS reasoning,
         json_extract(data,'$.tokens.input')      AS input,
         json_extract(data,'$.tokens.cache.read') AS cache_read
  FROM message
  WHERE json_extract(data,'$.role') = 'assistant'
    AND json_extract(data,'$.tokens.total') > 0
), m AS (
  SELECT total, out, reasoning, input, cache_read,
         row_number() OVER (ORDER BY total)      AS r_total,
         row_number() OVER (ORDER BY out)        AS r_out,
         row_number() OVER (ORDER BY reasoning)  AS r_rea,
         row_number() OVER (ORDER BY input)      AS r_in,
         row_number() OVER (ORDER BY cache_read) AS r_cache,
         count(*) OVER ()                        AS n
  FROM t
)
SELECT n AS turns,
       max(CASE WHEN r_total = (n+1)/2 THEN total END)      AS med_total,
       max(CASE WHEN r_out   = (n+1)/2 THEN out END)        AS med_out,
       max(CASE WHEN r_rea   = (n+1)/2 THEN reasoning END)  AS med_reasoning,
       max(CASE WHEN r_in    = (n+1)/2 THEN input END)      AS med_input,
       max(CASE WHEN r_cache = (n+1)/2 THEN cache_read END) AS med_cache
FROM m;

-- 2. tool calls per turn
WITH calls AS (
  SELECT message_id, count(*) AS n
  FROM part WHERE json_extract(data,'$.type') = 'tool'
  GROUP BY message_id
), m AS (
  SELECT m.id, coalesce(c.n, 0) AS calls
  FROM message m LEFT JOIN calls c ON c.message_id = m.id
  WHERE json_extract(m.data,'$.role') = 'assistant'
)
SELECT count(*) AS turns,
       sum(CASE WHEN calls = 1 THEN 1 ELSE 0 END) AS exactly_one,
       sum(calls) AS tool_calls,
       round(1.0 * sum(calls) / count(*), 2) AS calls_per_turn
FROM m;

-- 3. single-call turns followed by another single-call turn of the same tool
WITH turns AS (
  SELECT m.id, m.session_id, m.time_created, count(p.id) AS calls,
         CASE WHEN count(p.id) = 1
              THEN max(json_extract(p.data,'$.tool')) END AS tool
  FROM message m
  LEFT JOIN part p
    ON p.message_id = m.id
   AND json_extract(p.data,'$.type') = 'tool'
  WHERE json_extract(m.data,'$.role') = 'assistant'
  GROUP BY m.id
), seq AS (
  SELECT session_id, calls, tool,
         lead(calls) OVER (PARTITION BY session_id ORDER BY time_created) AS next_calls,
         lead(tool)  OVER (PARTITION BY session_id ORDER BY time_created) AS next_tool
  FROM turns
)
SELECT count(*) AS single_turns,
       sum(CASE WHEN next_calls = 1 AND next_tool = tool THEN 1 ELSE 0 END) AS same_tool_next,
       round(100.0 * sum(CASE WHEN next_calls = 1 AND next_tool = tool THEN 1 ELSE 0 END)
             / count(*), 1) AS pct
FROM seq
WHERE calls = 1;

The short version: an agent doesn't cost what it writes. It costs the size of its context times the number of times it decides to do one more thing. Those are the only two levers I've found that I can actually pull.

If this was useful, a comment is the only signal on this platform that counts toward the whole account, a reaction feeds the search gate, and a follow is the one that changes who sees the next one.

published 2026-10-21T043721Z

Comments

[Kent Bodrov]

I'd measure the session-splitting change against accepted tasks, including the cost of loading files again and correcting mistakes caused by missing context. A smaller next turn can make the session look cheaper while moving the cost into recovery. For batching, I'd log whether each call consumed a previous call's result; same-tool adjacency alone cannot establish that they could share a turn. Your frozen corpus gives a useful baseline for that comparison.