Originally published on hexisteme notes.
I run a small fleet of coding agents, and one number had been bothering me for weeks: spawning a general-purpose sub-agent cost roughly 70,000 tokens before it did anything. I knew the rough shape of where that went — system prompt, built-in tool schemas, some kind of project configuration, and whatever the MCP layer was contributing — but I didn't have the breakdown, and "roughly 70k, mostly overhead" isn't something you can act on. You can't decide whether to trim the config injection or the tool catalog if you don't know which one is bigger.
So I built an instrument. A tiny local HTTP proxy that sits between my machine and the vendor's API, forwards every request unmodified, and writes the request and response bodies to disk before passing them through. Deliberately, it never writes headers to disk — request and response bodies are the only thing it records. I pointed my coding-agent CLI at it using the documented base-URL environment override, the one meant for routing through a gateway, and ran a batch of headless sessions through it to capture exactly what got sent.
The first captures looked like a different bug
The captures came back showing roughly 240,000 tokens per sub-agent spawn — a general-purpose spawn at 246,525 tokens, a restricted-tool explorer spawn at 212,736. That's more than three times what I expected. My first instinct was to go check real, un-proxied session transcripts for the same spawn types, and those showed something completely different: 70,733 tokens for a general-purpose spawn, 40,810 for an explorer spawn. Same spawn types, same tool configuration, off by roughly 3x in both cases.
The interpretation I reached for was: the headless entrypoint I was driving my captures through must not defer MCP tool schemas the way the interactive path does. My CLI has a lazy-loading behavior for its MCP tool catalog — instead of inlining every tool definition into every request, it defers most of them and only pulls in the full schema for a tool when it's actually about to be used. If that deferral simply wasn't wired up on the code path I was capturing from, the gap would make sense: the full catalog, on my setup, runs to 373 tool definitions across roughly 21 connected MCP servers — about half a megabyte of schema text — and inlining all of that into every request is exactly the kind of thing that would triple your token count.
It was a clean story. It matched the ratio. I wrote it down and moved on to the next question, which was how to close the actual decomposition I'd started this investigation to get.
Reproducing a number is not the same as checking an interpretation
Before trusting it further, I ran the finding through independent adversarial verification — three separate passes, each re-measuring from the raw captures rather than trusting my summary. All three reproduced the numbers exactly. Same 240k-ish figures, same ~70k anchors, same ratio.
That felt like confirmation, and for the numbers, it was. But three verifiers agreeing that a number is correct tells you nothing about whether the story you've attached to that number is correct. All three were checking arithmetic against the same captures I'd made — and the captures were all made the same way, through the same proxy, on the same headless entrypoint. If something about that setup was the actual cause, three independent people re-deriving the same average from the same contaminated inputs will agree with each other and still be wrong about why. Verification that stays inside the boundary of one measurement method can only tell you the measurement was done correctly, not that the thing being measured means what you think it means.
I didn't catch this at the time. I treated 3-for-3 as settling the question and moved forward with "headless doesn't defer" as an established fact for about half a day, until I ran the experiment that was actually designed to test it rather than confirm it.
Experiment one: it isn't headless-specific
The natural discriminating test was to stop varying the entrypoint and instead hold everything else constant while changing one thing at a time. First: is this actually about headless execution, or something else? I drove a full interactive session — the same kind a person would run at a terminal, via a pty — through the same proxy setup, and watched what its main turns cost.
288,259, then 289,372, then 291,366 tokens per main turn — and the sub-agents it spawned came back at 244,142 (general-purpose) and 213,420 (explorer). Just as inflated as the headless captures, in the same range, on a session type that has nothing to do with the headless entrypoint at all. Whatever was happening, it wasn't specific to how I'd been driving the headless runs. That ruled out my working theory in about the time it took the session to finish.
Experiment two: the proxy is the variable
If it wasn't the entrypoint, the next candidate was the one thing every inflated capture had in common: the proxy itself, specifically the base-URL override needed to route through it. So I ran the identical headless command again, unchanged, with the override removed — talking straight to the vendor's API, no proxy in the path.
First turn: 79,761 tokens. Normal. In line with the real transcript anchors I'd been comparing against all along, not the 240k-plus figures the proxy had been producing.
That closed it. Setting the base-URL override — which is what routing through any gateway or capture proxy requires — silently disables the tool-schema deferral. With the override in place, the CLI stops lazy-loading MCP tool definitions and inlines the full catalog into every request instead: 241,659 tokens for the same headless first turn that ran 79,761 without the override, roughly 3x, matching the ratio I'd been chasing since the very first capture. The proxy hadn't been passively observing the requests my CLI would normally send. Its presence changed what those requests were.
What the deferral is actually worth
Once I understood the mechanism, the anchors I already had turned into a measurement in their own right. The interactive run's proxied general-purpose spawn cost 244,142 tokens (its headless twin was 246,525 — both regimes inflate to nearly the same shape); the real, un-proxied equivalent ran 70,733. The difference — 173,409 tokens — is what MCP tool-schema deferral saves on a single sub-agent spawn, on a setup with roughly 21 connected MCP servers and a full catalog of 373 tool definitions. That's not overhead I'd been failing to find; it's overhead the deferral mechanism was already quietly absorbing, on every spawn, invisibly, because it worked.
It also meant the proxy could never observe the thing I actually cared about. With the base-URL override set, deferral-on assembly is unobservable by capture in principle — not because I hadn't built the proxy correctly, but because the override that lets the proxy see traffic at all is the same override that turns deferral off. There is no configuration of this instrument that watches the real, deferred request shape. The act of inserting a meter changes which bill gets generated.
Closing the decomposition indirectly
I still wanted the original answer: what is that 70,733-token spawn preamble actually made of? With direct capture off the table, I closed it by combining the shared components visible in the (inflated but structurally informative) proxied captures with the real usage anchors from un-proxied transcripts. The pieces that don't depend on deferral status — system prompt, built-in tool schemas, the per-session configuration injection — are visible in both regimes and consistent between them; only the tool-catalog portion swaps between "full schema" and "deferred stub" depending on which regime produced the request.
The reconstruction: a 70,733-token general-purpose spawn breaks down as roughly 2.4k tokens of system prompt, about 19.9k tokens of built-in tool schemas, around 20.1k tokens of injected user/project configuration, and approximately 28.2k tokens covering the deferred tool catalog stub plus whatever skills, sub-agent definitions, and MCP server instructions get carried along with it. None of those four numbers came from a single clean capture — each is the product of holding the other three fixed across regimes and reading off what changes. It's a slower way to get a decomposition than "read it off a proxy log," but it's the only way available once you know the direct route is closed.
What this is actually about
None of this is a claim that anything was broken or misbehaving. The deferral-disabling behavior on a custom base URL is, as far as I can tell, an intentional tradeoff — some capability, probably related to a beta feature gate, isn't available when requests are routed through an arbitrary endpoint, so the CLI falls back to sending everything inline instead of trusting a remote party to have the deferred-tool machinery. That's a reasonable thing for a vendor to do. It just means the instrument I built to measure normal behavior cannot see normal behavior, and I needed to find that out the hard way rather than assume it away.
Three things I'd tell someone building similar instrumentation:
First, an observability tap on an LLM pipeline is not passive by default. Anything that requires changing how a client talks to its backend — a proxy, a gateway, a custom endpoint, a header rewrite — is a configuration change to the system under test, not a window into it. Assembly logic can and does key off transport configuration in ways that have nothing to do with what you're trying to observe.
Second, validate the instrument against an un-instrumented baseline before you trust any absolute number it gives you. A single control run — the same command, the same inputs, proxy removed — would have caught this before I'd spent a verification cycle on the wrong theory. I had the anchors to make that comparison from the start; I just didn't run it until the interactive experiment forced the question.
Third, and this is the one I've made before in a different shape: reproduced numbers can carry a confounded interpretation, and only a discriminating experiment — one designed to separate two candidate causes, not just recheck arithmetic — kills the wrong reading. Three verifiers agreeing that 244,142 is really 244,142 tells you the measurement was executed correctly. It tells you nothing about whether "headless doesn't defer" or "the proxy disables deferral" is the right story behind that number, because both stories predict the exact same captures. Only removing the proxy and keeping everything else fixed could tell those two apart, and that's true of every instrumentation result that comes from a single method: the check that matters isn't whether the number replicates, it's whether the thing you changed to take the measurement is also the thing that explains it.
The one-time cost of running this investigation, proxy captures included, came to about 2.7 million cache-creation tokens — which is its own small irony, spent entirely on measuring measurement.
More notes at hexisteme.github.io/notes.
Top comments (0)