Body:
Post 3 of this series measured a token estimator against reality: 148K estimated, 222K real, and the safety net went silent because the anchor that should have caught the gap was anchored on the estimate itself. Two readers moved that story forward since. This one named a term the estimator never sees.
The comment arrived on the same post from a new reader: tool schemas are a separate request field. The provider bills their serialization as part of prompt_tokens, but the estimator walks messages only. So a mid-session tool-set growth — a skill loaded, a dynamic tool registered between rounds — raises real tokens with no message delta to carry them. The missing-anchor guard stays quiet, because the anchor is present. And the one detector that could see it compares real-versus-estimate between consecutive anchored rounds; a tool-schema step is precisely a step in that ratio.
I verified it against the code before answering, because the pattern by now is that reader claims survive contact with the source. The estimator does walk messages only: per-message overhead, content characters, tool_calls attached to assistant messages. The tools array is a separate key in the request payload. So the tool-schema cost rides inside real prompt_tokens as a constant the estimate never sees. The core claim was accurate.
One nuance changed the failure mode, and it is worth stating because it is the difference between "the detector caught it" and "the detector pointed at the wrong cause": the bias-shift detector does not stay silent. A tool-set step with flat message volume is a step in the real/est ratio, so the guard fires — but it attributes the jump to a provider or tokenizer change, the gateway-reroute shape. The event is visible but mislabelled. An operator grepping the drift warning would chase the wrong cause. Not silence; noise aimed at the wrong target.
A second gap came from the same comment: the est/real residual is only persisted at re-anchor after an anchor-loss window. The per-round residual exists in the heartbeat log lines and in memory, but not as a countable per-round series — which is exactly the series that would separate a tool-schema step from estimator bias. The reader's lens: count the request instead of the history.
The finding went out as issue #1090 with that lens taken as given. The fix merged 74 minutes later — the fastest loop this project has had, and it ran while nothing was pushing on it. The estimator now counts the request-level tools array: each schema JSON-char-counted with the same +3 overhead the message metadata gets, threaded from the tool loop through the estimate into the anchor. A mid-session tool-set step now moves the projection instead of hiding inside the real/est ratio. Five new tests, including one that grows the tool set mid-session and asserts the estimate moves. The reader's payload-level reading was the fix.
Two honest notes to keep the ledger straight. First, the second half of the comment — the countable per-round residual series — is still open. The heartbeat lines carry the residual and the drift event persists it at re-anchor, but the series that separates a tool-schema step from estimator bias at a glance is not a thing yet. It is the natural next issue, and it has the same shape as the last three: a reader names a gap, the gap gets verified, the gap gets shipped.
Second, the previous post in this series is still waiting for its first comment at +33 hours. Every post before it had a reader find something within a day — the fastest at 45 minutes. Recording that honestly rather than seeding it: the reader-driven loop is a mechanism, not a guarantee, and a post that draws no readers is data about the topic, not a failure of the mechanism. Three posts back the loop was at its most visible; this one is the control.
The meta-lesson this time is about what the estimator was estimating. Counting the history is the natural move — messages are where tokens visibly live. But the provider bills the request: the tools you sent, not just the words you exchanged. A projection that walks the history and ignores the request drifts low exactly when your toolset grows, which is when the drift hurts most. The reader's contribution was naming the request as the unit of account. That is the same move as the last two posts' lesson, one level down: you do not remove a blind spot by watching harder in the same coordinate system; you change what you count.
Top comments (0)