The Token Meter Hit 100% and the Server Disagreed: A Usage-Accounting Retrospective

Last week my chat widget locked its own input at an estimated 9,830,000 tokens while the server's ledger said we had actually spent about 8,100,000. The meter was built to protect a free 10M-token allowance, so its early full stop had the right failure shape with the wrong numbers. Users who still had close to two million tokens of headroom were suddenly blocked, and a false alarm that disables your send button is a great way to lose trust.

I built the demo against MonkeyCode, the open-source project, using its free model access and its free server option, because a 10M-token pool sounded like plenty for a weekend prototype. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The cloud bill came to zero dollars; the debugging bill came to one long Saturday, and most of it was my own fault.

The Symptom That Deserved a Reproduction

A tester pasted a long conversation, the meter crossed its critical threshold, and the UI disabled the chat input with a warning banner that told her the allowance was gone. The next request would have been perfectly valid, yet the app went into lockdown, and the displayed "remaining" number refused to match the server dashboard. My first instinct was to blame rounding; my second was to capture the exact transcript and replay it against both counters.

The state table made the lie obvious:

UI state	Server ledger (10M pool)	What the user experienced
Input active	8,100,000 used	Normal
Warning banner	9,580,000 used	Confusion
Input disabled	9,580,000 used	Blocked for a reason that did not exist

Repro First, Theories Second

I wrote a tiny script that replayed the same message list through my estimation function and compared it with the server's reported usage, and the gap grew linearly with conversation length. A linear divergence means the bug lives in how history is counted, not in some single message, so I hunted for anything that multiplied with turns. Three suspects emerged, and all three turned out to be guilty.

Bug One: I Counted the System Prompt on Every Turn

My estimator summed the system prompt, tool definitions, and full history for every new request, then subtracted that total from the allowance. The system prompt was a fixed 2,400 tokens, so a 40-turn session quietly ate 96,000 phantom tokens, and the error only got worse as the conversation grew.

Bug Two: The Server Compacted History, and My Counter Did Not

The free server condensed older turns to keep context small, which is a normal cost-saving trick for long chats, but my client kept counting the original full messages each time. The prompt I actually sent was shorter than the prompt my meter believed I sent, and I had unknowingly built an odometer for a conversation that no longer existed.

Bug Three: Retries Billed Twice, and the Client Saw Only One Outcome

A flaky network dropped a response mid-stream, the client retried, the server charged both attempts, and my meter counted the single optimistic send. The usage object in the final streamed chunk was the only place the truth lived, and I was silently ignoring it. Is your retry button a free second chance, or a second invoice? In my UI, it was the latter.

Bug Four: My Tokenizer Was a Heuristic, Not a Tokenizer

text.length / 4 is a party trick, not an accounting method, and dense code, emoji, and non-Latin text diverge badly under that assumption. My calibration sample showed a stable ratio for prose and a wildly different ratio for code, which meant no single fudge factor could save me.

The Fix: Estimate for UX, Reconcile for Truth

The pattern that finally worked treats the client estimate as a fast, labeled guess and the server's usage report as the source of truth that corrects it. You display the estimate instantly, you replace it when the final chunk arrives, and you never let a guess disable the input.

const decoder = new TextDecoder();
let buffer = "";
let finalUsage = null;

const res = await fetch("/api/chat", { method: "POST", body: JSON.stringify({ messages }) });
const reader = res.body.getReader();

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split("\n");
  buffer = lines.pop();
  for (const line of lines) {
    if (!line.startsWith("data:")) continue;
    const event = JSON.parse(line.slice(5).trim());
    if (event.usage) finalUsage = event.usage; // the server's ledger
  }
}

reconcileMeter(finalUsage); // Server truth replaces the guess.

The UI state machine became explicit, and gating on confirmed numbers instead of estimates ended the false lockdowns:

Meter state	Condition	UI behavior
estimate	no usage chunk yet	Show "approx." badge; never block
confirmed	usage chunk received	Replace estimate; announce via aria-live
warned	confirmed > 85%	Non-blocking banner with a dismiss button
blocked	confirmed > 98%	Disable input and state the number

Hysteresis matters here, too: I only promote warned to blocked after two consecutive confirmed samples cross the threshold, so a single noisy report cannot brick the chat. The meter now answers two different questions, and the UI says which one it is answering at any moment. Estimated what you might spend, or confirmed what the server actually charged — those are not the same number, and pretending otherwise caused the entire bug.

The Calibration Artifact You Can Run

Here is the minimal calibration script I used to measure how far my heuristic drifted from the provider's real tokenizer. Run it against a known sample before you trust any client-side meter.

// calibrate.mjs — measure the ratio, then apply it as a correction
export function estimateTokens(text) {
  return Math.ceil(text.length / 4);
}

const REAL_SAMPLE = "Your longest system prompt plus a realistic user turn.";
const ACTUAL_TOKENS = 187; // from the provider's usage object for REAL_SAMPLE

export const calibrationFactor = ACTUAL_TOKENS / estimateTokens(REAL_SAMPLE);

export function correctedEstimate(text) {
  return Math.round(estimateTokens(text) * calibrationFactor);
}

A correction factor fixes systematic bias, not structural errors, so it cannot rescue my history-compaction bug. The real fix was deleting my homegrown ledger entirely and replaying the server's usage object as the only number that matters. That is the uncomfortable lesson: most client-side token meters are guesses wearing a calculator costume.

Accessibility Notes for Quota UI

Blocking input without announcing why is a classic accessibility failure, and my first version did exactly that. Keep the live region polite (aria-live="polite"), never rely on color alone to convey the threshold, and make the warning's dismiss button reachable by keyboard and screen reader. A meter is only useful if every user understands what it is measuring, so label estimate and confirmed states explicitly. The user who is blocked deserves a reason they can quote to support.

Here is the QA matrix I used; run it in your own environments and treat every cell as failing until you have data. My own first pass found one real regression in the Safari row.

Environment	Screen reader	Threshold announced?	Retry flow clear?	Blocked state explained?
Chrome / macOS	VoiceOver	?	?	?
Safari / iOS	VoiceOver	?	?	?
Firefox / Windows	NVDA	?	?	?

Limitations and Who Should Skip This

The 10M-token allowance and free server were the terms I worked with, and those terms can change, so verify the current limits before you build a hard cap around them. Server usage reporting can arrive late, batched, or not at all in some stream implementations, which is why the client estimate still exists as a fallback. If you are selling tokens or enforcing prepaid budgets, server-side enforcement is mandatory, and a client meter is decoration, not a billing system. Teams that need exact per-request accounting should skip client-side calibration entirely and query the provider's usage API after every turn.

The Hard-Won Takeaway

The meter was the product's most trusted surface, and it was wrong in four distinct ways, all of which looked reasonable in a code review. Counting is not estimation, and the server's ledger is the only ledger that pays the bill. Build your meter as a reconciliation loop, gate every hard decision on confirmed numbers, and let the interface say "approximately" out loud. Take the calibration snippet, feed it one of your own real conversations, and I suspect you will find at least one bug of your own.