The Free Budget Vanished in 90 Minutes: Debugging a Token Ledger That Charged Every Retry

A demo chat page went out to three people last month, and all three used it exactly the way a keyboard user does: type, press Enter, Tab to the Stop button, press Enter again because nothing seemed to happen. Ninety minutes later the free allowance was empty, and a screen reader had announced the remaining balance fourteen times in a row. The model was fine. My request bookkeeping was not.

What made this bug expensive is that nothing looked broken from the outside. Messages appeared, streaming worked, the Stop button even stopped things — eventually. The only visible symptom was a number going down faster than the conversation grew, and a live region that talked over itself on every chunk. So I stopped reading the UI and started reading the requests.

Here is the check order I now use before touching any chat code that spends someone else's budget. Print it above your desk and run it top to bottom; the first row that fails is usually the whole bug.

Symptom you can see	First thing to check	Typical root cause
Balance drops faster than the transcript grows	Count of outbound requests per user action	Retries sent without an idempotency key
Same answer appears twice in history	Size of the assembled prompt	Partial assistant turn re-sent as context
Two charges for one keypress	Event listeners on keydown and submit	Enter handler plus button handler both firing
Screen reader repeats the balance	Where textContent is written	Live region updated inside the stream loop
Stop button "works" but usage keeps rising	Attempt status at abort time	Aborted request never settled in the ledger

The three leaks hid behind one another

The reason this took me a day instead of an hour is that the leaks compounded. Retries were charged separately because they carried fresh random IDs, so the server had no way to recognise a replay. The transcript grew because an aborted assistant message stayed in state with partial text and no completion flag. And the double charge happened because a <form> submit and a keydown handler both called the same send function on Enter.

Notice what none of these are: model problems, network problems, or prompt problems. They are all state-identity problems. If you cannot point at a single value that identifies "this attempt", everything downstream — cancel, retry, dedupe, announce — is guesswork.

Instrument first: one ledger row per attempt, never per message

A message is a thing the user sees. An attempt is a thing the server charges for. My first fix was to name the difference in code and make the ledger the only writer of usage.

// ledger.ts — the single source of truth for attempts and usage
export type AttemptStatus = "streaming" | "done" | "aborted" | "failed";

export type Attempt = {
  attemptId: string;
  idempotencyKey: string;
  status: AttemptStatus;
  sentAt: number;
  settledAt?: number;
  promptChars: number;
  historyChars: number;
  reported?: { promptTokens: number; completionTokens: number };
};

export function createLedger() {
  const rows = new Map<string, Attempt>();

  function begin(idempotencyKey: string, promptChars: number, historyChars: number) {
    const row: Attempt = {
      attemptId: crypto.randomUUID(),
      idempotencyKey,
      status: "streaming",
      sentAt: Date.now(),
      promptChars,
      historyChars,
    };
    rows.set(row.attemptId, row);
    return row;
  }

  // Idempotent by design: settling twice is a no-op, not a second charge.
  function settle(attemptId: string, status: Exclude<AttemptStatus, "streaming">, reported?: Attempt["reported"]) {
    const row = rows.get(attemptId);
    if (!row || row.status !== "streaming") return row;
    row.status = status;
    row.settledAt = Date.now();
    row.reported = reported;
    return row;
  }

  function summary() {
    const all = [...rows.values()];
    const tokens = all.reduce(
      (n, r) => n + (r.reported?.promptTokens ?? 0) + (r.reported?.completionTokens ?? 0),
      0,
    );
    return {
      attempts: all.length,
      completed: all.filter((r) => r.status === "done").length,
      wasted: all.filter((r) => r.status !== "done").length,
      reportedTokens: tokens,
    };
  }

  return { begin, settle, summary };
}

The wasted counter is the one I actually watch during development. It counts attempts that started streaming and never finished, which is exactly the set of requests you paid for and the user never read.

Leak one: retries that were never marked as retries

The original code called crypto.randomUUID() for every send, including the one triggered from the error banner's Retry button. From the server's point of view those were three unrelated requests, and from my ledger's point of view they were three unrelated rows. Nothing was lying; nothing was deduplicating either.

The fix is to derive the key from what the user is asking, not from when they asked it. A conversation revision plus the prompt text is enough, and it stays stable across retries:

async function keyFor(conversationRevision: number, prompt: string) {
  const bytes = new TextEncoder().encode(`${conversationRevision}\u0000${prompt}`);
  const digest = await crypto.subtle.digest("SHA-256", bytes);
  return [...new Uint8Array(digest)]
    .slice(0, 8)
    .map((b) => b.toString(16).padStart(2, "0"))
    .join("");
}

Two details matter here. crypto.subtle only exists in a secure context, so this breaks on plain http:// LAN testing and you will spend twenty minutes on it. And the revision number must bump only when the history changes, not on every keystroke, or the key rotates and deduplication silently stops working.

Leak two: the transcript included the half-written answer

When a stream aborts, most implementations keep the partial text so the user does not lose it. That is good UX and terrible prompt hygiene, because the next request serialises that partial assistant turn back into the context window. You pay for it twice and you also confuse the model with an answer that was never finished.

The minimal repair is a complete flag on every turn and a filter during assembly:

 type Turn = { role: "user" | "assistant"; text: string; complete: boolean };

 function buildContext(turns: Turn[], maxChars = 12_000) {
-  const safe = turns;
+  const safe = turns.filter((t) => t.role === "user" || t.complete);
   const kept: Turn[] = [];
   let used = 0;
   for (let i = safe.length - 1; i >= 0; i--) {
     const cost = safe[i].text.length;
     if (used + cost > maxChars) break;
     kept.unshift(safe[i]);
     used += cost;
   }
   return { turns: kept, chars: used };
 }

A character budget is deliberately crude. It is not a token count and I do not pretend it is — I use it as a hard ceiling, then reconcile against the usage numbers the server reports in settle. The rule I follow is simple: estimate for control flow, trust reported usage for accounting, and never mix the two in the same variable.

Leak three: Enter and the button both fired

This one is boring and worth writing down anyway. A <form> with a submit button already sends on Enter in a text input. If you also listen for keydown and call send(), you get two attempts per keypress, and if the input is a <textarea>, Enter should insert a newline rather than send at all.

let inFlight = false;

async function onSubmit(event: SubmitEvent) {
  event.preventDefault();
  if (inFlight) return;
  inFlight = true;
  try {
    await send();
  } finally {
    inFlight = false;
  }
}

// Textarea: Cmd/Ctrl+Enter sends, plain Enter is a newline.
input.addEventListener("keydown", (event) => {
  if (event.key === "Enter" && (event.metaKey || event.ctrlKey)) {
    event.preventDefault();
    form.requestSubmit();
  }
});

requestSubmit() rather than submit() matters: it fires validation and the submit event, so there is exactly one path from keystroke to request.

The announcement bug: thresholds instead of a stream

My original announcer wrote the new balance into a aria-live="polite" region inside the chunk handler. With a fast stream that is a burst of DOM mutations, and screen readers queue them rather than coalesce them. Fourteen announcements for one message is not a bug the user can ignore.

The pattern that fixed it: announce only when a threshold is crossed, with a floor on repetition.

const THRESHOLDS = [50, 25, 10]; // percent remaining

export function attachBudgetAnnouncer(region: HTMLElement, totalBudget: number) {
  let deepestCrossed = -1;
  let lastSpoken = 0;

  return function report(remainingTokens: number) {
    const percent = Math.max(0, Math.floor((remainingTokens / totalBudget) * 100));
    const crossed = THRESHOLDS.findIndex((t) => percent < t); // -1 while above 50%
    const now = Date.now();
    const firstCrossing = crossed > deepestCrossed;
    const overdue = now - lastSpoken > 30_000;

    if (!(firstCrossing || overdue)) return;

    deepestCrossed = Math.max(deepestCrossed, crossed);
    lastSpoken = now;
    region.textContent = `About ${percent} percent of the free allowance left.`;
  };
}

Call it from settle(), never from the stream loop. The polite region stays quiet during generation, which is what a listener actually wants, and the visual counter can still update every chunk for sighted users.

A reproduction you can run in a few minutes

You do not need a real provider to see these leaks; a fake SSE endpoint that caches by key is enough and it makes the failure deterministic. Here is a dependency-free one:

// fake-model.mjs — node fake-model.mjs, then point the demo at localhost:8080
import { createServer } from "node:http";

const cache = new Map();
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

createServer(async (req, res) => {
  if (req.url !== "/stream") return res.writeHead(404).end();

  const key = req.headers["idempotency-key"] ?? "missing";
  res.writeHead(200, { "content-type": "text/event-stream" });

  if (cache.has(key)) {
    res.setHeader("x-replay", "1");
    for (const chunk of cache.get(key)) res.write(`data: ${chunk}\n\n`);
    return res.end();
  }

  const out = ["Retries ", "are cheap ", "until they ", "are not."];
  cache.set(key, out);
  for (const chunk of out) { res.write(`data: ${chunk}\n\n`); await sleep(150); }
  res.end();
  console.log("charged for key", key);
}).listen(8080);

Then run the three-step check below and watch the server log rather than the UI:

1. Send one message and count distinct keys in the log. One key means the happy path is clean.
2. Set the browser to offline at the second chunk, then press Retry. Count keys again — a new key per retry is leak one.
3. Hold Enter in a textarea for one second. Two log lines for one attempt is leak three.

This is where a free server option genuinely helped me, and the project I used is worth naming because it changed the loop. MonkeyCode is an open-source project whose free model access and free server option let me run this fake endpoint on a real host and drive the demo from a phone, which is the only way I caught the double-submit: the phone keyboard has no physical Enter, so the soft keyboard's "send" action was hitting a different code path than my laptop. The operator states that 10 million tokens are available on the free model access, which was enough for the harness and a handful of real-model comparisons; check the current terms on the project's docs before you plan around it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server also removed a subtler problem: testing keyboard flows across two devices used to mean two tunnels and a certificate warning. If you want to try the same setup, the docs are the place to start — and pick your own cap on requests before you share a link.

What I would still not do with a ledger like this

Not every team should copy this. Be honest with yourself about these cases.

• If you already pay a provider and have a staging environment, the free tier adds a variable instead of removing one. Use recorded fixtures and keep the ledger.
• If your product must be auditable, character estimates are disqualifying. Store server-reported usage as the only authoritative number.
• If your chat has no cancel or retry, you have less to deduplicate today, but the ledger is still cheaper than the incident that teaches you why you need it.
• If the demo is public, never point it at an uncapped account. A key derivation scheme reduces duplicates; it does not stop a stranger with a loop.

And a warning about the free tier specifically: free things change. Free model access and the free server are operator-supplied claims at the time of writing, not guarantees, so build the harness so it can point at any OpenAI-compatible endpoint and you are not stranded when terms move.

The QA matrix I run before sharing a link

Scenario	Keyboard only	Screen reader	Expected ledger result
Send one message	Enter in input	One polite announcement at settle	1 attempt, 1 key
Cancel mid-stream	Tab to Stop, Enter	No announcement during stream	1 attempt, status aborted
Retry after failure	Tab to Retry, Enter	Announced once, not per chunk	1 key reused, no double charge
Hold Enter	Soft keyboard action	Silent	1 attempt
Budget crosses 50%	n/a	Exactly one announcement	Threshold recorded once

Add your browser, OS, and assistive technology versions to that table when you file a bug. "Screen reader repeats the balance" is not a report anyone can reproduce; "NVDA 2026.x on Windows 11, Chrome, after the third chunk in a 200 ms stream" is.

If your streaming chat has a counter that only ever goes down, open the network panel and count keys for one message. My guess is you will find at least two, and one of them is a retry you never knew you were paying for.