When the Free Token Bucket Is Empty, Tell the User Before They Ask

The first useful lesson a free model endpoint teaches you is not how well the model writes; it is how the endpoint behaves when the boundary is hit. A free tier is a production environment in one specific sense: it can reject you, throttle you, and expire at an inconvenient moment, and if your interface collapses all of those states into a single vague error, you have replaced a precise system message with a mystery. This article shows how to turn that boundary into an accessible, announced state by running a small budget gate on a free server, because the failure path is exactly where most streaming AI demos stop being keyboard-operable or screen-reader-legible.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The free model access, the free server option, and the 30,000,000-token allowance I mention are operator-supplied availability claims. I did not treat the token figure as a benchmark or a promise, and the workflow below works with any budgeted model endpoint.

Current AI discussions are full of gatekeepers for agent tools and debates about watermarking, but those conversations tend to focus on what the model should be allowed to do or how its output can be recognized later. For a front-end developer, a more immediate problem is what happens when a request simply stops because the allowance is gone. The HTTP response may arrive as a 402, a 429 with a retry-after header, a provider-specific JSON payload, or a mixed format that a generic fetch wrapper was never taught to classify. The user, meanwhile, sees a spinner, then a button that says Retry, and then the same spinner again.

Think of an allowance the way a car treats a low-fuel warning. When the fuel is low, the driver gets a dedicated state with a clear decision: refuel soon or choose a nearer station. A chat interface should do the same. It needs to distinguish retryable throttling from terminal allowance exhaustion from a real upstream outage, because the correct next action is different in each case. One state might mean wait a few seconds, another might mean shorten the prompt, and another might mean check credentials or come back later.

MonkeyCode is an open-source project, but the part that matters here is the combination of a free model endpoint and a free server deployment path. I used that combination as a cheap way to provoke a real quota failure without spending my own infrastructure budget. If you already have a budgeted endpoint, you can place the same gateway in front of it.

Build the budget gate on the free server

Rather than teaching the client every proprietary error schema, put a normalizing gateway in front of the model request. The gateway passes through successful responses without inventing new meaning. When a request fails, it converts the upstream status and body into a small problem object the client can render as one announced state.

export default {
  async fetch(request) {
    const auth = request.headers.get('authorization') ?? '';
    const payload = await request.text();

    const upstream = await fetch('https://model-gateway.example/v1/chat', {
      method: request.method,
      headers: {
        'content-type': 'application/json',
        authorization: auth,
      },
      body: payload,
    });

    const retryAfter = upstream.headers.get('retry-after');
    const raw = await upstream.text().catch(() => '');

    if (!upstream.ok) {
      return Response.json({
        reason: classifyUpstream(upstream.status, raw),
        status: upstream.status,
        retryAfterMs: retryAfter ? Number(retryAfter) * 1000 : null,
      }, { status: 502 });
    }

    return new Response(raw, {
      status: upstream.status,
      headers: {
        'content-type': upstream.headers.get('content-type') ?? 'application/json',
      },
    });
  },
};

This is a Worker-style handler, so it can run in many free server offerings. The endpoint placeholder is deliberately not a real MonkeyCode path. Keep the gateway server-side, not in the browser, because the browser should never hold long-lived model credentials. The classifyUpstream function is the part you own: treat a 402, or a 429 whose body mentions balance, billing, quota, or insufficient tokens, as allowance_exhausted; treat a 429 with a retry-after header as rate_limited; treat 401 or 403 as unauthorized so you can explain a credential problem instead of asking someone to wait for a token bucket that is not the cause.

Make the client announce once, not constantly

On the client, do not announce every streaming chunk. Reserve live regions for state transitions. Keep an aria-live region mounted in the DOM from the start, then change only its text. Many screen readers will miss a live region if you change its role after insertion, so choose role=status for ordinary updates and switch the message, not the role, for urgent failures. For an allowance_exhausted transition, set the text to something specific, such as: The model allowance is empty. Shorten your prompt or retry after fifteen minutes. The retry control should be a real button in the tab order immediately after the status message, not a clickable span, so keyboard users do not have to search for it.

const region = document.querySelector('#stream-status');
const retry = document.querySelector('#retry-button');

function announce(state, retryAfterMs) {
  const message = {
    throttled: 'The request is rate-limited. Wait a moment and try again.',
    allowance_exhausted: `The model allowance is empty. Shorten your prompt${retryAfterMs ? ` or retry after ${Math.ceil(retryAfterMs / 60000)} minutes` : ''}.`,
    unavailable: 'The model service is not reachable. Your prompt was not sent.',
    unauthorized: 'The request could not be authorized. Check the server token.',
  }[state];
  region.textContent = message ?? '';
}

const result = { reason: 'allowance_exhausted', retryAfterMs: 120000 };
announce(result.reason, result.retryAfterMs);
retry.textContent = result.retryAfterMs ? 'Retry when ready' : 'Retry now';

Under the hood, model the request as explicit states: idle, waiting_for_tokens, streaming, throttled, allowance_exhausted, unavailable, retrying, and canceled. Each transition gets exactly one announcement. That keeps a screen reader from reading the full stream twice, and it keeps cancel and retry deterministic. When the user activates Retry, move to waiting_for_tokens, clear the old message, and announce again only when the next state actually changes.

What to test before you trust it

Run the failure path with NVDA and VoiceOver, then repeat it with a keyboard alone and with a pointer alone. The transition worth checking is this one: start a stream, activate Cancel, send a second request that hits allowance_exhausted, then verify that focus did not jump into a modal, the status message was read exactly once, and the Retry button is reachable with Tab. On iOS and older WebKit, test whether an assertive live region causes a focus change; usually you want aria-live=assertive without moving focus. Also open the browser console with network throttling enabled and confirm that the gateway returned the problem object instead of a blank caught promise.

The gateway does not create more allowance, does not make a free endpoint production-grade, and can be ignored by a client that never renders the problem object. Treat the free server route as a controlled space for prototypes and failure drills. Do not push personal messages, long-lived secrets, or high-stakes business traffic through it without checking the provider terms. If you need billing, per-user quotas, audit logs, or guaranteed latency, a managed API gateway is the correct tool rather than a small worker.

The next time you try a free model, do not measure how quickly it streams a successful answer first. Instead, force the boundary: exhaust the allowance, cancel mid-stream, hit a throttle, and watch how the screen reader and Tab key interpret each transition. If you have MonkeyCode's free server option available, host this gateway there and make the first version fail loudly on purpose. That is the fastest way to learn whether your interface treats an empty token bucket as a real state or just another generic error. Share the reproduction with your browser, operating system, and assistive technology versions, plus the exact transition that failed.