Free AI endpoints usually do not fail because the model ran out of capability. They fail because the quota, rate limit, or upstream gateway returned a status your browser code never learned to read. The fix is not a more polished error toast; it is a small relay that forwards the operational metadata you need to turn a silent limit into an actionable state.
This week's AI feed has been occupied by watermarking and agent tool gates, but the failure that shows up in free-model demos is less dramatic and more common. A 429 or 402 from an upstream provider reaches the browser as a generic 500 from the developer's own server, so the user hears "Something went wrong" and cannot distinguish between "wait and retry," "you have spent the allowance," and "the connection actually died."
MonkeyCode positions itself as an open-source option with a free model allowance, currently stated as 30 million tokens, and a free server path you can use to lab-test this exact relay. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Treat the token figure as a claim to confirm in your own dashboard rather than a permanent contract; the implementation below is provider-agnostic.
The relay owns the failure contract
Once you add your own server between the browser and the model, you own the part that usually damages the failure contract. The browser fetch can only inspect the response you give it. If your relay catches the upstream 429, logs it, and returns res.status(500).json({ error: 'Upstream failed' }), you have deleted the information that would have told the client to wait.
app.post('/api/generate', async (req, res) => {
const upstream = await fetch(process.env.MODEL_URL, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(req.body),
});
const retryAfter = upstream.headers.get('retry-after');
res.status(upstream.status);
if (retryAfter) res.set('retry-after', retryAfter);
for await (const chunk of upstream.body) {
res.write(chunk);
}
res.end();
});
The important line is res.status(upstream.status), not the stream forwarding. When your server accidentally collapses every upstream status into 500, the frontend can only guess, and guessing produces the worst kind of retry button: one that offers a fresh request while the quota is still exhausted.
Map the status to a state, not a message
On the client, model the response as a state machine before you write any DOM updates. A rate limit is not an error; it is a temporary state with a clock. A spent allowance is terminal for the session. A transport failure is a different problem entirely.
type ChatState =
| { name: 'streaming'; controller: AbortController }
| { name: 'rateLimited'; retryAt: number }
| { name: 'quotaExhausted' }
| { name: 'failed'; message: string };
function handleResponse(res: Response): ChatState | null {
if (res.status === 429) {
const seconds = Number(res.headers.get('retry-after') ?? 0);
return { name: 'rateLimited', retryAt: Date.now() + seconds * 1000 };
}
if (res.status === 402) return { name: 'quotaExhausted' };
if (!res.ok) return { name: 'failed', message: `HTTP ${res.status}` };
return null;
}
A screen-reader user should not have to infer this from a red border or a toast that disappears. Place a single aria-live="polite" region in the DOM and never replace it. When the state changes, write a sentence, not a status name: "Rate limit reached. You can try again in 18 seconds." For a spent allowance, say "Free allowance used. Start a new session or switch the model route." That is the difference between a recoverable wait and an opaque failure.
Keep focus separate from announcement
Do not move focus to the error message just because it feels helpful. Focus movement is a navigation action; a live region is an update. For terminal states such as quota exhaustion, you may move focus to the retry or settings button, but do so once and only after the announcement has a chance to run. Keyboard users should not be yanked away from the transcript while a stream is still settling.
When a retry starts, call controller.abort() on the previous stream and replace the transcript writer. A free endpoint will sometimes deliver several chunks after the client has already moved on. If you share one reader variable across prompts, you can end up with old tokens appearing after the new prompt, and the screen reader will read a transcript that no longer matches the user's intent.
Test the four states without spending a paid key
Using the free server option rather than calling the model directly from the browser has two practical benefits here. You keep credentials out of frontend code, and you create a single place to standardize upstream status codes from any provider. That is what makes the retry logic testable instead of being a special case buried in the UI.
Run four upstream conditions against the same interface: a successful stream, a 429 with a valid Retry-After, a 429 without Retry-After, and a terminal allowance response such as 402. In each case, test with keyboard only, then with VoiceOver on macOS and NVDA or Narrator on Windows. Record the exact announced string and where focus rests. If the screen reader repeats the same "Something went wrong" phrase for all four states, the relay or the client mapping is wrong.
Limits of the approach
This design does not make a free allowance larger, and it cannot help if the upstream provider hides its status under a load balancer, or if the free server itself introduces a separate rate limit you have not designed for. If Retry-After is missing, show an estimated countdown but label it as an estimate. Never let the UI imply that a user can skip a wait the provider has not disclosed.
The valuable work is not the retry button. It is preserving the upstream's operational answer long enough for a keyboard or screen-reader user to act on it. Run the matrix once against a free endpoint such as the MonkeyCode free tier and free server path, and you will catch the quota bug before a paying user does.
Top comments (0)