When the Free Model Says 429: An Accessible Fallback Pipeline for AI Chat

A user asked a question. The widget spun. Then a red banner appeared: "Model unavailable." The user refreshed. The banner returned. The user closed the tab.

This scene repeats daily on shared AI infrastructure. Free models are shared. Free servers are shared. Rate limits are part of the agreement. Most frontends treat a model failure as a terminal error. They render a message and stop. The user is left with one option: leave.

The fix is not a better error message. The fix is a fallback pipeline. The frontend should try the next available model before it gives up. And when it switches, the user must know.

MonkeyCode offers free models and a free server option for experimentation. Both are shared resources, so throttling is real and observable. That makes it a practical lab for building fallback logic. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat model failure as a state, not an exception

A shared free server returns 429 Too Many Requests when the window is exhausted. That is not a bug. It is a scheduling signal. The frontend needs a state machine that understands the difference between:

• Temporary throttling — retry or fall back after a short delay.
• Model outage — skip this model entirely for the session.
• Permanent failure — stop and show a clear recovery path.

A boolean isLoading flag cannot express these three states. A typed union can.

type ModelStatus = 'available' | 'degraded' | 'unavailable';
type PipelineState =
  | { status: 'idle' }
  | { status: 'trying'; modelId: string; attempt: number }
  | { status: 'streaming'; modelId: string }
  | { status: 'fell-back'; from: string; to: string }
  | { status: 'exhausted' };

Every transition is explicit. Every transition can be announced. Every transition can be tested.

Build the fallback loop

The pipeline iterates through a priority-ordered model list. It skips models marked unavailable. It falls back on throttling. It stops only when every model has failed.

interface Model {
  id: string;
  priority: number;
  status: ModelStatus;
}

async function streamWithFallback(
  models: Model[],
  messages: Message[],
  signal: AbortSignal
): Promise<ReadableStream | null> {
  for (const model of [...models].sort((a, b) => a.priority - b.priority)) {
    if (model.status === 'unavailable') continue;
    if (signal.aborted) return null;

    try {
      const stream = await requestStream(model.id, messages, signal);
      return stream;
    } catch (error) {
      if (isThrottled(error)) {
        model.status = 'degraded';
        announce(`Model ${model.id} is busy. Trying the next one.`);
        continue;
      }
      if (isOutage(error)) {
        model.status = 'unavailable';
        announce(`Model ${model.id} is unavailable.`);
        continue;
      }
      throw error;
    }
  }

  announce('All models are busy. Please try again later.');
  return null;
}

Two details matter here. First, the loop checks signal.aborted between attempts. A user pressing Escape must stop the cascade immediately. Second, announce is not a console log. It is a live region update.

Make the fallback visible, announced, and cancellable

A silent model switch is an accessibility failure. Users need to know why the response quality changed. Screen reader users need the same information without visual cues.

<p id="pipeline-status" role="status" aria-live="polite">
  Checking available models…
</p>

The live region announces every transition. Focus stays on the input field. The user can keep typing while the pipeline tries the next model. The Cancel button remains enabled throughout.

Three rules apply to every fallback UI:

1. Never move focus during a fallback. Focus loss is disorienting for keyboard users.
2. Announce the model switch, not just the error. Users notice quality differences.
3. Keep cancellation pointer-independent. Escape must work during the entire pipeline.

Decision table for fallback actions

Not every error deserves a fallback. The table below maps error classes to actions.

Error class	Example	Action	User sees
Throttled	429 with Retry-After	Try next model	"Model busy. Trying next."
Server error	500, 502, 503	One retry, then fallback	"Retrying…"
Outage	503 with long duration	Mark unavailable	"Model unavailable."
Network loss	TypeError: fetch failed	Stop pipeline	"You are offline."
Abort	User pressed Escape	Cancel everything	"Stopped."

The table is a contract. The UI team and the API team agree on it before shipping.

Test the pipeline as an interaction

A fallback pipeline is not a component. It is a sequence of states. Test it with real throttling, not mocked delays.

1. Start a chat against the free server.
2. Send requests until the server returns 429.
3. Verify the live region announces the switch.
4. Verify focus stays on the input.
5. Press Escape mid-fallback.
6. Verify the pipeline returns to idle.

A minimal QA matrix for the fallback path:

Browser / OS	Screen reader	Transition tested	Result
Chrome / macOS	VoiceOver	throttled → fell-back → streaming	pass
Firefox / Windows	NVDA	streaming → Escape → idle	pass
Safari / iOS	VoiceOver	exhausted → retry button	pass

Run the matrix before you celebrate the feature.

Limitations and who should skip this

Fallback pipelines add complexity. They are unnecessary when you control a single model endpoint with generous limits. They are harmful when fallback models produce meaningfully worse answers without disclosure. Users would rather see an honest error than receive a silently degraded response.

The pipeline also does not fix capacity. If every model is throttled, the correct UI is a clear "try again later" state. A countdown that never lands is worse than no countdown at all.

Try it against a real shared server

MonkeyCode's free models and free server option reproduce these throttling patterns authentically. Send requests until the limit hits, then watch the pipeline walk down the model list. The failure is real, which makes the fix real.

The lesson is simple: a model failure is a transition, not a dead end. Build the state machine, announce every change, and let the user cancel at any moment. The chat widget that recovers gracefully is the one users trust.