Build an Accessible Waiting-for-Tokens Control With Stall Recovery

I tabbed into a tiny chat composer, typed one short prompt, and pressed Enter the way a keyboard-only user actually would. Three bouncing dots slid in under my message and looked extremely sure of themselves. Nothing spoke. I sat there for a long beat, still parked on a control that had just vanished, wondering whether the request had even left the browser. Have you ever watched a typing indicator and felt the whole interface go mute?

That mute stretch is the first-token gap. It shows up the moment a chat pane talks to a real remote server instead of a mocked stream that starts instantly.

The failure I was actually debugging

I was not chasing model quality, prompt style, or a cleverer system message. I was chasing a product failure that only existed after submit. The visual story said we were generating something useful. The accessibility story said absolutely nothing, and then a whole paragraph arrived at once. By then my focus had wandered into the browser chrome because the send button had unmounted. Does that sound like a streaming demo you have shipped on a happy path?

Here is the state table I should have drawn before I ever animated bouncing dots.

Phase	Visual	Focus	Live region
idle	Composer enabled, Send available	Composer	Silent
waiting	Status plus elapsed seconds, Stop available	Stay on Stop	Throttled “Waiting for the first token, N seconds”
streaming	Tokens append in place	Stop stays tabbable	One “Response started”, never per token
stalled	Recovery banner with Retry and Cancel	Move to Retry	“No tokens after N seconds. Retry available.”
error	Error text plus Retry	Retry	The error, once
complete	Composer enabled again	Restore composer	“Response complete”

If your machine only knows idle, streaming, and error, you are missing the state keyboard and screen-reader users actually get stuck in. Empty streaming is not streaming. It is a wait with no owner.

Symptom: the dots lied

I reproduced it with a single-file pane and a handler that slept before the first data: line. That delay is what a cold remote process feels like in the browser. The dots kept bouncing, so pointer users had motion to stare at. I had no motion in the accessibility tree.

What I observed, in order:

• A decorative typing indicator with no accessible name and no status text
• A Send button that disabled and then disappeared, so Tab lost its origin
• No elapsed time, so a two-second think looked identical to a dead socket
• A live region that stayed empty until tokens arrived, then spoke a novel

I kept asking one rude question out loud: is this loading, stalled, or broken? The UI refused to answer. That refusal is the bug, not the model.

Hypotheses I wrote down before touching CSS

I write hypotheses first because it stops me from “fixing” the spinner color. Debugging pixels is comforting. Debugging ownership is slower and usually correct.

1. The request never fired, and the composer submit handler was the real villain.
2. The request fired, but the first byte was slow because the remote process was cold.
3. Tokens arrived, but a remount made the live region miss the opening chunk.
4. The live region updated on every token and the speech queue fell behind.
5. Focus escaped to document.body when Send unmounted during the wait.

Number four is a real class of bugs, and I have chased it on other panes. This time it was not the backlog. Number two and number five were the actual faults. The dots were a costume over a missing waiting phase.

Root cause: no owner for “nothing has streamed yet”

Most streaming tutorials start at token zero as if the socket is already warm. Remote servers are often cold, especially when you are not paying for a dedicated box. The browser sits in waiting with zero tokens, and that is a real state. It has duration. It needs a cancel action. It needs a stall timeout that does not pretend to be a design flourish.

I had treated waiting as streaming with an empty string. Empty streaming looks identical to a hung fetch if you cannot see animation. Screen readers cannot see animation. Pointer users can at least glare at motion and wait. Keyboard users get a disabled control, silence, and a growing sense that the tab is dead. Why do we keep shipping that and calling it a chat UI?

The Send button also unmounted the moment the request started. That is how focus fell into the chrome. A Stop control that never receives focus is not a Stop control. It is a poster.

The typed states I should have started with

Treat the following as a reproduction, not a production SDK. I am showing the machine I used to debug the silent gap, including the stall threshold I picked for the harness.

type ChatPhase =
  | { status: "idle" }
  | { status: "waiting"; startedAt: number; elapsedMs: number }
  | { status: "streaming"; startedAt: number; text: string }
  | { status: "stalled"; elapsedMs: number; reason: "first-token-timeout" }
  | { status: "error"; message: string; recoverable: true }
  | { status: "complete"; text: string };

const FIRST_TOKEN_STALL_MS = 12_000;
const ANNOUNCE_EVERY_MS = 4_000;

Twelve seconds is a product choice, not a law of physics. Four seconds between wait announcements is also a choice, and it exists so you do not rebuild a live-region backlog. If you announce every tick, you will recreate a different failure and call it progress.

A runnable wait-state harness

This single file is enough to feel the bug and the fix. Serve it locally. Make the mock stream sleep before the first byte. Then tab through it with a screen reader and with the keyboard only.

<!doctype html>
<meta charset="utf-8" />
<title>Waiting-for-tokens harness</title>
<style>
  :root { font: 16px/1.45 system-ui, sans-serif; }
  #status { margin: 0.75rem 0; }
  #log { min-height: 4rem; white-space: pre-wrap; }
  [hidden] { display: none !important; }
  button:focus, textarea:focus { outline: 3px solid #222; }
</style>
<div>
  <label for="prompt">Prompt</label>
  <textarea id="prompt" rows="3">Explain focus restoration after submit.</textarea>
  <p>
    <button type="button" id="send">Send</button>
    <button type="button" id="stop" hidden>Stop</button>
    <button type="button" id="retry" hidden>Retry</button>
  </p>
  <p id="status" aria-live="polite" aria-atomic="true">Idle. Composer ready.</p>
  <div id="log" role="article" aria-label="Assistant draft"></div>
</div>
<script>
  const FIRST_TOKEN_STALL_MS = 12000;
  const ANNOUNCE_EVERY_MS = 4000;
  const els = {
    prompt: document.getElementById("prompt"),
    send: document.getElementById("send"),
    stop: document.getElementById("stop"),
    retry: document.getElementById("retry"),
    status: document.getElementById("status"),
    log: document.getElementById("log"),
  };

  let phase = { status: "idle" };
  let abort = null;
  let tick = null;
  let lastAnnounce = 0;

  function setPhase(next) {
    phase = next;
    const waiting = next.status === "waiting";
    const busy = waiting || next.status === "streaming";
    els.send.hidden = busy;
    els.stop.hidden = !busy;
    els.retry.hidden = next.status !== "stalled" && next.status !== "error";
    els.prompt.disabled = busy;
    if (next.status === "waiting") {
      const seconds = Math.floor(next.elapsedMs / 1000);
      maybeAnnounce(`Waiting for the first token, ${seconds} seconds.`);
    } else if (next.status === "streaming" && next.text.length === 0) {
      announce("Response started.");
    } else if (next.status === "stalled") {
      announce(`No tokens after ${Math.floor(next.elapsedMs / 1000)} seconds. Retry available.`);
      els.retry.focus();
    } else if (next.status === "error") {
      announce(next.message);
      els.retry.focus();
    } else if (next.status === "complete") {
      announce("Response complete.");
      els.prompt.focus();
    }
  }

  function announce(text) {
    els.status.textContent = text;
    lastAnnounce = Date.now();
  }
  function maybeAnnounce(text) {
    if (Date.now() - lastAnnounce >= ANNOUNCE_EVERY_MS) announce(text);
    else els.status.textContent = text; // visual elapsed time without extra speech
  }

  async function mockStream(signal) {
    // Local delay stands in for a cold remote first byte.
    await new Promise((resolve, reject) => {
      const id = setTimeout(resolve, 5000);
      signal.addEventListener("abort", () => {
        clearTimeout(id);
        reject(new DOMException("Aborted", "AbortError"));
      });
    });
    if (signal.aborted) return "";
    return "Focus stayed on Stop, then returned to the composer.";
  }

  async function start() {
    abort = new AbortController();
    const startedAt = Date.now();
    lastAnnounce = 0;
    els.log.textContent = "";
    setPhase({ status: "waiting", startedAt, elapsedMs: 0 });
    els.stop.focus();
    tick = setInterval(() => {
      if (phase.status !== "waiting") return;
      const elapsedMs = Date.now() - startedAt;
      if (elapsedMs >= FIRST_TOKEN_STALL_MS) {
        abort.abort();
        clearInterval(tick);
        setPhase({ status: "stalled", elapsedMs, reason: "first-token-timeout" });
        return;
      }
      setPhase({ status: "waiting", startedAt, elapsedMs });
    }, 500);
    try {
      const text = await mockStream(abort.signal);
      clearInterval(tick);
      if (phase.status === "stalled") return;
      setPhase({ status: "streaming", startedAt, text: "" });
      els.log.textContent = text;
      setPhase({ status: "complete", text });
    } catch (err) {
      clearInterval(tick);
      if (err.name === "AbortError") {
        if (phase.status !== "stalled") {
          setPhase({ status: "idle" });
          els.prompt.focus();
          announce("Generation stopped. Composer ready.");
        }
        return;
      }
      setPhase({ status: "error", message: "The request failed. Retry available.", recoverable: true });
    }
  }

  els.send.addEventListener("click", start);
  els.stop.addEventListener("click", () => abort && abort.abort());
  els.retry.addEventListener("click", start);
  els.prompt.addEventListener("keydown", (event) => {
    if (event.key === "Enter" && !event.shiftKey) {
      event.preventDefault();
      if (phase.status === "idle") start();
    }
    if (event.key === "Escape" && (phase.status === "waiting" || phase.status === "streaming")) {
      abort && abort.abort();
    }
  });
</script>

The important moves are boring on purpose. Stop exists before the first token. Focus moves to Stop, not into the void. Elapsed time is visible even when speech is throttled. Stall is a named transition with Retry as the new tab stop. Completion restores the composer. None of that requires ARIA fireworks. It requires a phase the dots were hiding.

Debugging technique: log the phase, not the pixels

I logged phase.status on every transition and ignored CSS until the log told a straight story. Pixel debugging would have nudged the bounce timing and left the mute gap intact. Phase debugging made the missing waiting owner obvious in about a minute.

Useful probes, in the order I actually used them:

1. Did fetch or the mock start before any token arrived, or did submit no-op?
2. What is document.activeElement right after Send unmounts?
3. Does the live region change during waiting, or only after streaming?
4. If I abort at eight seconds, does focus return, or does Stop vanish too?
5. If first byte exceeds the stall threshold, is Retry the next tab stop?

If you only watch the network panel, you will call a cold start “slow AI” and ship the same mute gap. The network panel cannot tell you who owns focus.

Where a remote free server made the wait honest

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A local five-second setTimeout is a clean reproduction, and you should keep it. It is also a liar in one important way: your laptop is already warm. I needed a remote target so first-token delay was not a cartoon. MonkeyCode is an open-source project with free model access and a free server option, so I pointed the same harness at that option and watched waiting become a real duration instead of a mocked sleep. I am not quoting latency numbers here because they move with load, and I will not invent model names, quotas, or hardware. The lesson is the state machine. The remote option only made the empty interval honest.

Expected UI states, in keyboard order

1. Focus the composer, type, and submit with Enter. Send must not steal the story by disappearing into nowhere.
2. Land on Stop immediately. Confirm you can press it without a pointer, including Escape as a synonym.
3. Hear a throttled wait announcement, not token-sized chatter and not total silence.
4. If the first byte never arrives, land on Retry and hear that stall is a named failure.
5. On success, restore the composer and announce completion once.
6. On user cancel, return to the composer with a short stopped message, not a blank status.

If step two fails, everything after it is theater. Users cannot cancel a wait they cannot reach.

Accessibility notes the bouncing dots skip

• Status text beats a CSS-only animation. Motion is not an accessible name.
• aria-live="polite" plus throttling beats a per-token feed. Speech queues are not free.
• Keep Stop mounted through waiting and streaming. Unmounting is a focus bug with extra steps.
• Pointer-independent paths matter: Enter to send, Escape to stop, Retry as a real button.
• Do not remount the transcript node when the first token arrives. In-place completion is a different article, but the wait state still has to survive that transition.

Environment-specific QA matrix

This is a regression matrix, not a conformance certificate. Please run the exact submit-to-first-token transition and write down versions.

Environment	What I check	Failure that counts
Chromium plus keyboard only	Submit, Stop, Escape, Retry	Focus lands on body after Send unmounts
Firefox plus keyboard only	Same path	Stall banner skipped in tab order
Safari plus VoiceOver (versions you record)	Wait announcements	Silence until a paragraph dump
NVDA plus Firefox or Chrome (versions you record)	Throttle vs backlog	Wait spoken every 500ms
Slow 3G throttle in DevTools	Elapsed time	Dots forever, no stall recovery

If you file a bug against this harness, I want the browser, OS, assistive technology, and the exact transition that failed. “It feels laggy” is not a transition.

Limitations, and who should not copy this blindly

This pattern does not make a cold server fast. It only makes a slow first byte explainable, cancelable, and recoverable. A twelve-second stall threshold will feel wrong in some products and too generous in others. Throttled polite announcements can still collide with other live regions on a busy page. A mock delay is not a service-level agreement, and a free remote server is not a promise about uptime, latency, or capacity.

Do not use this as a WCAG badge. Do not drop it into medical, legal, or emergency flows without a dedicated review. Do not use it if your “chat” is actually a fire-and-forget job with no user-facing cancel. And please do not treat bouncing dots as a wait state. They are decoration. Decoration cannot own focus.

Next time I feel tempted to add a typing cursor, I will ask a smaller question first. Who announces the wait, and where does the keyboard live until a token exists? If you need a remote endpoint while you answer that, MonkeyCode’s free model access and free server option are there to try against this harness.