DEV Community

babycat
babycat

Posted on

Streaming Chat Needs Cancel and Retry States, Not Just a Spinner

#a11y #webdev #javascript #ai

A chat demo that 'rewrites its own UI in real time' is fun to watch—until you try to stop it. I keep testing streaming AI interfaces where the keyboard experience is: press Enter, lose focus, hear nothing from my screen reader, and find that the only way out of a runaway generation is closing the tab. A spinner is not a state machine. This post builds a minimal streaming chat panel that stays keyboard-operable through loading, cancellation, failure, and retry, and uses a free hosted model so you can reproduce it without a GPU or a credit card.

The interaction failure, concretely

Open your favorite AI chat UI and try this sequence with only a keyboard:

  1. Submit a prompt that generates a long answer.
  2. Try to stop it mid-stream with Escape.
  3. Let the request fail (or throttle your network in DevTools).
  4. Try to retry the last prompt.

In many implementations: focus is stuck in the composer, Escape does nothing, the stream silently dies, and 'retry' means retyping your prompt. Screen reader users fare worse—aria-live either announces every token (a firehose) or nothing at all.

The state table first

Before any code, name the states. If a state isn't named, it isn't handled:

State Trigger Visual Keyboard Announcement
idle initial composer enabled Enter submits none
streaming submit Stop button visible, tokens render Esc or Tab→Stop cancels 'Response started' (once)
cancelling cancel 'Stopping…' — none
cancelled abort settled partial text kept, Retry offered focus returns to composer 'Response stopped. Partial answer kept.'
error fetch/stream throws error region + Retry button focus moves to Retry 'Error: . Retry available.'
done stream ends full text, Retry/Edit offered focus returns to composer 'Response complete, 42 words'

Two rules drive everything below: partial output is data, not garbage (keep it on cancel/error), and every transition that the user caused must be announced and focus-manageable.

A typed state machine 

type ChatState =
  | { status: 'idle' }
  | { status: 'streaming'; controller: AbortController; text: string }
  | { status: 'cancelling'; controller: AbortController; text: string }
  | { status: 'cancelled'; text: string; prompt: string }
  | { status: 'error'; message: string; text: string; prompt: string }
  | { status: 'done'; text: string; prompt: string };

type ChatEvent =
  | { type: 'SUBMIT'; prompt: string; controller: AbortController }
  | { type: 'TOKEN'; delta: string }
  | { type: 'CANCEL' }
  | { type: 'FAIL'; message: string }
  | { type: 'SETTLE' }; // stream ended, or abort settled
Enter fullscreen mode Exit fullscreen mode

The point isn't the syntax—it's that cancelled and error both carry text and prompt, so the UI can keep the partial answer and offer a real retry without re-asking the user for input.

Runnable single-file demo

Save as stream-chat.html and open in a browser. It uses EventSource-free fetch streaming against any OpenAI-compatible endpoint, with the endpoint URL and key read from two inputs so nothing is hardcoded. (For a zero-setup run, see the note on free hosted access below.)

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Accessible streaming chat</title>
<style>
  [hidden]{display:none}
  #log{min-height:8rem;border:1px solid #999;padding:.75rem;white-space:pre-wrap}
  #status{margin:.5rem 0}
  .error{color:#8b0000}
  button:focus-visible, textarea:focus-visible, input:focus-visible{
    outline:3px solid #005fcc; outline-offset:2px;
  }
</style>
</head>
<body>
  <h1>Streaming chat with cancel and retry</h1>

  <label>Endpoint <input id="endpoint" size="40"
    placeholder="https://your-host/v1/chat/completions"></label>
  <label>API key <input id="key" type="password" size="24"
    placeholder="leave blank if none needed"></label>

  <div id="log" aria-label="Conversation"></div>

  <!-- Polite, throttled announcements: never aria-live the token stream itself -->
  <div id="status" role="status"></div>

  <label for="prompt">Prompt</label>
  <textarea id="prompt" rows="3"></textarea>
  <button id="send">Send</button>
  <button id="stop" hidden>Stop (Esc)</button>
  <button id="retry" hidden>Retry last prompt</button>

<script type="module">
let state = { status: 'idle' };
const $ = id => document.getElementById(id);
const log = $('log'), statusEl = $('status'), promptEl = $('prompt');

function announce(msg){ statusEl.textContent = msg; }
function render(){
  $('stop').hidden  = state.status !== 'streaming';
  $('retry').hidden = !(state.status === 'cancelled' || state.status === 'error');
  $('send').disabled = state.status === 'streaming' || state.status === 'cancelling';
}

async function run(prompt){
  const controller = new AbortController();
  state = { status: 'streaming', controller, text: '' };
  render(); announce('Response started. Press Escape to stop.');

  const div = document.createElement('div');
  log.append(div);

  try {
    const res = await fetch($('endpoint').value, {
      method: 'POST',
      signal: controller.signal,
      headers: {
        'content-type': 'application/json',
        ...($('key').value ? { authorization: `Bearer ${$('key').value}` } : {})
      },
      body: JSON.stringify({
        model: 'default', stream: true,
        messages: [{ role: 'user', content: prompt }]
      })
    });
    if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);

    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let buf = '';
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      buf += decoder.decode(value, { stream: true });
      // Parse SSE lines; tolerate partial lines.
      for (const line of buf.split('\n')) {
        if (!line.startsWith('data:')) continue;
        const payload = line.slice(5).trim();
        if (payload === '[DONE]') break;
        try {
          const delta = JSON.parse(payload)?.choices?.[0]?.delta?.content ?? '';
          if (delta && state.status === 'streaming') {
            state = { ...state, text: state.text + delta };
            div.textContent = state.text; // visual only; NOT announced per-token
          }
        } catch { /* keep buffered partial JSON for next chunk */ }
      }
      buf = buf.endsWith('\n') ? '' : buf.slice(buf.lastIndexOf('\n') + 1);
    }
    const words = state.text.trim().split(/\s+/).length;
    state = { status: 'done', text: state.text, prompt };
    announce(`Response complete, ${words} words.`);
  } catch (err) {
    if (err.name === 'AbortError') {
      state = { status: 'cancelled', text: state.text, prompt };
      announce('Response stopped. Partial answer kept. Retry available.');
    } else {
      state = { status: 'error', message: err.message, text: state.text, prompt };
      div.classList.add('error');
      div.textContent += `\n[error: ${err.message}]`;
      announce(`Error: ${err.message}. Retry available.`);
      $('retry').focus(); // move focus to the recovery action
    }
  }
  render();
  if (state.status !== 'error') promptEl.focus(); // return focus to composer
}

$('send').addEventListener('click', () => {
  const prompt = promptEl.value.trim();
  if (prompt && state.status !== 'streaming') run(prompt);
});
$('retry').addEventListener('click', () => {
  if (state.status === 'cancelled' || state.status === 'error') run(state.prompt);
});
$('stop').addEventListener('click', () => {
  if (state.status === 'streaming') state.controller.abort();
});
document.addEventListener('keydown', e => {
  if (e.key === 'Escape' && state.status === 'streaming') state.controller.abort();
});
promptEl.addEventListener('keydown', e => {
  if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); $('send').click(); }
});
render();
</script>
</body>
</html>
Enter fullscreen mode Exit fullscreen mode

What to run it against (free tier works)

You need any OpenAI-compatible streaming endpoint. I reproduced this with MonkeyCode's free model access on their free server option—no local hardware, no card—which makes it a convenient sandbox for exactly this kind of failure-state testing. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Two honest caveats: I make no claims about quotas, model names, latency, or how long the free tier lasts, and a shared free server is the wrong place to measure streaming performance—use it to verify interaction correctness, not throughput. If you already have another compatible endpoint, it will work unchanged; that's the point of keeping the demo provider-agnostic.

Why the announcements are throttled

The tempting move is aria-live="polite" on the token container. Don't. Screen readers will queue every DOM mutation and read a garbage stream of fragments. The demo announces transitions, not tokens: started, stopped (with the fact that partial text was kept), complete (with a word count as a sanity signal), and error (with the message and the retry affordance). Users who want the full text can navigate to the log region themselves—it has a label and stable position.

QA matrix for your own build

Scenario Keyboard only Screen reader Expected transition
Submit → Esc mid-stream Esc aborts, focus → composer hears 'stopped, partial kept' streaming → cancelled
Network offline at submit Retry button receives focus hears error + retry hint streaming → error
Retry after error Enter on Retry resubmits last prompt hears 'Response started' error → streaming
Server 429/500 mid-stream partial text retained error announced, not silent streaming → error
Slow stream, user tabs to Stop Stop reachable by Tab button labeled 'Stop (Esc)' pointer-independent cancel

Test with real versions: I use Chrome/Edge + NVDA on Windows and Safari + VoiceOver on macOS, and I throttle to 'Slow 3G' in DevTools to make the mid-stream states actually observable.

Limitations and who shouldn't use this pattern as-is

  • Single-turn only. Multi-turn history, edits, and branching need the state machine extended, not bolted on.
  • No backpressure handling. Very fast streams can outrun textContent updates; batch with requestAnimationFrame in production.
  • The retry replays the same prompt verbatim. If your backend is non-idempotent (tool calls, side effects), retry needs a conversation-level idempotency key.
  • Free hosted tiers are sandboxes. Don't point production traffic or sensitive data at them; check the provider's terms and rate limits before building a workflow on top.

If you reproduce the sequence at the top of this article against your own chat UI and hit a transition that fails, tell me your browser, OS, and assistive technology versions plus exactly which transition broke—those bug reports are how this pattern gets better.

The whole point: 'generating…' is not one state. It's at least five, and your users can feel every one you skipped.

Top comments (0)