I was halfway through a keyboard-only pass on a streaming chat when the local path died. A skeleton overlay covered the composer, my focus jumped, and tokens started arriving from somewhere else. Did the interface tell me those tokens had left the browser? It did not, and that silence is the real defect.
Sighted teammates might notice a faint retry pulse and keep going. Keyboard and screen-reader users only felt a trap, then a voice that never named the new host. This write-up is a debugging retrospective, from that frozen caret to a typed origin state you can fail on purpose.
The failure I kept reproducing
I built a small chat shell that retries when a local inference call rejects. The retry helper reused status: "streaming", so the UI never distinguished on-device work from a remote server. That collapse is how a privacy change becomes a loading animation, and it is an easy mistake when failover feels like infrastructure rather than UX.
Here is the interaction I treated as the failing transition.
- Start: composer focused, local origin, chat idle
- Action: submit a prompt the local path cannot serve
- Observed: overlay appears, focus leaves the textarea, no announcement
- Then: tokens stream, overlay unmounts, focus is simply gone
- Expected: origin change is announced, consent is reachable, focus returns
The state table I wished I had on day one
If your table has one streaming row, you will hide origin changes from assistive tech. That is the pitfall, not the spinner’s CSS.
| State | Origin | Focus | Live region | Composer |
|---|---|---|---|---|
| idle | local | textarea | silent | enabled |
| streaming-local | local | textarea | Answer streaming on this device | disabled, cancel available |
| needs-consent | remote proposed | primary consent control | Local path failed. Remote send needs confirmation | disabled |
| streaming-remote | remote | textarea after confirm | Answer streaming from a remote server | disabled, cancel available |
| error | last known | retry, then composer | assertive error, once | enabled |
| cancelled | last known | composer | Request cancelled | enabled |
Why does a skeleton feel honest while still lying? Because waiting is visible, and destination is not.
Symptom to root cause
I started with the keyboard, not the network panel, because the complaint was focus loss. Tab after the overlay appeared cycled through chrome around the chat, then died against aria-hidden on the main landmark. The overlay had been marked aria-busy="true" on #app, which is a blunt instrument and a familiar false friend.
Was the fetch still using the local URL? DevTools said no. The retry wrapper swapped the endpoint after a single 503, then streamed as if nothing privacy-related had happened. The visual skeleton was honest about waiting and dishonest about where the bytes went.
Root cause, stacked in three mistakes I now look for first:
- One boolean,
isStreaming, represented local work, remote work, and the gap between them. - The overlay stole focus with
tabIndex={-1}plus an effect that calledoverlayRef.current?.focus(). - The live region only announced “Assistant is typing,” which is the same phrase for every origin.
None of those is an ARIA trivia problem. They are state-modeling problems that accessibility tools merely made visible. If you “fix” them by adding more live regions without an origin enum, you will drown the one sentence that mattered.
Debugging technique you can reuse
I now keep a three-column log while reproducing AI chat failures. It is slower than jumping into CSS, and it stops me from treating a privacy change like a spinner ticket.
- Input: key, pointer, or programmatic submit, plus the last focused element id.
- Network: which host answered, and whether the client chose it automatically.
- Accessibility tree: name, role, and live-region text at the moment of the transition.
If column two changes host while column three stays frozen, you have a silent origin bug. If column one and the accessibility tree disagree about focus, you have a trap. Write the log before you add another aria-* attribute, because attributes cannot recover a missing phase.
I also freeze the retry in a reducer so I can replay it without waiting on a real model. The sketch below does that with a local failure, then an optional remote path. Think of origin like a shipping label on a package you already started wrapping: the packing motion can look identical, but the destination is what people must hear.
A typed origin, not another spinner
A typed union forces every view to answer a rude question: where is this request going? Automatic failover is convenient for developers and hostile for people who cannot see the URL bar change.
type Origin = "local" | "remote";
type ChatPhase =
| { status: "idle" }
| { status: "streaming"; origin: Origin }
| { status: "needs-consent"; to: Origin; prompt: string }
| { status: "error"; message: string; origin: Origin }
| { status: "cancelled"; origin: Origin };
The needs-consent phase is the missing beat. Ask before the first remote token, keep that question in the tab order, then restore focus. I am not using a focus-trapped modal here on purpose: the last time I shipped a blocking dialog for a different chat prompt, testers lost the transcript. This banner stays in document flow.
stateDiagram-v2
[*] --> idle
idle --> streamingLocal: submit
streamingLocal --> idle: done
streamingLocal --> cancelled: stop
streamingLocal --> needsConsent: local 503
needsConsent --> streamingRemote: confirm
needsConsent --> cancelled: decline
streamingRemote --> idle: done
streamingRemote --> cancelled: stop
cancelled --> idle: type againMinimal reproduction
The following React sketch is a labeled demo, not a production privacy platform. It keeps the origin banner in tab order and returns focus to the composer after confirm or cancel. Screen-reader copy is short on purpose so the live region does not flood.
I needed a remote target that did not require a private GPU story in this article. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I pointed the failover branch at MonkeyCode’s free model access and free server option so the remote origin was real enough to test announcements, without standing up my own inference box.
import { useEffect, useReducer, useRef } from "react";
type Origin = "local" | "remote";
type Phase =
| { status: "idle" }
| { status: "streaming"; origin: Origin }
| { status: "needs-consent"; prompt: string }
| { status: "error"; message: string }
| { status: "cancelled" };
type State = {
phase: Phase;
draft: string;
answer: string;
log: string;
};
type Action =
| { type: "type"; value: string }
| { type: "submit" }
| { type: "local-fail"; prompt: string }
| { type: "consent-yes" }
| { type: "consent-no" }
| { type: "token"; chunk: string }
| { type: "done" }
| { type: "cancel" };
function reducer(state: State, action: Action): State {
switch (action.type) {
case "type":
return { ...state, draft: action.value };
case "submit":
if (state.phase.status === "streaming") return state;
return {
...state,
answer: "",
phase: { status: "streaming", origin: "local" },
log: "Answer streaming on this device.",
};
case "local-fail":
return {
...state,
phase: { status: "needs-consent", prompt: action.prompt },
log: "Local path failed. Confirm before sending to a remote server.",
};
case "consent-yes":
return {
...state,
phase: { status: "streaming", origin: "remote" },
log: "Answer streaming from a remote server.",
};
case "consent-no":
return {
...state,
phase: { status: "cancelled" },
log: "Remote send cancelled. Composer is ready.",
};
case "token":
return { ...state, answer: state.answer + action.chunk };
case "done":
return { ...state, phase: { status: "idle" }, log: "Answer complete." };
case "cancel":
return {
...state,
phase: { status: "cancelled" },
log: "Request cancelled.",
};
default:
return state;
}
}
export function OriginAwareChat() {
const [state, dispatch] = useReducer(reducer, {
phase: { status: "idle" },
draft: "",
answer: "",
log: "Chat idle on this device.",
});
const composerRef = useRef<HTMLTextAreaElement>(null);
const confirmRef = useRef<HTMLButtonElement>(null);
const abortRef = useRef<AbortController | null>(null);
useEffect(() => {
if (state.phase.status === "needs-consent") {
confirmRef.current?.focus();
return;
}
if (
state.phase.status === "idle" ||
state.phase.status === "cancelled" ||
state.phase.status === "error"
) {
composerRef.current?.focus();
}
}, [state.phase.status]);
async function send(origin: Origin, prompt: string, signal: AbortSignal) {
const response = await fetch(
origin === "local" ? "/local-infer" : "/remote-infer",
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ prompt }),
signal,
}
);
if (!response.ok || !response.body) throw new Error("infer-failed");
const reader = response.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { value, done } = await reader.read();
if (done) break;
dispatch({ type: "token", chunk: decoder.decode(value) });
}
}
async function onSubmit(event: React.FormEvent) {
event.preventDefault();
const prompt = state.draft.trim();
if (!prompt) return;
dispatch({ type: "submit" });
abortRef.current?.abort();
const controller = new AbortController();
abortRef.current = controller;
try {
await send("local", prompt, controller.signal);
dispatch({ type: "done" });
} catch {
if (controller.signal.aborted) dispatch({ type: "cancel" });
else dispatch({ type: "local-fail", prompt });
}
}
async function confirmRemote() {
if (state.phase.status !== "needs-consent") return;
const prompt = state.phase.prompt;
dispatch({ type: "consent-yes" });
const controller = new AbortController();
abortRef.current = controller;
try {
await send("remote", prompt, controller.signal);
dispatch({ type: "done" });
} catch {
if (controller.signal.aborted) dispatch({ type: "cancel" });
else dispatch({ type: "done" });
}
}
const blocked =
state.phase.status === "streaming" ||
state.phase.status === "needs-consent";
return (
<div className="chat">
<p aria-live="polite" aria-atomic="true" className="sr-only">
{state.log}
</p>
{state.phase.status === "streaming" && (
<p className="origin-banner" tabIndex={0}>
Origin:{" "}
{state.phase.origin === "local" ? "this device" : "remote server"}
</p>
)}
{state.phase.status === "needs-consent" && (
<div
className="origin-banner"
role="region"
aria-labelledby="consent-title"
>
<h2 id="consent-title">Send this prompt to a remote server?</h2>
<p>The local path failed. A remote model will see the prompt text.</p>
<button ref={confirmRef} type="button" onClick={confirmRemote}>
Send remotely
</button>
<button type="button" onClick={() => dispatch({ type: "consent-no" })}>
Stay on this device
</button>
</div>
)}
<div aria-live="polite">{state.answer}</div>
<form onSubmit={onSubmit}>
<label htmlFor="composer">Message</label>
<textarea
id="composer"
ref={composerRef}
value={state.draft}
disabled={blocked}
onChange={(event) =>
dispatch({ type: "type", value: event.target.value })
}
/>
<button type="submit" disabled={blocked}>
Send
</button>
<button
type="button"
onClick={() => {
abortRef.current?.abort();
dispatch({ type: "cancel" });
}}
disabled={state.phase.status !== "streaming"}
>
Stop
</button>
</form>
</div>
);
}
Notice the origin banner is a tab stop while streaming, not a toast. Toasts expire, and expired copy cannot be reviewed by a screen-reader cursor. The consent region sits in document order ahead of the composer, so Tab does not wander through the chrome first.
A little CSS keeps the banner readable without becoming another overlay that steals the page.
.origin-banner {
border: 2px solid currentColor;
padding: 0.75rem 1rem;
margin-bottom: 1rem;
}
.origin-banner:focus {
outline: 3px solid currentColor;
outline-offset: 2px;
}
.sr-only {
position: absolute;
width: 1px;
height: 1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
}
Do not put aria-busy on the document root during failover. Busy on #app hides the consent controls you just rendered, which is how I created the original trap.
Expected UI states
- Idle: composer enabled, no banner, live region quiet after the last completion.
- Local streaming: banner says “this device,” Stop is enabled, composer disabled but not
aria-hidden. - Failover: banner asks for consent, initial focus on “Send remotely,” live region fires once.
- Remote streaming: banner says “remote server,” same keyboard map as local streaming.
- Cancel or decline: polite copy once, focus back on the textarea, draft text preserved.
If any of those states reuse a single “loading” string, you are back to the original bug. Would you ship a checkout that retried against another processor without naming it? Then do not ship chat that way either.
Keyboard and screen-reader regressions
Run these as regressions, not as a one-off demo day. I write the exact transition into the bug title so the next failure is comparable.
- Submit with local failure. Focus must land on the consent primary button, not the overlay root.
- Press “Stay on this device.” Focus must return to the composer, not the document body.
- Confirm remote send. The live region must mention remote origin once, not on every token.
- Tab during remote streaming. The origin banner must be reachable without a pointer.
- Activate Stop. Streaming ends, the banner clears, and the composer must accept keys.
Do not announce every token. Token-level live regions bury the origin change under a speech queue nobody can interrupt cleanly. If your reader is still speaking fragments when consent appears, the user will confirm a prompt they never heard.
Environment-specific QA matrix
I am not claiming conformance from this matrix. I am claiming a reproducible path you can fail on purpose, with browser, OS, and AT versions recorded beside the transition.
| Environment | What I check | Failed transition to log |
|---|---|---|
| NVDA + Chrome on Windows | polite live region on consent, banner in browse mode | local fail → needs-consent |
| VoiceOver + Safari on macOS | focus move to confirm, then back to textarea | consent-no → cancelled |
| Keyboard only, Firefox | tab order banner → stop → no trap | streaming-remote |
| Reduced motion | no overlay animation that implies a different state | needs-consent |
Invite a teammate to reproduce with their versions plus the exact transition that failed. “It works on my laptop” is not a QA record when the bug is a silent host change.
What this does not solve
This pattern is for product teams who already stream chat in a browser and who sometimes leave the device. It is not a privacy policy, a DPA, or a substitute for documenting retention on the remote host. The sketch also skips auth, rate limits, and model catalogs on purpose.
Do not use this approach if your product never leaves the device. Do not use it if legal consent must be a separately captured workflow with audit storage. I am not attaching quotas, hardware, duration, or benchmark numbers to the free server path, because those claims would be invented here.
The accessibility fix is the state machine, the tab-stop banner, and the focus return. A reachable remote origin only made the failing transition cheaper to replay. If you want that replay without standing up your own box, try the free model access and free server option against the consent states above, and log the AT combo that still misses the origin change.
Top comments (1)
The "skeleton feels honest while still lying" framing is exactly right: the overlay tells the user that waiting is happening, which is the one thing they could already infer, and hides the one thing they can't - where their tokens are about to go. For a screen-reader user the skeleton isn't even neutral, it's an aria-busy shout over the live region you needed for the announcement.
Splitting streaming-local from streaming-remote in the state machine is the fix I wish more retry helpers had. The consent-first transition is also the right default: failover looks like infrastructure to the developer and like a data exfiltration event to the user. When you re-announce the origin mid-stream, do you keep the earlier on-device tokens in the same message, or do you mark the seam so the user knows part of the answer never left the browser?