A Heartbeat-Only Stream Needs a First-Token Deadline, Not a Forever Generating Label

I rebuilt a one-file chat panel after a keyboard pass left the composer disabled and the status stuck on Generating. The mock inference server had already sent response headers, so the network row looked healthy and I almost blamed the model. A screen reader repeated the old polite status once, then stayed silent, as if tokens were still arriving somewhere. Have you ever left a spinner running because the status code never left the success range at all?

What the failure felt like in the tab order

I started from the keyboard, not from the console, because that is how the bug actually landed on a waiting user. Focus sat in a disabled textarea, the Stop control was missing, and the only change on screen was a pulsing label. The live region had announced Generating when the request began, and it had nothing new to say after that moment. Would you keep waiting if the interface never offered a spoken failure or a reachable cancel action?

The visual story and the accessibility story had diverged, which is a familiar trap in streaming chat interfaces. Sighted debugging saw a spinner and assumed latency, while assistive technology heard a frozen status with no recovery path. I wrote the expected states down before I touched the parser, because guessing from pixels had already wasted a pass. A table is slower to write than a guess, but it is the only place the missing state can show up clearly.

A state table before another spinner tweak

UI state	Entered when	Status text	Focus	Live announcement
idle	No request	Ready to send	Composer	None
connecting	Fetch started	Connecting to server	Stop button	Connecting
awaiting-token	Headers arrived, no token yet	Waiting for the first token	Stop button	Waiting for the first token
streaming	First text token	Generating	Stop button	Generating, then chunks by policy
stalled	Deadline elapsed with no token	No token arrived	Retry button	No token arrived. Retry is available.
cancelled	User activated Stop	Cancelled	Composer	Generation cancelled
done	Stream ended with text	Reply complete	Composer	Reply complete

That table made the hole obvious, because my code jumped from connecting straight into a forever Generating label. Headers and Server-Sent Events comments were being treated as proof that an answer had already started for the user. A heartbeat is a sign of life for the socket, not evidence that a reply exists and can be announced. If those two facts stay collapsed, every later accessibility tweak is decorating the wrong state and will miss the stall.

The frames I thought were tokens

I reproduced the stall with a local mock so the bug would not depend on a vendor outage or a quiet network. The server returns a success status and an event-stream content type, then writes comment heartbeats and never a data token. Curl makes that shape obvious if you refuse to let the browser developer tools hide the raw body from you. Have you checked whether your last slow token was actually a colon comment with no data line behind it?

What the raw frames are doing

curl -N --max-time 12 http://127.0.0.1:8787/mock-stream

// mock-server.mjs — local fixture only, not a vendor client
import { createServer } from "node:http";

createServer((req, res) => {
  res.writeHead(200, {
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache",
    Connection: "keep-alive",
  });
  const timer = setInterval(() => res.write(": keepalive\n\n"), 1000);
  req.on("close", () => clearInterval(timer));
}).listen(8787);

A naive reader increments a received counter on every chunk, including comments, so the UI believes generation has begun. I split frames into heartbeats and tokens, and I refused to change the visible state until a non-empty text delta appeared. The parser below is a fixture for that distinction, and it is not a complete Server-Sent Events implementation for production. Empty data lines stay null on purpose, because silence should not be promoted into assistant text that a reader might trust.

type Frame =
  | { kind: "heartbeat" }
  | { kind: "token"; text: string }
  | { kind: "done" };

function parseSseBlock(block: string): Frame | null {
  const lines = block.split("\n");
  if (lines.every((line) => line.startsWith(":") || line.trim() === "")) {
    return { kind: "heartbeat" };
  }
  const data = lines
    .filter((line) => line.startsWith("data:"))
    .map((line) => line.slice(5).trimStart())
    .join("\n");
  if (data === "[DONE]") return { kind: "done" };
  if (!data) return null;
  return { kind: "token", text: data };
}

These snippets are a local fixture assembled for this walkthrough, not a transcript from a production incident or a measured outage. I want the failure to be boring and repeatable before I point any client at a host I do not control. A green status code plus a moving waterfall is not a reply, and the fixture exists to keep that temptation visible. If your own log cannot mark comment frames, you will keep fixing the spinner and missing the stall.

Root cause: connected was masquerading as generating

The root cause was a collapsed state, not a slow paint and not a missing animation on the generating label. Fetch resolved, the reader loop ran, and each keepalive chunk cleared an inactivity timer that I had attached to the wrong event. Because the status string never changed, a polite live region had no mutation to announce, which is how a quiet failure hides. Why would a live region speak if we never changed the string it was watching and never moved focus to a recovery control?

I also found a focus bug sitting underneath the stall, and it would have survived a purely visual timeout tweak. The composer was disabled on submit, and Stop was rendered only in the streaming state, so awaiting-token had no keyboard action. A deadline that only logged to the console would have repeated a mistake I already knew from earlier stream bugs. Invisible recovery is not recovery, and a user who cannot tab to Stop is still trapped even after you detect the stall.

The fix is a deadline tied to tokens

I moved the clock to the awaiting-token state and let heartbeats update a debug line without touching that clock at all. Eight seconds is a demo threshold for this mock, not a recommendation for every model or every network path you ship. When the deadline fires, I abort the reader, move to stalled, and announce a specific failure that names the missing token. Would a generic something-went-wrong string tell a keyboard user whether to retry, wait, or check the connection?

type Phase =
  | "idle"
  | "connecting"
  | "awaiting-token"
  | "streaming"
  | "stalled"
  | "cancelled"
  | "done";

const FIRST_TOKEN_MS = 8000;

function onFrame(frame: Frame, phase: Phase, startedAt: number, now: number): Phase {
  if (phase === "awaiting-token" && now - startedAt >= FIRST_TOKEN_MS) return "stalled";
  if (frame.kind === "heartbeat") return phase;
  if (frame.kind === "token") return "streaming";
  if (frame.kind === "done") return phase === "streaming" ? "done" : "stalled";
  return phase;
}

function armFirstTokenDeadline(signal: AbortSignal, onStall: () => void) {
  const timer = setTimeout(onStall, FIRST_TOKEN_MS);
  signal.addEventListener("abort", () => clearTimeout(timer), { once: true });
  return () => clearTimeout(timer);
}

Status copy and focus

The status node stays in the same place in the accessibility tree, which avoids a focus-stealing toast when the phase changes. I use one assertive announcement only for the stall and cancellation, because those are interruptions a waiting user actually needs. Token appends stay polite or visual only, so a long reply does not bury the later stall message under a flood of characters. Stop remains a real button in tab order from the moment connecting begins, and it aborts the controller even before a token arrives.

<p id="stream-status" role="status">Waiting for the first token</p>
<p id="stream-alert" aria-live="assertive" aria-atomic="true"></p>
<button type="button" id="stop-stream">Stop generating</button>

Why aria-busy is not the announcement

Setting aria-busy on the log does not tell a screen reader that no token arrived, and it does not move focus to Stop. Busy is a hint about a region, not a recovery plan, and some users never hear it as a distinct failure at all. I still set aria-busy while connecting, but the stalled transition clears it and writes the assertive text beside the status. If you only toggle busy, you have changed an attribute and left the spoken status stuck on Generating for the user.

Retry creates a new request id and a new announcement, and it does not rewrite the previous empty attempt into fake assistant prose. The empty attempt stays in a diagnostics line as no token arrived, so a later success cannot be confused with the stalled turn. I log phase, frame kind, and elapsed milliseconds on each transition, because that one line made the heartbeat mistake undeniable. If your log cannot tell a comment from a token, you are still debugging the spinner rather than the stream itself.

A debugging loop I can reuse

1. Reproduce against a local mock before you blame the remote model, a remote server, or a flaky office network.
2. Capture raw frames with curl and the N flag, and mark comments, data lines, and the terminal event separately.
3. Draw the state table, including the waiting gap between response headers and the first user-visible text token.
4. Confirm the live region string actually changes on every transition you expect a screen reader to notice.
5. Tab the recovery path with a screen reader running, including the case where Stop is pressed before any token.
6. Only then point the same client at a remote endpoint, and compare frame kinds rather than the feeling of slowness.

These steps stay useful when the stall comes from a proxy, a sleeping laptop, or padding that keeps a socket open. The technique is the artifact I want to keep, which is separating transport liveness from answer progress in the client. Announcements and focus should bind to that distinction, or the interface will keep narrating a generation that never started. A remote endpoint is a later comparison, not the first place I try to understand a silent generating label.

A tiny log shape that survives the next bug

I keep a ring buffer of the last twenty frames as time, kind, and bytes, and I print it when the deadline fires. The buffer belongs in the bug report, not a spinner screenshot, because a screenshot cannot show the missing data line. Bytes matter here, because a comment heartbeat can be tiny and still look like real traffic in the network waterfall. If the buffer shows only heartbeat kinds, I stop profiling renders and go back to the parser before I touch CSS.

Trying the same probe against a free server

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I mention the project here because a free server is a practical place to rerun this probe after the mock is green. A vendor name does not fix a silent live region, and I do not want this section to sound like a loading-state miracle. The outreach notes for this draft describe free model access, a free server option, and a stated allowance of ten million tokens.

I have not measured that allowance, I am not listing model names or hardware, and I am not treating the offer as permanent. Read the current project terms before you depend on them, because an outreach figure can change without this article changing with it. If you want a remote target, the free server option is a fair next socket after your mock fails on purpose. Keep the first-token deadline, the raw frame log, and the keyboard Stop path when you leave localhost for that server.

A free endpoint can still heartbeat, stall, or return a body that is not a token, so the same bug remains yours to fix. Open-source availability does not remove those checks, and a large token allowance does not announce itself to a screen reader. If this probe matches a bug you already have, try the current MonkeyCode free server only after you confirm the live terms yourself. I would rather you reproduce the stall locally first than treat a hosted socket as proof that the interface states are finished.

Who should not use this deadline as-is

• Skip this client timer if your protocol already emits a typed error or a trusted first-token event before any heartbeat.
• Do not use the eight-second demo constant as a capacity plan, a quota monitor, or a benchmark of any hosted model.
• Do not treat aborting the reader as a security boundary, because it only returns control of the interface to the user.
• Avoid this pattern if your product must preserve partial tokens across reconnects, since this demo discards an empty attempt instead.
• Teams that cannot run a screen reader pass should not ship the status copy from this article without another review.

Limits of the reproduction

The mock never sends a real model delta, so it cannot tell you how a particular free model chunks text or emits comments. The eight-second constant will be wrong for long cold starts, and you should replace it from your own latency notes. A single polite region will still be too chatty if you later announce every token instead of phase changes only. I did not run a formal conformance audit, and this markup is a regression fixture rather than a claim of WCAG conformance.

Check	Environment note	Pass condition
Heartbeat does not enter streaming	Local mock, any browser	Status stays on waiting for the first token
Deadline announces stall	Screen reader on	No token arrived. Retry is available. is spoken
Stop before first token	Keyboard only	Focus returns to the composer and the request aborts
Retry after stall	Same page, no reload	New status announcement, old attempt not rewritten
Remote free server	After current terms are confirmed	Frame log distinguishes comments from tokens

Browser, operating system, and assistive technology versions will change announcement timing, which is why the matrix is part of the fix. If you rerun this, send the browser, the operating system, the assistive technology version, and the exact transition that failed. I care most about the jump from headers received to stalled, because that is the transition my first spinner completely hid. A generating label that never changes is not a loading strategy, and the first-token clock is what made that missing state visible.