Streaming Tokens Need a Frame Budget: Keeping Chat Input Responsive at 60 FPS

I asked a free model to write a 2,000-word essay inside a streaming chat widget, and the input box froze so badly that my cursor stopped blinking. The network tab showed a healthy connection; the model was generating tokens faster than my renderer could paint them, and the whole interface paid the price.

That experiment ran on MonkeyCode, an open-source AI assistant that offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The free tier made the test trivial to set up, but the performance problem it exposed is universal: every streaming token is a DOM write, and DOM writes are not free.

This post is a rendering retrospective, not a network debugging story. I measured the frame drops, found three bottlenecks, fixed them in order, and ended up with a chat interface that stays responsive even when the model is outputting a hundred tokens per second.

The 12 FPS Symptom

The first sign of trouble was the caret. I typed a follow-up message while the model was still streaming, and the caret moved in visible jumps instead of a smooth glide. Then the input started dropping keystrokes entirely, and I knew the main thread was saturated.

I attached a requestAnimationFrame loop to measure frame times, and the numbers confirmed the diagnosis. The median frame time during streaming was 83 milliseconds, which works out to roughly 12 frames per second. A healthy interface needs 16.7 milliseconds per frame to hit 60 FPS; I was five times over budget.

The key measurement was key-to-render latency: the time between a keystroke and the character appearing on screen. At rest, it was 8 milliseconds. During streaming, it ballooned to 240 milliseconds, which is the difference between typing feeling instant and typing feeling like remote desktop software.

Bottleneck 1: One DOM Write per Token

The naive render loop appends each token as it arrives, and each appendChild invalidates the layout of the entire chat container.

// The naive approach: one DOM write per token
async function naiveRender(tokens) {
  for (const token of tokens) {
    const span = document.createElement("span");
    span.textContent = token;
    container.appendChild(span);
    // Each appendChild forces a synchronous layout
  }
}

This looks harmless, but it has a quadratic cost. Every new token forces the browser to recalculate the layout of every previous token in the container, and as the conversation grows, each append gets more expensive. A 500-token response is not 500 layout calculations; it is 125,000 of them.

Fix 1: Batch With requestAnimationFrame

The first fix is almost embarrassingly simple: accumulate tokens in a buffer, then flush them all in a single DOM write on the next animation frame.

// Batched approach: one DOM write per frame
let tokenBuffer = [];
let rafId = null;

function scheduleFlush() {
  if (rafId) return;
  rafId = requestAnimationFrame(() => {
    const fragment = document.createDocumentFragment();
    for (const token of tokenBuffer) {
      const span = document.createElement("span");
      span.textContent = token;
      fragment.appendChild(span);
    }
    container.appendChild(fragment);
    tokenBuffer = [];
    rafId = null;
  });
}

function renderToken(token) {
  tokenBuffer.push(token);
  scheduleFlush();
}

The median frame time dropped from 83 milliseconds to 21 milliseconds, and key-to-render latency fell to 45 milliseconds. The browser was going to paint all those DOM changes on the next frame anyway; batching them just eliminated the redundant layout calculations in between.

Bottleneck 2: The Chat Log Grows Without Bound

Batching fixed the per-token cost, but the chat log itself kept growing. Every new token appended to the container made the next layout calculation more expensive, and after a few thousand tokens, even batched appends started to hurt.

The fix was virtualization: render only the tokens that are actually visible in the viewport, and recycle DOM nodes as the user scrolls.

// Virtualized chat log: render only visible tokens
class VirtualizedChatLog {
  constructor(viewport, rowHeight = 20) {
    this.viewport = viewport;
    this.rowHeight = rowHeight;
    this.tokens = [];
    this.container = document.createElement("div");
    this.container.style.position = "relative";
    viewport.appendChild(this.container);
  }

  appendToken(token) {
    this.tokens.push(token);
    this.render();
  }

  render() {
    const scrollTop = this.viewport.scrollTop;
    const height = this.viewport.clientHeight;
    const start = Math.floor(scrollTop / this.rowHeight);
    const end = Math.ceil((scrollTop + height) / this.rowHeight);

    const fragment = document.createDocumentFragment();
    for (let i = start; i < Math.min(end, this.tokens.length); i++) {
      const div = document.createElement("div");
      div.textContent = this.tokens[i];
      div.style.position = "absolute";
      div.style.top = `${i * this.rowHeight}px`;
      div.style.left = "0";
      div.style.right = "0";
      fragment.appendChild(div);
    }

    this.container.innerHTML = "";
    this.container.appendChild(fragment);
    this.container.style.height = `${this.tokens.length * this.rowHeight}px`;
  }
}

Virtualization keeps the DOM size proportional to the viewport, not the conversation length. A 5,000-token conversation renders the same number of nodes as a 50-token one, and the frame time stays flat.

The Measured Results

Here is the before-and-after table from the same 2,000-word generation:

Strategy	Median frame time	Key-to-render latency	DOM nodes at 5,000 tokens
One DOM write per token	83 ms	240 ms	5,000+
rAF batching	21 ms	45 ms	5,000+
Batching + virtualization	16 ms	16 ms	~40

The combination of batching and virtualization brought the interface from 12 FPS to a solid 60 FPS, and typing during generation now feels the same as typing at rest.

The Accessibility Angle: Live Regions Need Throttling

Batching fixes the renderer, but it creates a new problem for screen reader users. If every token triggers a live region update, NVDA or VoiceOver will try to announce a stream of word fragments, which is useless and exhausting.

The fix is to throttle live region updates to a fixed interval, separate from the render batching:

// Throttled live region: announce progress, not tokens
const liveRegion = document.getElementById("chat-status");
let lastAnnouncement = 0;

function announceProgress() {
  const now = performance.now();
  if (now - lastAnnouncement < 500) return;
  liveRegion.textContent = `Generated ${tokenCount} characters so far`;
  lastAnnouncement = now;
}

This gives screen reader users a progress update every 500 milliseconds instead of a token-by-token narration, which is both informative and tolerable. The final completion announcement still fires exactly once, when the stream ends.

Limitations and Who Should Skip This

The rAF batching strategy assumes the model generates tokens at a steady pace. If your model emits in bursts—50 tokens at once, then a 2-second pause—the batching may add artificial latency between bursts, and a timer-based flush (every 100 milliseconds) would feel snappier.

Virtualization is overkill for short conversations. If your chat interface rarely exceeds a few hundred tokens, batching alone will get you to 60 FPS, and the complexity of virtual scrolling is not worth it. It also adds complexity around focus management and text selection, which can break keyboard navigation if implemented carelessly.

This post is about rendering performance, not network recovery. If your stream drops mid-sentence, no amount of batching will save you; you need a resume protocol, which is a separate problem entirely.

The Frame Budget Is the Contract

A streaming chat interface makes a promise to the user: you can read the response as it arrives, and you can keep typing while it does. That promise is broken the moment the main thread saturates, and the fix is not a faster model or a bigger server—it is a frame budget on the client.

The next time you build a streaming interface, measure the frame time before you measure the network latency. The model will always be faster than the renderer eventually, and the renderer is the side you can actually control.

If you want to reproduce this experiment, point any streaming chat widget at MonkeyCode's free model endpoint and watch the performance panel while a long response streams in. The free server and free model access make the test cost nothing, and the frame meter will tell you exactly where your renderer stands.