Fix Silent 30s AI Stream Timeouts with a Client Watchdog

Your AI stream can die silently at exactly 30 seconds because the proxy's read timeout kills idle connections—not because the model stopped. I traced this while testing a chat widget against MonkeyCode's free server, and the fix is a client-side watchdog that turns a lying spinner into an honest state machine.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The product wasn't misbehaving; the architecture around it was. The same silent death awaits anyone who streams AI responses through a proxy that has opinions about time.

Why AI streams die silently at 30 seconds

The stream died at exactly 30.0 seconds. Not 29.4, not 31.2—30.0, like a metronome. The chat widget sat there with its spinner spinning, the last token rendered halfway through a sentence, and no error, no close event, no status change. Just silence. I waited, then refreshed. The response was gone, and so was whatever the server had been generating.

Before changing anything, I instrumented the stream with timestamps on every chunk:

Time	Event
0.0s	Request sent, first token arrives
0.0–29.8s	Tokens arrive steadily
29.8s	Last chunk rendered
60.0s+	Nothing. No error, no close, no timeout

The stream did not die after 30 seconds of total time. It died after 30 seconds of silence. The model had paused—likely thinking, or waiting on a tool boundary—and the proxy treated that pause as a dead connection. It was not measuring activity; it was measuring the gap between reads.

This is classic proxy_read_timeout behavior in reverse proxies like Nginx. The timer resets on every read from the upstream. When the upstream goes quiet for 30 seconds, the proxy closes the socket without sending a response. The client's fetch stream just ends: no status code, no error body, no close event with a reason. The promise chain waits for a done that never comes. From the UI's perspective, the model is still thinking.

How I isolated the timeout layer by layer

I ran the classic debugging ladder, one layer at a time, so I would not "fix" the wrong thing. A stall that only happens in Chrome is a fetch quirk. A stall that also happens in curl is infrastructure.

1. Reproduce outside the browser. Same prompt, same endpoint, no UI.
2. Hit the provider directly, bypassing the proxy. If generation continues past 30 seconds, the model is not the problem.
3. Build a minimal local proxy that pauses. If a 35-second sleep between two SSE frames dies only when routed through the gateway, you have found the timeout.

curl -N https://free-server.example/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "Write a very long story..."}'

Same result: tokens flow, then stop at 30.0s. Not a browser issue. The same prompt through the upstream API kept generating past 30 seconds without a pause. The provider was not the problem.

Then I simulated a thinking pause:

app.post('/api/chat', async (req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/event-stream' });
  res.write('data: first\n\n');
  await new Promise(r => setTimeout(r, 35000)); // simulate a model thinking pause
  res.write('data: second\n\n');
  res.end();
});

When I ran this through the free server's gateway, the client never received second. Locally, it worked. The gateway was killing connections that went silent for 30 seconds.

Root cause: the gateway enforces a 30-second proxy_read_timeout. That timer measures the gap between consecutive reads from upstream. Bytes flowing reset it. A quiet stretch—a reasoning pause or a tool call waiting on an API—expires it, and the gateway closes the socket.

Add a client-side watchdog to the streaming loop

You often cannot change the gateway timeout on a free server, so the fix has to live on the client. I added a watchdog to the streaming loop. If no chunk arrives within a threshold, the UI moves to a visible stalled state. If silence lasts past the known timeout, it offers recovery.

function createWatchdog(onStall, onTimeout, stallMs = 15000, timeoutMs = 30000) {
  let timer;
  const reset = () => {
    clearTimeout(timer);
    timer = setTimeout(() => {
      onStall();
      timer = setTimeout(onTimeout, timeoutMs - stallMs);
    }, stallMs);
  };
  const clear = () => clearTimeout(timer);
  return { reset, clear };
}

Reset the watchdog on every chunk:

const watchdog = createWatchdog(
  () => setState({ status: 'stalled' }),
  () => setState({ status: 'timed_out' })
);

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  watchdog.reset();
  buffer += decoder.decode(value, { stream: true });
  onToken(buffer);
}

Honest states instead of a perpetual spinner:

type ChatState =
  | { status: 'idle' }
  | { status: 'streaming' }
  | { status: 'stalled'; received: number }
  | { status: 'timed_out'; received: number }
  | { status: 'recovering'; attempt: number }
  | { status: 'error'; message: string };

ReadableStream gives you no signal when the socket is killed silently—MDN notes that read() just never resolves. The watchdog is how you detect the gap.

Compare the two UX paths:

• Spinner-only: the user waits, assumes the model is thinking, then refreshes and loses the partial answer.
• Watchdog states: the user sees stalled after 15 seconds of silence, then timed_out at 30 seconds, with a choice to continue or restart.

Recover the partial answer and announce the stall

A model cannot resume a dead stream—it is stateless. You can send the partial transcript and ask it to continue. That is reconstruction, not resume, but it works surprisingly well:

async function continueFrom(buffer) {
  const continuationPrompt = `
The previous response was interrupted mid-sentence.
Here is exactly what was generated so far:

${buffer}

Continue from the exact point where it stopped.
Do not repeat any of the text above. Do not summarize it.
`;

  return stream(continuationPrompt);
}

Offer an explicit choice: "The connection stalled after 340 tokens. [Continue from here] [Restart]". Let the user decide, rather than silently reconnecting and duplicating tokens.

A silent stall is worse for screen reader users: they hear nothing, the accessibility tree does not change, and they reasonably assume the response is complete. Announce state changes through a live region:

function setState(next) {
  state = next;
  if (next.status === 'stalled') {
    statusRegion.textContent = 'The connection stalled. Waiting for more data.';
  } else if (next.status === 'timed_out') {
    statusRegion.textContent = 'The connection timed out. You can continue from the last received text or restart.';
  }
}

Recovery buttons need keyboard focus. Escape should cancel a stalled stream just like an active one: pointer-independent, announced, and honest about the in-between state.

Skip or simplify this pattern when:

• You need the original ending, not a reconstructed continuation.
• Your stall threshold is so low that "model thinking" looks dead. Fifteen seconds worked for me.
• The provider bills for the abandoned generation and the continuation.
• The proxy already supports SSE comment-frame heartbeats—prefer those.
• The provider has a resumable stream API or generation ID. Use that instead of a prompt hack.
• A simple "Try again" button is enough. Watchdogs are for long generations where restarting hurts.

The 30-second curse taught me to instrument first, isolate layer by layer, and never trust a spinner. Free tiers surface these failures because their gateways protect cost, not long generative sessions. MonkeyCode's free server is a fine place to reproduce this: you can afford to break things and learn the failure modes before they hit production.

Try it now: spin up MonkeyCode's free server, reproduce the 30-second stall, and add the watchdog to your streaming client. Your users deserve an honest spinner.

MonkeyCode provides free models that can run this workflow.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.