Streaming AI Responses in Next.js: SSE, Fetch Streams, and What Breaks in Production

Headline: Streaming an AI response in the Next.js App Router takes one Route Handler that returns a streaming Response and one client hook — the AI SDK's streamText and useChat handle the wire protocol. The hard parts live around that code: proxies that buffer the stream into a single blob, compression that swallows token chunks, refreshes that kill in-flight generations, and abort wiring that decides whether you keep paying for tokens nobody is reading.

Key takeaways

• Streaming in the Next.js App Router is a Route Handler returning a streamed Response. The AI SDK's streamText plus toUIMessageStreamResponse() emit an SSE-based message stream, and the useChat hook from @ai-sdk/react consumes it.
• Chat UIs do not use the browser EventSource API. EventSource only supports GET requests with no body, so AI chat clients POST with fetch and read the SSE-formatted response through a ReadableStream reader.
• A stream that works locally but arrives all at once in production is almost always intermediary buffering — compression middleware, nginx proxy_buffering, or a corporate proxy. Cache-Control: no-transform and X-Accel-Buffering: no fix most cases.
• Passing the request's AbortSignal into streamText means a closed tab cancels the upstream model call and stops token spend immediately.
• A page refresh kills the default fetch stream. Persist finished messages in onFinish on the server; reach for resumable streams only when mid-generation continuity is a real product requirement.

Every AI fuature I have shipped in the past year — chat assistants, summarizers, report generators — streams its output token by token. Users will sit through a long generation when text appears immediately; they abandon a silent spinner much sooner. The streaming code itself has become short. The failure modes around it are what fill my notes.

What does it take to stream an AI response in Next.js?

One Route Handler and one hook. On the server, streamText from the AI SDK (Vercel's TypeScript toolkit for calling language models) starts the model call and returns immediately; toUIMessageStreamResponse() converts the result into a streaming Response that speaks the SDK's SSE-based UI message stream protocol. On the client, useChat from @ai-sdk/react POSTs the conversation, parses the stream, and re-renders as parts arrive.

// app/api/chat/route.ts
import { streamText, convertToModelMessages, type UIMessage } from "ai";

export const maxDuration = 300; // seconds — cap for long generations

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-4-6", // AI Gateway provider/model string
    messages: convertToModelMessages(messages),
    abortSignal: req.signal, // closed tab → cancel the model call
  });

  return result.toUIMessageStreamResponse();
}

"use client";
import { useChat } from "@ai-sdk/react";

export function Chat() {
  const { messages, sendMessage, stop, status } = useChat();
  // render messages; sendMessage({ text }) on submit; stop() aborts the stream
}

Two details matter on serverless. The function has to live as long as the generation, so I export a maxDuration from the route — on Vercel, Fluid Compute streams responses and the default execution cap is 300 seconds. And I pass the model as an AI Gateway provider/model string, which keeps switching providers a one-line change instead of a dependency swap.

Should I use SSE, WebSockets, or plain fetch streaming?

For streaming tokens from server to client, SSE format read over fetch is the right default, and it is what the AI SDK implements. Server-Sent Events (SSE) is a plain-HTTP wire format where the server writes data: lines down one long-lived response. WebSockets earn their place only when traffic is genuinely bidirectional — voice conversations, multiplayer cursors, collaborative editing. For a chat box they add connection state and infrastructure without buying anything.

The subtle point that took me a while to internalize: the browser's built-in EventSource API is not what chat apps use. EventSource can only issue GET requests and cannot attach a request body or custom headers, while a chat request needs to POST the conversation history. So the AI SDK sends a normal fetch POST and parses the SSE format off the response body with a ReadableStream reader. You get the SSE wire format without the EventSource limitations — at the cost of its automatic reconnection, which is exactly the gap resumable streams fill later.

Concern	fetch + SSE format	EventSource	WebSockets
Direction	Server → client	Server → client	Bidirectional
Request	Any method, body, headers	GET only, no body	Full duplex after upgrade
Reconnection	Manual (or resumable streams)	Automatic with Last-Event-ID	Manual
Serverless fit	Good	Good	Poor — needs a persistent connection
Best for	AI chat, token streams	Tickers, notifications	Voice, multiplayer, collab editing

Why does my stream work locally but arrive all at once in production?

Because something between your function and the browser is buffering. Compression middleware is the most common culprit: gzip and brotli want complete chunks to compress, so small token deltas sit in the compressor's buffer until the response ends and the stream arrives as one block. Reverse proxies are second: nginx's proxy_buffering collects the entire upstream response by default. Corporate proxies and some antivirus products do the same, and those are entirely outside your control.

// A raw streaming Response with buffering-hostile headers
return new Response(stream, {
  headers: {
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache, no-transform",
    "X-Accel-Buffering": "no", // tells nginx: do not buffer this response
  },
});

On Vercel, streaming responses pass through unbuffered out of the box, so I have only ever debugged this on self-hosted deployments — the classic broken case is Next.js behind an nginx config nobody has touched in a year. If you proxy through nginx, either send X-Accel-Buffering: no or disable proxy_buffering for the route, and exclude text/event-stream responses from any compression layer.

How do I stop paying for tokens nobody is reading?

Wire aborts end to end. The incoming Request carries an AbortSignal that fires when the user closes the tab, navigates away, or clicks stop (the useChat hook exposes a stop() that aborts the fetch). Passing it as streamText's abortSignal option propagates the cancellation to the model provider, which stops generating — and stops billing.

const result = streamText({
  model: "anthropic/claude-sonnet-4-6",
  messages: convertToModelMessages(messages),
  abortSignal: req.signal, // abort upstream on disconnect — saves tokens
});

// The opposite policy: finish the generation even if the client leaves,
// so onFinish can persist the complete message.
// result.consumeStream();

return result.toUIMessageStreamResponse({
  onFinish: async ({ messages }) => {
    await saveChat({ chatId, messages }); // completed messages survive refresh
  },
});

There is a genuine trade-off in that snippet. consumeStream() does the opposite of the abort signal: it detaches the generation from the client connection, so the model runs to completion and onFinish can persist the full message even after a disconnect. Abort-on-disconnect saves tokens but loses the partial response; consume-to-completion keeps the response but pays for every token. I wire the abort signal for cheap conversational chat and consume-plus-persist for expensive long-form generations. The mistake is choosing one policy globally instead of per feature.

What happens when the user refreshes mid-stream?

With the default setup, the fetch stream dies with the page, and if the abort signal is wired, the server-side generation dies with it. After the reload, the chat shows whatever onFinish had persisted: completed messages survive, the in-flight one vanishes. For most chat products I consider that acceptable — and more honest than pretending the message still exists somewhere.

When it is not acceptable — long report generations a user kicks off and checks back on — resumable streams close the gap. Chunks are published to a store such as Redis as they arrive, and a client that reconnects re-subscribes from where it left off, so the generation survives refreshes and network drops. The AI SDK supports this pattern, but it adds real infrastructure and operational state. My rule: persist finished messages always; add resumability only when a product requirement names it explicitly.

FAQ

Q: Do I need WebSockets to stream ChatGPT-style responses?

A: No. Token streaming is one-way, server to client, and SSE format over a fetch POST handles it on plain HTTP. WebSockets are for bidirectional traffic such as voice or collaborative editing.

Q: Why does my AI stream not work behind nginx?

A: nginx buffers upstream responses by default. Send X-Accel-Buffering: no on the response or disable proxy_buffering for the route, and make sure no compression layer re-buffers the stream.

Q: Does streaming work on serverless platforms like Vercel?

A: Yes. Vercel Functions stream responses, and Fluid Compute keeps the function alive for the whole generation — the default execution cap is 300 seconds, configurable per route with maxDuration.

Q: How do I save the AI message if the user closes the tab mid-stream?

A: Call consumeStream() on the server so the generation runs to completion and onFinish persists the full message. This conflicts with aborting to save tokens — each route picks one behavior.

Q: Can EventSource send a POST body?

A: No. EventSource issues GET requests only, with no body or custom headers, which is why AI chat clients read the SSE format from a fetch POST via a ReadableStream reader instead.

---

Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.