Streaming LLM responses in TypeScript: SSE, ReadableStream, and the React 19 useChat hook.

The first time I wired an LLM response that streamed token by token instead of arriving as one lump after 4 seconds, I shipped it to production the same afternoon. The difference in perceived speed is that obvious to anyone who has used ChatGPT and then tried a non-streaming competitor. Users will wait 8 seconds if they can see the cursor moving. They will not wait 4 seconds for a blank screen.

This tutorial walks the full stack from scratch. By the end you have a working Next.js API route that streams from an LLM over Server-Sent Events, a frontend that parses the stream manually with ReadableStream, and then the same UI rebuilt with the Vercel AI SDK's useChat hook so you can see what the abstraction actually buys you.

TL;DR

Layer	What you build	Key API
Next.js route	Streams LLM output as SSE	streamText + toUIMessageStreamResponse
Vanilla client	Parses the stream by hand	ReadableStream, TextDecoderStream
React 19 client	Managed state + cancellation	useChat from ai/react
Edge cases	Backpressure, cancel, tool chunks	AbortController, partial JSON guard

1. Why streaming matters

A standard fetch returns after the entire response body is ready. For short completions, that is fine. For anything over 100 tokens, users see a spinner, then a wall of text, then confusion about whether the app is fast or slow.

Streaming changes the shape of that experience. The first token lands in under 300ms for most hosted models. The user starts reading while the model is still writing. Perceived latency drops by 60 to 70 percent even if the total time to complete the response does not change.

Cost transparency is the second reason to care. When you stream, you count tokens as they arrive. If your route has a budget ceiling and the response is going to blow past it, you can cut the stream at 800 tokens without ever waiting for the full completion. That cut is not possible with a blocking call.

2. The plumbing in one diagram

Here is how the pieces connect for a streaming chat request:

Browser                     Next.js API Route          LLM Provider
  |                               |                         |
  |-- POST /api/chat ------------>|                         |
  |   { messages: [...] }         |                         |
  |                               |-- streamText() -------->|
  |                               |                         |
  |<-- HTTP 200 (SSE stream) -----|<-- token chunks --------|
  |   Content-Type: text/event-stream                       |
  |                               |                         |
  | chunk: "The "                 |                         |
  | chunk: "quick "               |                         |
  | chunk: "brown "               |                         |
  |   ...                         |                         |
  | [DONE]                        |                         |

The route opens a persistent HTTP response with Content-Type: text/event-stream. Each chunk the LLM returns gets written to that response immediately. The browser receives chunks as they arrive and renders them without waiting for the response to close.

3. The backend: a streaming Next.js route

Install the Vercel AI SDK and the Anthropic provider:

npm install ai @ai-sdk/anthropic

Create app/api/chat/route.ts:

import { streamText, UIMessage, convertToModelMessages } from "ai";
import { anthropic } from "@ai-sdk/anthropic";

export const runtime = "edge";

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: anthropic("claude-sonnet-4-6-20250929"),
    messages: await convertToModelMessages(messages),
    maxTokens: 1024,
    onError: (error) => {
      console.error("[chat] stream error:", error);
    },
  });

  return result.toUIMessageStreamResponse();
}

Three things worth noting here.

runtime = "edge" puts the route on the V8 isolate runtime where streaming responses work without any special Node.js stream adapter. If you need the Node runtime, replace toUIMessageStreamResponse() with pipeTextStreamToResponse(res) and a writable HTTP response object.

The model ID pins to a dated snapshot. The alias claude-sonnet-4 rolls forward whenever Anthropic ships a patch. A pinned ID means your streaming format cannot change under you between deploys.

onError handles stream-level errors. The AI SDK swallows errors into the stream by default to prevent server crashes. If you want an error boundary in the client, emit a special chunk and handle it there rather than relying on an HTTP 500, which the stream transport will not surface cleanly.

What the SSE wire format looks like

Open Network tab on the request and you will see chunks like this:

f:{"messageId":"msg_01ABC..."}
0:"The "
0:"quick "
0:"brown fox"
e:{"finishReason":"stop","usage":{"promptTokens":24,"completionTokens":12}}
d:{"finishReason":"stop","usage":{"promptTokens":24,"completionTokens":12}}

The prefix before the colon is a type code. 0: is a text delta. f: is a message ID annotation. e: and d: are finish events. This is the Vercel AI SDK's data stream protocol, not raw SSE. The raw OpenAI SSE format uses data: {"choices":[...]} lines. The SDK normalizes both.

4. The frontend: parsing the stream manually

Before reaching for useChat, build the vanilla version. Knowing what happens at this layer means you can debug anything the abstraction hides.

// hooks/useManualStream.ts
import { useState, useRef } from "react";

export function useManualStream() {
  const [output, setOutput] = useState("");
  const [status, setStatus] = useState<"idle" | "streaming" | "done" | "error">("idle");
  const abortRef = useRef<AbortController | null>(null);

  async function send(userMessage: string) {
    abortRef.current?.abort();
    const controller = new AbortController();
    abortRef.current = controller;

    setOutput("");
    setStatus("streaming");

    try {
      const res = await fetch("/api/chat", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({
          messages: [{ role: "user", parts: [{ type: "text", text: userMessage }] }],
        }),
        signal: controller.signal,
      });

      if (!res.ok || !res.body) {
        throw new Error(`HTTP ${res.status}`);
      }

      const reader = res.body
        .pipeThrough(new TextDecoderStream())
        .getReader();

      let accumulated = "";

      while (true) {
        const { value, done } = await reader.read();
        if (done) break;

        // Each SSE chunk may contain multiple lines
        for (const line of value.split("\n")) {
          const trimmed = line.trim();
          // Text delta lines start with "0:"
          if (trimmed.startsWith("0:")) {
            try {
              // The value after "0:" is a JSON-encoded string
              const text = JSON.parse(trimmed.slice(2)) as string;
              accumulated += text;
              setOutput(accumulated);
            } catch {
              // Partial JSON guard: skip malformed chunks
            }
          }
        }
      }

      setStatus("done");
    } catch (err) {
      if ((err as Error).name === "AbortError") {
        setStatus("idle");
      } else {
        console.error("[stream] failed:", err);
        setStatus("error");
      }
    }
  }

  function cancel() {
    abortRef.current?.abort();
    setStatus("idle");
  }

  return { output, status, send, cancel };
}

A usage example in a component:

// app/page.tsx (vanilla version)
"use client";
import { useState } from "react";
import { useManualStream } from "@/hooks/useManualStream";

export default function ChatPage() {
  const [input, setInput] = useState("");
  const { output, status, send, cancel } = useManualStream();

  return (
    <main style={{ maxWidth: 680, margin: "0 auto", padding: "2rem" }}>
      <div style={{ minHeight: 200, marginBottom: "1rem", whiteSpace: "pre-wrap" }}>
        {output}
        {status === "streaming" && <span aria-live="polite"> ▋</span>}
      </div>
      <textarea
        value={input}
        onChange={(e) => setInput(e.target.value)}
        rows={3}
        style={{ width: "100%", marginBottom: "0.5rem" }}
      />
      <button onClick={() => send(input)} disabled={status === "streaming"}>
        Send
      </button>
      {status === "streaming" && (
        <button onClick={cancel} style={{ marginLeft: "0.5rem" }}>
          Stop
        </button>
      )}
    </main>
  );
}

No conversation history, no multi-turn context. The point is to see the ReadableStream + TextDecoderStream pipeline in isolation before you add state management.

5. The frontend: the useChat version

The Vercel AI SDK's useChat hook handles message history, status transitions, streaming, and cancellation. Install ai if you have not already:

npm install ai

// app/chat/page.tsx (useChat version)
"use client";
import { useChat } from "ai/react";
import { DefaultChatTransport } from "ai";

export default function ChatPage() {
  const { messages, sendMessage, status, stop } = useChat({
    transport: new DefaultChatTransport({ api: "/api/chat" }),
  });

  function handleSubmit(e: React.FormEvent<HTMLFormElement>) {
    e.preventDefault();
    const form = e.currentTarget;
    const text = new FormData(form).get("message") as string;
    if (!text.trim()) return;
    sendMessage({ text });
    form.reset();
  }

  return (
    <main style={{ maxWidth: 680, margin: "0 auto", padding: "2rem" }}>
      <ul style={{ listStyle: "none", padding: 0, minHeight: 300 }}>
        {messages.map((m) => (
          <li key={m.id} style={{ marginBottom: "1rem" }}>
            <strong>{m.role === "user" ? "You" : "AI"}:</strong>{" "}
            {m.parts
              .filter((p) => p.type === "text")
              .map((p, i) => <span key={i}>{p.type === "text" ? p.text : ""}</span>)}
          </li>
        ))}
      </ul>
      {status === "streaming" && (
        <button onClick={stop} style={{ marginBottom: "0.5rem" }}>
          Stop generating
        </button>
      )}
      <form onSubmit={handleSubmit} style={{ display: "flex", gap: "0.5rem" }}>
        <input name="message" style={{ flex: 1 }} autoComplete="off" />
        <button type="submit" disabled={status === "streaming" || status === "submitted"}>
          Send
        </button>
      </form>
    </main>
  );
}

The hook's status field cycles through "submitted" (request sent, waiting for first token), "streaming" (chunks arriving), "ready" (complete), and "error". That four-state machine replaces the manual tracking in the vanilla version.

messages is an array of UIMessage objects. Each message has a parts array rather than a flat content string. That structure supports tool call results, images, and file attachments alongside text in the same message. For text-only chat, filter parts to type === "text".

The stop() function calls AbortController.abort() on the underlying fetch internally. The stream closes, the model stops generating, and status returns to "ready".

6. Edge cases

Backpressure

streamText uses backpressure by design. The consumer drives token generation: the model produces a chunk only when the reader asks for the next one. In the manual version, the while (true) { reader.read() } loop applies backpressure automatically because the next read does not start until the previous chunk lands. If you pipe to two readers using tee(), the slower reader controls the pace and the faster reader buffers. For most chat UIs one reader is correct.

Cancellation

Both versions above cancel via AbortController. In the manual version, aborting the fetch closes the ReadableStream on the next read() call and throws an AbortError. Catch it and reset state. In the useChat version, stop() handles the same thing internally.

One gotcha: if you cancel on the client, the server-side streamText call continues running until the streaming HTTP response detects the closed connection and propagates the abort. On edge runtime this propagation is fast, under 100ms in practice. On Node runtime with some hosting providers, the server-side model call may run an extra second or two before it halts. Budget for that if you charge per token.

Partial JSON for tool calls

When you add tool calls to streamText, the wire format includes partial JSON chunks as the tool arguments stream in:

b:{"toolCallId":"call_01","toolName":"search","argsTextDelta":"{\"q"}
b:{"toolCallId":"call_01","toolName":"search","argsTextDelta":"uery\":\""}
b:{"toolCallId":"call_01","toolName":"search","argsTextDelta":"typescript\"}"}

The b: prefix marks a tool call delta. Do not try to parse the args until you receive the 9: tool result chunk that confirms the call finished. If you parse on each delta, your JSON parser will throw on every partial chunk except the last one. The useChat hook handles this accumulation internally. In the manual version, accumulate argsTextDelta into a buffer keyed by toolCallId, then parse only when you see the matching tool result.

The bottom line

Streaming is not a polish feature. For any LLM feature that generates more than two sentences, the choice between streaming and blocking decides the entire perceived quality of the interaction.

The manual ReadableStream path takes about 40 lines of TypeScript and gives you full control over chunk handling, cancellation, and partial JSON. The useChat path from the Vercel AI SDK cuts that to a dozen lines and adds multi-turn history, status management, and tool call accumulation for free.

Start with the manual version once so you understand what the wire looks like. Then move to useChat for anything you ship. The abstraction earns its overhead at the point where you need tool calls, file parts, or reconnect logic, and you will not want to rebuild those from scratch.

What is the first LLM feature in your app where you wished the response arrived token by token instead of all at once? Drop a comment with the use case.

---

GDS K S · thegdsks.com · follow on X @thegdsks

Streaming is not about speed. Give users something to read while the model finishes thinking.

Comments

[Mustafa ERBAY]

Streaming absolutely changes perceived performance, and I like that you show both the manual path and the higher-level useChat abstraction.

There are a few version-sensitive details worth revisiting, though. In the current AI SDK, useChat is imported from @ai-sdk/react, and toUIMessageStreamResponse() uses the newer SSE-based UI Message Stream Protocol with structured data: {"type":"text-delta", ...} events. A parser looking only for legacy 0: or b: prefixes would not correctly process that stream.

I’d also be careful with the backpressure claim. A slow ReadableStream consumer can apply pressure within the HTTP pipeline, but that does not necessarily mean the upstream model provider stops generating tokens one-for-one. Likewise, client-visible chunks should not be treated as authoritative token accounting; usage and budget enforcement belong on the server using provider or SDK usage metadata.

One more nuance: Next.js Route Handlers support Web streams in both Node.js and Edge runtimes, so Edge is a deployment choice rather than a requirement for streaming.

The overall teaching approach is strong—especially encouraging developers to inspect the transport once before relying on an abstraction—but pinning the examples to a specific AI SDK version would make the tutorial much safer to copy into production.