Your AI Chat Feels Slow Because You Are Waiting for the Whole Answer: Streaming in Next.js

You build an AI feature. The model takes eight seconds to write the full answer. You show a spinner for eight seconds, then the whole answer appears at once.

Technically it works. Your users think it is broken.

Open any chat product you like and watch what it does instead: words start appearing almost immediately, and the answer grows as you read it. The model is not faster there. The waiting is just hidden, because the first words show up in a second or less.

That is streaming, and it is the cheapest way to make an AI feature feel fast. It takes about thirty lines. It also has a handful of traps that only show up in production.

The Idea

A language model produces text a piece at a time. Most AI APIs let you receive those pieces as they are generated instead of waiting for the end. Your job is to pass them along to the browser without holding them back.

model  ->  your route handler  ->  browser
       pieces              pieces

Two places need to change: the server must send a stream, and the client must read one.

The Server: A Route Handler That Streams

// app/api/chat/route.ts
export async function POST(req: Request) {
  const { prompt } = await req.json();

  // 1. Do auth, validation and limits HERE, before streaming starts.

  const encoder = new TextEncoder();

  const stream = new ReadableStream({
    async start(controller) {
      try {
        for await (const chunk of model.stream({ prompt, signal: req.signal })) {
          controller.enqueue(encoder.encode(chunk.text));
        }
        controller.close();
      } catch (err) {
        controller.error(err);
      }
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
      'Cache-Control': 'no-cache, no-transform',
    },
  });
}

(model.stream is a placeholder for your provider's SDK, which usually gives you an async iterable of chunks. The shape is the same for all of them.)

What is happening:

• A ReadableStream is a pipe you fill over time. controller.enqueue pushes a piece in.
• Streams carry bytes, so the text is encoded with TextEncoder.
• Returning new Response(stream) sends the first bytes to the browser as soon as you enqueue them, without waiting for close().
• no-transform asks proxies and CDNs not to reshape the response, which matters in the buffering trap below.

The Client: Read the Stream

async function ask(
  prompt: string,
  onText: (text: string) => void,
  signal?: AbortSignal
) {
  const res = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
    signal,
  });

  if (!res.ok || !res.body) throw new Error('Request failed');

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let text = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    text += decoder.decode(value, { stream: true });
    onText(text);
  }

  text += decoder.decode(); // flush anything left
  onText(text);
}

And in a component:

'use client';
import { useRef, useState } from 'react';

export function Chat() {
  const [answer, setAnswer] = useState('');
  const abortRef = useRef<AbortController | null>(null);

  async function onSubmit(prompt: string) {
    abortRef.current?.abort();
    abortRef.current = new AbortController();
    setAnswer('');

    try {
      await ask(prompt, setAnswer, abortRef.current.signal);
    } catch (err) {
      if ((err as Error).name !== 'AbortError') setAnswer('Something went wrong. Try again.');
    }
  }

  return (
    <>
      {/* input omitted */}
      <button onClick={() => abortRef.current?.abort()}>Stop</button>
      <p>{answer}</p>
    </>
  );
}

That is the whole feature. Now the traps.

Trap 1: decoder.decode(value) Without { stream: true }

A single character can be several bytes, and a chunk boundary can land in the middle of one. Emoji and many non-English letters are the usual victims. Decode each chunk on its own and you get broken characters, the � symbol, in the middle of a sentence.

{ stream: true } tells the decoder that more bytes are coming, so it holds an incomplete character until the rest arrives. The final decoder.decode() with no arguments flushes anything left. Skip either one and it passes every English-only test you write.

Trap 2: Errors After the Response Has Started

Once the first bytes are sent, the HTTP status is already 200. You can no longer change it to 500 if the model fails halfway.

What happens instead: the stream errors, and reader.read() rejects on the client. If you did not wrap the loop in a try/catch, the user sees half an answer and nothing else.

So split your errors in two:

• Before streaming (bad input, not logged in, rate limit hit): return a normal Response.json({ error }, { status: 4xx }). The client checks res.ok.
• During streaming: catch the rejection on the client, keep what arrived, and show a clear "answer interrupted, try again" message.

This is also why the comment in the route handler says to do auth and limits before the stream. Cost guardrails belong at the start, not inside the loop. I covered those in another post on AI cost limits.

Trap 3: Closing the Tab Does Not Stop the Bill

If the user clicks Stop or closes the tab, the browser stops reading. But your server may keep asking the model for the rest of the answer, and you may keep paying for it.

The fix is in the code above: pass req.signal to your model call. It is aborted when the client disconnects, and most SDKs accept a signal to cancel the upstream request. On the client, the AbortController is what lets the Stop button work at all.

Behavior can differ between hosting platforms, so test it: start a long answer, click Stop, and check your provider's usage to see whether the generation really stopped.

Trap 4: It Streams Locally and Arrives All at Once in Production

This is the most confusing one. On localhost the words flow. Deployed, the whole answer lands in one lump after eight seconds.

Something between the server and the browser is buffering: collecting the response before passing it on. Common causes are a reverse proxy, a compression layer or a CDN that waits for the full body.

What to check:

• The Cache-Control: no-cache, no-transform header, as in the example.
• Your proxy or CDN settings for response buffering and compression on that route.
• Your hosting provider's docs on streaming responses, because support and limits vary.

The lesson: test streaming on the real deployment, not only on your machine.

Trap 5: Function Time Limits

A long answer means a long request. Serverless platforms cap how long a function may run, and the default can be shorter than a slow answer.

In Next.js you can set a maximum duration on the route:

export const maxDuration = 60; // seconds, check your plan's limit

What number is allowed depends on your hosting plan, so check it. A better habit is to also cap the model's output length, so an answer cannot run forever.

A Small Polish: Do Not Re-render on Every Single Chunk

Updating React state on every tiny chunk is fine for a short answer. For long ones, or when you render Markdown, it can make the page stutter, and half-finished Markdown can flicker: an unclosed ** or code block changes how everything above it looks.

Two easy improvements: update the state at most once per animation frame, and render plain text while streaming, then switch to formatted Markdown when the stream ends.

Quick Checklist

• Auth and limits run before the stream starts
• Pre-stream errors return real status codes
• Client has a try/catch around the read loop
• decode(value, { stream: true }) plus a final decode()
• req.signal passed to the model, and a Stop button on the client
• Tested on the real deployment, not just localhost
• maxDuration and an output cap set

The Rule

The first words matter more than the last. Stream the answer so users see progress in a second, then handle the parts that streaming makes harder: errors after the status is sent, cancelling work you are paying for, and anything in the path that buffers.

I build SaaS products with Next.js, and AI features like this one sit in the same route handlers as authentication and billing. If you want to see the kind of work I do, or a typed starting point for your own project, you can browse my templates on pixelanas.com.

Get the templates: https://pixelanas.gumroad.com

Does your AI feature stream yet? If you have hit a buffering problem in production, tell me what the culprit was in the comments.

---

Anas, full-stack Next.js developer building SaaS products and premium templates. X: @ASheikh69751