Your AI Feature Isn't Slow. Your Architecture Is Making It Feel Slow.

You've seen it a hundred times. A user types a question into an AI feature, hits submit, and then just stares at a blank screen for eight seconds before a wall of text dumps onto the page all at once. That gap between "I asked something" and "something happened" is exactly where users lose patience, and it's rarely the model's fault. It's almost always a streaming problem, not a speed problem. This is one of the core patterns behind modern AI services and AI web application development, and getting it right in Next.js is simpler than most developers expect, especially now that the framework's App Router and Server Actions map almost perfectly onto how real-time AI responses actually need to flow.

Here's how to build a genuinely responsive, streaming AI feature in Next.js from the ground up, understanding what's actually happening under the hood instead of just wiring up a library and hoping.

Why Streaming Changed What "Fast" Means for AI Features

A non-streaming AI feature waits for the entire response to generate before showing anything. For short answers that's tolerable. For anything longer, a paragraph, a summary, a generated document, it means staring at a static loading spinner for several seconds with zero feedback that anything is actually happening.

Streaming flips this. Tokens appear as the model generates them, the same way ChatGPT-style interfaces work. The perceived speed difference is enormous even when the total generation time is identical, because the user sees progress immediately instead of waiting in silence.

The Real-World Difference This Makes

Picture a support tool generating a written summary of a long customer thread. The generation itself takes about six seconds either way.

Without streaming: the user submits, sees a static spinner, and waits the full six seconds staring at nothing. To someone who isn't sure the request even registered, six seconds of silence feels much longer than it actually is, and a meaningful number of users will refresh or click again out of doubt, sometimes triggering a duplicate request.

With streaming: the user sees the first words appear within a few hundred milliseconds, and text keeps flowing steadily until the summary is complete. The total time is identical, but it reads as fast because there's constant, visible progress instead of dead air.

Same backend, same model, same six seconds of actual compute. The only difference is architecture, and it's the difference between a feature that feels broken and one that feels instant.

Choosing Your Transport Mechanism

Before writing code, it's worth understanding the three real options for getting streaming data from server to client, since picking the wrong one for your use case creates real problems later.

Server-Sent Events (SSE)

• One-directional, server to client only
• Works over standard HTTP, no special infrastructure required
• The natural fit for AI text generation, since the client only needs to receive tokens, not send data mid-stream

WebSockets

• Bidirectional, full duplex communication
• Better suited for genuinely interactive real-time features like collaborative editing or live multiplayer state
• Overkill for most AI chat features, and adds real infrastructure complexity you don't need for one-way token streaming

Web Streams API (ReadableStream)

• The lowest-level option, giving direct control over how chunks are read and processed
• What the AI SDK and most Route Handler implementations are actually built on under the hood
• Best when you need fine-grained control over parsing structured data mid-stream, like tool calls or JSON fragments

For the vast majority of AI chat and generation features, SSE built on top of the Web Streams API is the right choice. Reach for WebSockets only when you genuinely need bidirectional communication during generation, which is rarer than most implementations assume.

Step-by-Step: Building a Streaming AI Route Handler

Step one: set up the project

npx create-next-app@latest ai-streaming-app
cd ai-streaming-app

Select the App Router when prompted. This pattern depends on Route Handlers, which live specifically in the App Router architecture.

Step two: create the streaming Route Handler

// app/api/chat/route.ts
export async function POST(req: Request) {
  const { message } = await req.json();

  const stream = new ReadableStream({
    async start(controller) {
      const response = await callModelProvider(message);

      for await (const chunk of response) {
        controller.enqueue(new TextEncoder().encode(chunk));
      }
      controller.close();
    },
  });

  return new Response(stream, {
    headers: { "Content-Type": "text/event-stream" },
  });
}

The key detail here: the API key and the call to the model provider stay entirely server-side. Nothing sensitive ever ships to the client bundle, which matters both for security and for keeping your provider costs from being exposed.

Step three: consume the stream on the client

// app/components/ChatInput.tsx
'use client';
import { useState } from 'react';

export default function ChatInput() {
  const [response, setResponse] = useState('');

  async function handleSubmit(message: string) {
    setResponse('');
    const res = await fetch('/api/chat', {
      method: 'POST',
      body: JSON.stringify({ message }),
    });

    const reader = res.body?.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader!.read();
      if (done) break;
      setResponse((prev) => prev + decoder.decode(value));
    }
  }

  return <div>{response}</div>;
}

This is the piece that actually creates the typing effect. Each chunk arrives and appends to state as it comes in, instead of waiting for one giant response.

Step four: add proper error and cancellation handling

const controller = new AbortController();

async function handleSubmit(message: string) {
  try {
    const res = await fetch('/api/chat', {
      method: 'POST',
      body: JSON.stringify({ message }),
      signal: controller.signal,
    });
    // stream reading logic here
  } catch (err) {
    if (err.name === 'AbortError') {
      console.log('Generation cancelled by user');
    } else {
      setError('Something went wrong, please try again');
    }
  }
}

function handleCancel() {
  controller.abort();
}

A stop button that actually cancels an in-flight generation is a small detail users notice immediately when it's missing, and it costs very little to implement properly.

Step five: validate structured output when you need more than plain text

import { z } from 'zod';

const responseSchema = z.object({
  summary: z.string(),
  confidence: z.number(),
});

function validateResponse(raw: string) {
  const parsed = JSON.parse(raw);
  return responseSchema.parse(parsed);
}

Instead of hoping a model returns well-formed JSON, defining a schema and validating against it catches malformed output before it ever reaches your UI, which matters a lot once an AI feature is doing more than just displaying conversational text.

Step six: add rate limiting before this ever reaches production

// app/api/chat/route.ts
const requestCounts = new Map<string, number>();

export async function POST(req: Request) {
  const userId = getUserIdFromSession(req);
  const currentCount = requestCounts.get(userId) ?? 0;

  if (currentCount >= 20) {
    return new Response("Rate limit exceeded", { status: 429 });
  }

  requestCounts.set(userId, currentCount + 1);
  // proceed with streaming logic
}

This is a simplified example, a real implementation would use a proper store like Redis rather than an in-memory map, but the principle holds regardless of scale. Without rate limiting, a single user or a scripted abuse pattern can run your model provider costs up fast, and this is one of the most commonly skipped steps in early AI feature builds.

Step seven: track cost per feature, not just in aggregate

function logUsage(feature: string, tokenCount: number, model: string) {
  const cost = tokenCount * getCostPerToken(model);
  analytics.track("ai_usage", { feature, tokenCount, cost, model });
}

Breaking cost down by feature rather than watching one aggregate monthly number lets you catch a specific feature quietly ballooning in usage long before it shows up as a surprise on the bill.

Common Mistakes Worth Avoiding

• Forgetting to handle stream errors mid-response, leaving the UI stuck in a permanent loading state if the connection drops partway through
• Skipping the cancellation handling entirely, which leaves users stuck watching a response they no longer want finish generating
• Calling the model provider directly from client-side code, which exposes API keys and lets costs run up uncontrolled
• Not validating structured output, and discovering malformed data only when it breaks something downstream in production
• Shipping without rate limiting and finding out the hard way when a single user or bot script drives up provider costs overnight
• Choosing WebSockets by default out of habit when SSE would be simpler, cheaper to run, and perfectly sufficient for one-way token streaming

A Quick Self-Check Before You Ship

• Does your UI show visible progress within the first second, or does it sit on a static spinner while waiting for the full response?
• Can a user actually cancel a generation mid-stream, or does clicking away just leave the request running in the background?
• Is there a rate limit in place, or is the endpoint currently wide open to unlimited requests per user?
• Are you tracking cost per feature, or would a sudden spike in usage only show up once the monthly bill arrives?

Why This Is Worth Building Properly, Not Just Quickly

Getting a demo chat working in an afternoon is genuinely easy now. Getting it production-ready, proper error handling, rate limiting, cost monitoring, structured output validation, and a UI that gracefully degrades when something goes wrong, is where most teams underestimate the actual scope of the work. Custom AI development and API integration work focused specifically on shipping these features at production scale tends to close that gap far faster than assembling it piece by piece under a deadline, especially for teams integrating multiple providers or building on top of an existing enterprise codebase.

The Takeaway

The gap between an AI feature that feels instant and one that feels sluggish usually isn't the model's raw speed, it's whether the response streams token by token or dumps all at once after a long, silent wait. Next.js's architecture, particularly Server Actions and Route Handlers, maps naturally onto this pattern, which is exactly why it's become the default framework for AI-powered web applications this year.

Has your team already shipped a streaming AI feature in production, or are you still working through the plumbing on this one? Curious what's tripped people up most.