How to Build a RAG App in Next.js (2026 Guide) — Claude, Pinecone, Voyage AI

If you've tried to build an AI app that answers questions about your own content — docs, PDFs, a knowledge base — you've probably run into RAG (retrieval-augmented generation). It's the pattern behind almost every "chat with your documents" product, and it's simpler than it looks once you break it into pieces.

This guide walks through building one in Next.js: scraping content, embedding it, storing it in a vector database, and streaming answers back with citations. By the end you'll have a working doc-chat pipeline you can extend into your own product.

The four pieces of any RAG app

Every RAG system, no matter how fancy, is four steps wearing a trench coat:

1. Ingestion — get your content (a webpage, a PDF, a doc) into plain text
2. Embedding — turn chunks of that text into vectors (numbers that capture meaning)
3. Storage & retrieval — put those vectors in a database you can search by similarity
4. Generation — hand the most relevant chunks to an LLM along with the user's question, and stream back an answer

That's it. Everything else — chunking strategy, reranking, multi-tenancy — is refinement on top of these four steps. Let's build each one.

Step 1: Ingestion without a headless browser

If you've scraped web pages in a Next.js serverless function before, you've probably hit the Puppeteer/Playwright wall: huge bundle sizes, slow cold starts, and occasional ESM import conflicts on Vercel. For most content — docs sites, blog posts, articles — you don't need a full browser at all. A DOM parser like Cheerio is enough, because you're not interacting with the page, just reading it.

import * as cheerio from 'cheerio';

async function scrapeUrl(url) {
  const res = await fetch(url, {
    signal: AbortSignal.timeout(15000), // don't hang forever
  });
  const html = await res.text();
  const $ = cheerio.load(html);

  // Strip the noise before extracting text
  $('script, style, nav, footer, header, aside').remove();

  // Prefer semantic content zones, fall back to body
  const main = $('main').text() || $('article').text() || $('body').text();

  return main.replace(/\s+/g, ' ').trim();
}

This runs cleanly inside a standard serverless function — no special runtime config needed. It won't work on JS-rendered SPAs (you'd need a real browser for those), but for the majority of docs and content sites, it's faster and lighter.

Step 2: Chunking and embedding

Raw text is too long to embed as one block — you need to split it into chunks small enough to be semantically coherent but large enough to carry context.

import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';

const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 200,
});

const chunks = await splitter.splitText(scrapedText);

The overlap matters more than people expect — without it, a sentence that spans a chunk boundary gets cut in half and loses meaning in both halves.

Once you have chunks, you embed them — turn each one into a vector using an embedding model. Here's an underexplained detail worth knowing: most production RAG systems use two different embedding calls, one for documents and one for queries. This is called asymmetric retrieval. A short question ("how do I reset my password?") and a long passage that answers it aren't symmetric in structure, so some embedding models (Voyage AI's among them) let you specify an input_type to optimize the vector space for each side of that mismatch.

import { VoyageEmbeddings } from '@langchain/community/embeddings/voyage';

const documentEmbedder = new VoyageEmbeddings({
  apiKey: process.env.VOYAGEAI_API_KEY,
  modelName: 'voyage-3.5',
  inputType: 'document',
});

const queryEmbedder = new VoyageEmbeddings({
  apiKey: process.env.VOYAGEAI_API_KEY,
  modelName: 'voyage-3.5',
  inputType: 'query',
});

const vectors = await documentEmbedder.embedDocuments(chunks);

Step 3: Storing and retrieving with a vector database

Once you have vectors, you need somewhere to store and search them. Pinecone's serverless tier is a common choice for this because it needs no infrastructure management.

A detail that matters once you have more than one user: namespace-based multi-tenancy. Instead of provisioning a separate index per user or organization, you scope every upsert and query to a namespace string within a single index.

async function upsertVectors(vectors, chunks, namespace) {
  const index = pinecone.index(process.env.PINECONE_INDEX);

  const records = vectors.map((values, i) => ({
    id: `${namespace}-${i}-${Date.now()}`,
    values,
    metadata: { text: chunks[i] },
  }));

  await index.namespace(namespace).upsert(records);
}

async function querySimilar(queryVector, namespace, topK = 5) {
  const index = pinecone.index(process.env.PINECONE_INDEX);
  const results = await index.namespace(namespace).query({
    vector: queryVector,
    topK,
    includeMetadata: true,
  });

  // Filter out weak matches — don't force irrelevant context into the prompt
  return results.matches.filter(m => m.score >= 0.3);
}

One index, unlimited namespaces, and zero cross-contamination between tenants — swap namespace for a userId or orgId and you have multi-tenant isolation without any schema migration.

Step 4: Streaming the answer

This is where retrieval meets generation. You embed the user's question, retrieve the most relevant chunks, inject them into the system prompt, and stream the response back token by token.

export default async function handler(req, res) {
  const { query, namespace } = req.body;

  const queryVector = await queryEmbedder.embedQuery(query);
  const matches = await querySimilar(queryVector, namespace);

  const context = matches.map(m => m.metadata.text).join('\n\n');

  res.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    Connection: 'keep-alive',
  });

  const stream = anthropic.messages.stream({
    model: 'claude-haiku-4-5-20251001',
    max_tokens: 1024,
    system: `Answer using only the following context. If the context doesn't contain the answer, say so — don't make one up.\n\n<context>\n${context}\n</context>`,
    messages: [{ role: 'user', content: query }],
  });

  // Send sources before any text, so the UI can render citations immediately
  res.write(`data: ${JSON.stringify({ sources: matches.map(m => m.metadata) })}\n\n`);

  stream.on('text', (text) => {
    res.write(`data: ${JSON.stringify({ text })}\n\n`);
  });

  stream.on('end', () => {
    res.write(`data: ${JSON.stringify({ done: true })}\n\n`);
    res.end();
  });
}

Two things worth calling out here:

• The system prompt explicitly permits "I don't know." Without that instruction, LLMs tend to answer confidently even when the retrieved context doesn't actually support an answer — graceful degradation beats confident fabrication.
• Sending sources before text starts means your frontend can render citation cards immediately, instead of waiting for the full response to know what was cited.

Putting it together

That's a complete, working RAG pipeline: scrape or upload → chunk → embed → store → retrieve → stream. From here, the places to go deeper are chunking strategy (fixed-size vs. semantic chunking), reranking (a second, more expensive relevance pass on your top results), and hybrid search (combining vector similarity with keyword search for queries that need exact matches).

If you'd rather skip the wiring than build it from scratch, I packaged this exact pattern — plus PDF ingestion, demo rate limiting, and multi-tenant namespace isolation — into a Next.js starter kit called FastRAG. There's a live, unauthenticated demo at fastrag.live/demo if you want to see the retrieval quality before deciding either way.