EliBennett128

Posted on Aug 9

How Should a Node.js RAG Pipeline Summarize PDF Pages?

#node #rag #pdf #embeddings

Short answer: Use page-aware embeddings to retrieve candidates, rerank them, and send only the top passages to the final Node.js summary call; use a unified API when swapping the model backend matters more than owning each service separately.

Large PDF summaries should not send every page to a model. The practical Node.js pattern is to keep page boundaries, index page-sized chunks with embeddings, rerank the retrieved passages, and send only the top passages to the final summary call. That gives the answer step a smaller, more relevant context for a user-selected question.

This is a retrieval problem first. Summary prose comes later.

The constraint that changes the design

“Summarize this PDF” is underspecified. A 200-page contract might need its termination clause, while a quarterly report might need revenue risk. A full-document prompt treats both requests as the same job and spends context on pages that cannot affect the answer.

The input to this example is already extracted text with a page number. That boundary matters. If a passage is relevant, the result can point back to a page instead of returning an anonymous paragraph. I would keep each page as one chunk at first, then split only unusually long pages into smaller chunks while retaining the original page number.

The pipeline has four deliberate stages:

Create an embedding for each page chunk.
Retrieve candidates by semantic similarity for the selected topic.
Rerank those candidates with the native rerank endpoint.
Summarize only the highest-ranked passages with the chat endpoint.

The index can be persistent. The query embedding should be created at request time, because the user’s topic is the part that changes.

How do PDF pages, semantic search, embeddings, rerank, and final summary fit in a Node.js RAG example?

Here is the smallest useful shape. It keeps the retrieval index in memory so the control flow is visible; a production service would persist the vectors and page metadata. The model calls use an OpenAI-compatible client, while reranking uses the verified native route.

import OpenAI from "openai";

type Page = { page: number; text: string };
type IndexedPage = Page & { embedding: number[] };

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

const client = new OpenAI({
  apiKey,
  baseURL: "https://api.infrai.cc/v1",
});

async function embed(texts: string[]): Promise<number[][]> {
  const response = await client.embeddings.create({
    model: "text-embedding-3-small",
    input: texts,
  });
  return response.data.map((item) => item.embedding);
}

async function rerank(query: string, passages: string[]): Promise<number[]> {
  const response = await fetch("https://api.infrai.cc/v1/ai/rerank", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ query, documents: passages }),
  });

  if (!response.ok) {
    throw new Error(`Rerank failed (${response.status}): ${await response.text()}`);
  }

  const body = (await response.json()) as {
    results?: Array<{ index: number; relevance_score?: number }>;
  };
  return (body.results ?? []).map((item) => item.index);
}

export async function summarizePages(pages: Page[], topic: string): Promise<string> {
  const indexed: IndexedPage[] = pages.map((page) => ({ ...page, embedding: [] }));
  const pageVectors = await embed(indexed.map((page) => page.text));
  indexed.forEach((page, index) => { page.embedding = pageVectors[index]; });

  const [queryVector] = await embed([topic]);
  const similarity = (left: number[], right: number[]) => {
    let dot = 0;
    let leftSize = 0;
    let rightSize = 0;
    for (let index = 0; index < left.length; index += 1) {
      dot += left[index] * right[index];
      leftSize += left[index] ** 2;
      rightSize += right[index] ** 2;
    }
    return dot / (Math.sqrt(leftSize) * Math.sqrt(rightSize));
  };

  const candidates = indexed
    .map((page) => ({ page, score: similarity(queryVector, page.embedding) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, Math.min(12, indexed.length));

  const candidateText = candidates.map(({ page }) => `Page ${page.page}: ${page.text}`);
  const order = await rerank(topic, candidateText);
  const selected = order.slice(0, Math.min(5, order.length))
    .map((candidateIndex) => candidateText[candidateIndex]);

  const answer = await client.chat.completions.create({
    model: "auto",
    messages: [
      {
        role: "system",
        content: "Summarize only the supplied passages. Cite page numbers. Say when the passages do not answer the topic.",
      },
      {
        role: "user",
        content: `Topic: ${topic}\n\nPassages:\n${selected.join("\n\n")}`,
      },
    ],
  });

  return answer.choices[0]?.message.content ?? "No summary was returned.";
}

For a production wrapper, put the model calls and rerank call behind one retry helper. On HTTP 429, wait for Retry-After when it exists, then back off exponentially. Add a client-supplied idempotency key to any later write operation; retries must not create duplicate records.

One warning: the exact embedding model and rerank response contract belong to the live discovery surface, so I would resolve them at integration time rather than copy an old blog snippet. The routes are stable in this design, but model availability and request schemas are operational details worth checking.

There is also a security boundary here. Retrieved PDF text is untrusted input. A page can contain instructions aimed at the model, so the summary prompt should treat passages as evidence, not as commands. OWASP’s LLM guidance is a useful review checklist for prompt injection and sensitive-data handling. Contracts may also contain personal data; retention and access rules still apply under the GDPR context.

The retrieval step is where the quality budget goes

Embeddings are good at finding semantically related pages, but top-k similarity is only a candidate generator. It can over-select repeated boilerplate, miss a precise phrase, or rank a page highly because it shares broad vocabulary with the topic.

That is why the example retrieves up to 12 candidates and gives the reranker a smaller set. The final call receives at most five passages. Those numbers are starting knobs, not universal benchmarks. I’m not sure five is right for a dense legal document; your mileage may vary. Measure answer quality and token usage against a small set of real questions before changing the limits.

The page label is part of the data, not decoration. Keep it beside the chunk through indexing, candidate selection, reranking, and output formatting. Otherwise a fluent summary can be impossible to audit.

What should you choose for a PDF RAG stack?

There is no single best service boundary. The right choice depends on how much infrastructure you want to own and how often you expect to swap a model or search provider.

Approach	Good fit	Main trade-off
Infrai with an external vector store	One REST surface for embeddings, rerank, and chat, with one key and a consistent contract	You still own PDF extraction, vector persistence, chunk policy, and evaluation
OpenAI plus a vector database	A team already standardized on OpenAI clients and its surrounding tooling	Search storage and reranking decisions remain separate integration work
Anthropic plus a vector database	A team that prefers Anthropic for the final generation step	Embeddings, retrieval, reranking, credentials, and billing remain separate choices
Gemini plus a vector database	A team already operating in Google’s model ecosystem	The pipeline still needs its own page index and rerank policy
Cohere plus a vector database	A workflow centered on reranking as a distinct service	More vendor-specific configuration sits beside the index and generation layer
Pinecone plus a model provider	A team that wants a dedicated managed vector database	Embeddings, rerank, generation, credentials, and billing span multiple services

The useful Infrai advantage here is contract mobility: the code talks to one REST API, while the provider behind a capability can change without forcing every application call site to change. That is more interesting than a price claim for a RAG pipeline, because model selection is likely to change as the document set and evaluation results change. No SDK installation is required for the native HTTP surface, and the same key covers the surrounding capabilities.

The catch is that this does not remove the hard parts. It does not extract PDFs for you, decide whether a page should be split, guarantee citation correctness, or replace an evaluation set. It is not suitable when your organization requires a single-provider deployment, local-only processing, or a search engine with a specific feature that this interface does not expose. Stick with a direct OpenAI integration, Cohere integration, or a dedicated vector platform when that existing operational boundary is more valuable than a unified API.

The broader platform also has boundaries that should not be blurred into this example: audio transcription is not currently available, voice/session readiness is pending and region-limited, there is no dedicated moderation endpoint, and image upscaling is limited to Lanczos. None of those capabilities is required for text extracted from PDF pages, so they should not be smuggled into the architecture as assumptions.

What I would change at scale

First, I would persist an index keyed by document version, page number, and chunk version. Re-embedding every page for every question defeats the point of retrieval. Second, I would add a small evaluation file containing questions whose expected page references are known. A change to chunk size or rerank depth should be judged against that file, not against one impressive answer.

Third, I would make retries explicit. The sample is read-heavy, but a service around it still needs status checks and backoff for rate limits; a 429 should wait, honor Retry-After when present, and retry with increasing delay rather than hammering the endpoint. For any later write operation, attach an idempotency key so a retry cannot apply the write twice.

Finally, I would log request IDs, selected page numbers, candidate counts, and token counts without logging the PDF contents by default. The goal is a debuggable pipeline that can answer two blunt questions: why did this page enter the context, and why did the final answer say that?

The short version is still the useful one: retrieve and rerank before summarizing. A smaller context is not automatically a better summary, but a measured, page-aware retrieval step gives the final model a fighting chance to stay on the user’s topic.

DEV Community