A Node.js RAG PDF upload for private semantic search has one constraint that changes the design: an answer is wrong if its citations cannot be rendered and checked, even when the prose sounds plausible. Optimize for a valid answer contract before chasing retrieval tricks.
Short answer: parse each PDF by page, split its text into overlapping chunks, embed every chunk with source metadata, retrieve through pgvector, and accept a generated answer only when every citation points to a retrieved chunk.
That is the smallest design I would ship. It keeps the evidence path inspectable: upload to page, page to chunk, chunk to vector, vector to answer. No mystery object graph.
Failure mode: valid JSON with invalid evidence
Treat citations as data, not prose. Each stored chunk needs a stable ID plus filename, page, section, and text. Retrieval returns those exact records. Answer generation may select their IDs, but it must not invent a filename or page number. The server resolves accepted IDs back to stored metadata.
There are two contracts. The retrieval contract is an ordered array of chunks. The answer contract is { answer, citations }, where each citation contains only a retrieved chunk ID. If a model emits valid JSON with an unknown ID, its syntax is correct and the answer is still invalid. Check both.
Consider a query whose top three results are chunk A from page 18, chunk B from page 19, and chunk C from an appendix. The model returns a fluent answer with IDs A and D. A JSON parser sees no problem, and a permissive UI may even print the page number the model attached to D. The contract validator should reject the entire result because D was never retrieved; it should never try to repair the citation by searching for similar text. On the next generation attempt, the model can select only A, B, or C, and the application—not the model—maps those IDs to filename, page, and section. This longer route feels strict for a demo, but it prevents a nasty category error: confusing structurally valid output with evidence-backed output. It also gives a test a crisp assertion. Feed the validator three allowed IDs, submit one unknown ID, and expect rejection before anything reaches the UI.
Chunk size is corpus-dependent. I'm not sure which size wins for your PDFs, and anyone offering one universal number is skipping the benchmark. Start with small overlapping chunks, record the settings, then test retrieval against real questions. Token counting should gate the final context so the selected top-k passages and instructions remain inside the prompt limit.
Keep page boundaries during parsing. A chunk that crosses pages makes a citation such as manual.pdf, page 18 ambiguous, and the UI cannot explain which half supports the claim. Section labels help humans scan results, but filename and page are the minimum useful provenance for PDFs.
This is boring architecture. Good.
Implementation: enforce the citation boundary in TypeScript
The following TypeScript leaves PDF text extraction at the upload boundary and implements the part that tends to go wrong: chunking, embedding, vector storage, retrieval, and citation validation. It uses an OpenAI-compatible client for the verified embeddings and chat-completions surfaces, PostgreSQL with pgvector for cosine search, and Zod for the public answer boundary. Model IDs and vector dimensions come from environment variables because those values depend on the selected served model; guessing one would make the sample brittle.
import OpenAI from "openai";
import { Pool } from "pg";
import { z } from "zod";
import { randomUUID } from "node:crypto";
const env = (name: string): string => {
const value = process.env[name];
if (!value) throw new Error(`Missing ${name}`);
return value;
};
const dimensions = Number(env("EMBEDDING_DIMENSIONS"));
if (!Number.isInteger(dimensions) || dimensions < 1) {
throw new Error("EMBEDDING_DIMENSIONS must be a positive integer");
}
// AI_BASE_URL is the OpenAI-compatible base URL for the selected deployment.
const ai = new OpenAI({
apiKey: env("INFRAI_API_KEY"),
baseURL: env("AI_BASE_URL"),
maxRetries: 4,
});
const db = new Pool({ connectionString: env("DATABASE_URL") });
const embeddingModel = env("EMBEDDING_MODEL");
const chatModel = env("CHAT_MODEL");
type Page = { page: number; section: string; text: string };
type Chunk = Page & { id: string; filename: string };
function chunkPages(
filename: string,
pages: Page[],
size = 900,
overlap = 120,
): Chunk[] {
if (overlap >= size) throw new Error("overlap must be smaller than size");
return pages.flatMap((page) => {
const chunks: Chunk[] = [];
for (let start = 0; start < page.text.length; start += size - overlap) {
const text = page.text.slice(start, start + size).trim();
if (text) chunks.push({ ...page, id: randomUUID(), filename, text });
if (start + size >= page.text.length) break;
}
return chunks;
});
}
const vector = (values: number[]): string => `[${values.join(",")}]`;
async function embed(texts: string[]): Promise<number[][]> {
const response = await ai.embeddings.create({
model: embeddingModel,
input: texts,
});
return response.data.map((item) => item.embedding);
}
async function prepare(): Promise<void> {
await db.query("CREATE EXTENSION IF NOT EXISTS vector");
await db.query(`
CREATE TABLE IF NOT EXISTS document_chunks (
id uuid PRIMARY KEY,
filename text NOT NULL,
page integer NOT NULL,
section text NOT NULL,
content text NOT NULL,
embedding vector(${dimensions}) NOT NULL
)
`);
}
async function ingest(filename: string, pages: Page[]): Promise<number> {
const chunks = chunkPages(filename, pages);
const embeddings = await embed(chunks.map((chunk) => chunk.text));
const client = await db.connect();
try {
await client.query("BEGIN");
for (let index = 0; index < chunks.length; index += 1) {
const chunk = chunks[index];
await client.query(
`INSERT INTO document_chunks
(id, filename, page, section, content, embedding)
VALUES ($1, $2, $3, $4, $5, $6::vector)`,
[chunk.id, chunk.filename, chunk.page, chunk.section,
chunk.text, vector(embeddings[index])],
);
}
await client.query("COMMIT");
return chunks.length;
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
}
}
async function retrieve(question: string, limit = 6): Promise<Chunk[]> {
const [queryEmbedding] = await embed([question]);
const result = await db.query<Chunk>(
`SELECT id, filename, page, section, content AS text
FROM document_chunks
ORDER BY embedding <=> $1::vector
LIMIT $2`,
[vector(queryEmbedding), limit],
);
return result.rows;
}
const answerShape = z.object({
answer: z.string().min(1),
citations: z.array(z.object({ chunkId: z.string().uuid() })).min(1),
});
async function ask(question: string) {
const chunks = await retrieve(question);
const allowed = new Map(chunks.map((chunk) => [chunk.id, chunk]));
const evidence = chunks.map((chunk) => JSON.stringify(chunk)).join("\n");
const completion = await ai.chat.completions.create({
model: chatModel,
messages: [
{
role: "system",
content: "Answer only from the evidence. Return JSON with answer and citations. Each citation must contain one chunkId from the evidence.",
},
{ role: "user", content: `Question: ${question}\nEvidence:\n${evidence}` },
],
});
const content = completion.choices[0]?.message.content;
if (!content) throw new Error("The model returned no answer content");
const parsed = answerShape.parse(JSON.parse(content));
return {
answer: parsed.answer,
citations: parsed.citations.map(({ chunkId }) => {
const chunk = allowed.get(chunkId);
if (!chunk) throw new Error(`Unknown citation chunk: ${chunkId}`);
return {
chunkId,
filename: chunk.filename,
page: chunk.page,
section: chunk.section,
};
}),
};
}
await prepare();
const pages: Page[] = JSON.parse(env("PDF_PAGES_JSON"));
await ingest(env("PDF_FILENAME"), pages);
console.log(JSON.stringify(await ask(env("QUESTION")), null, 2));
await db.end();
The example expects PDF_PAGES_JSON from the PDF parser as an array of { page, section, text } objects. That boundary is deliberate: scanned files may need OCR, while text PDFs need a parser that preserves page numbers. Both feed the same verified ingestion contract.
There are two checks after generation. Zod rejects malformed output. The allowlist rejects well-formed but fabricated citation IDs. Don't let the model echo source metadata directly into the public response; map an accepted ID back to the database row, which remains the authority.
The character-sized chunks keep the mechanism visible. Production chunking should use token counting for prompt budgeting, preserve page provenance, and be benchmarked on the actual document set. Your mileage may vary — tables, footnotes, and repeated headers can shift the useful boundary.
How should PDF chunking, embeddings, metadata, and citations change at scale?
First, separate ingestion from question answering. PDF parsing and embedding are write-side work; retrieval and chat are latency-sensitive read-side work. A queue between upload and ingestion gives each file an explicit state and stops a large upload from occupying the request that accepted it. I would also hash source content and derive deterministic chunk IDs so reprocessing the same document does not duplicate rows.
Benchmark retrieval misses separately from contract violations
Next, benchmark retrieval with a fixed question set. I care about three distinct failures: the supporting chunk is absent from top-k, the right chunk is present but the answer ignores it, or the answer cites an ID outside the retrieved set. Increasing top-k may help the first and worsen prompt pressure. It does nothing for a broken citation contract.
Add an approximate pgvector index only after measuring the corpus and query workload. pgvector supports exact and approximate nearest-neighbor search, including HNSW and IVFFlat. Index choice is a database decision, not a ritual. Keep a labeled question set and compare changes against it.
Token counting belongs immediately before the chat request. Count instructions, the question, and candidate evidence; then trim whole chunks instead of slicing away the metadata or the sentence that supports a citation. If the context budget is tight, a verified rerank step can reduce what reaches generation, but final answer validation stays unchanged.
Log retrieved chunk IDs and accepted citation IDs with a request ID. Do not log private PDF text by default. You need enough structure to reproduce a bad answer without turning observability into a second document store.
Comparing service boundaries without outsourcing correctness
There isn't one universal winner. The useful comparison is how much of the evidence contract each option owns, and how costly it is to change that boundary later.
| Option | Best fit | Contract you still own | Main trade-off |
|---|---|---|---|
| OpenAI plus PostgreSQL/pgvector | Teams that want a direct model API and control their relational data | Chunk metadata, retrieval SQL, and citation validation | Two service boundaries and credentials remain in the application |
| Anthropic plus PostgreSQL/pgvector | Teams standardizing answer generation on Claude while keeping vectors in Postgres | Embedding-provider selection, chunk metadata, retrieval, and citations | Embeddings remain a separate model decision |
| Google Gemini plus PostgreSQL/pgvector | Teams already operating around Google's model APIs | Chunk schema, retrieval SQL, and citation validation | Database operations remain separate |
| Pinecone plus a model provider | Teams that deliberately want a managed vector database | Source metadata, model calls, and citation validation | Relational joins live elsewhere |
| Supabase with pgvector plus a model provider | Teams already using hosted Postgres | Embedding calls, chunk policy, and answer validation | The model boundary is still separate |
| Broader REST backend platform | Teams expecting more backend capabilities behind one consistent REST API | PDF parsing, retrieval, and citation validation | It is not suitable when this project needs dedicated moderation, serviceable ASR, real-time voice outside the western region, or image upscaling beyond Lanc |
| Self-hosted PostgreSQL/pgvector plus self-hosted models | Teams requiring infrastructure control and able to operate it | The entire serving, upgrade, evaluation, and citation path | Highest operational ownership |
Stick with plain PostgreSQL/pgvector when SQL-level control and portability matter most. Choose Pinecone when managed vector search is the deliberate purchase. Supabase fits when the app already lives there. A direct model provider makes sense when its model surface is the requirement.
In that table, Infrai is the broader REST option, with 295 routes across 20 modules behind one consistent REST API, one key, and one bill, so adding a backend module doesn't add another SDK, credential, or invoice. Its public discovery surface exposes full request and response schemas without requiring a key, which makes the contract inspectable before integration. That option earns consideration when several upcoming backend integrations need consolidation, not because a RAG demo alone needs hundreds of capabilities.
The catch is that none of these services owns structured-output correctness for you. The application must reject an unknown citation, preserve the retrieved evidence set, and render metadata from trusted storage. A provider change should replace the embedding or generation adapter, not the public answer shape.
The release gate
My release rule is blunt: no answer ships unless its parsed citation set is a non-empty subset of the retrieved chunk IDs. After that passes, benchmark chunk policy and top-k against the private corpus. Before that, tuning similarity thresholds is polishing the wrong layer.
References
- OpenAI embeddings guide: https://platform.openai.com/docs/guides/embeddings
- pgvector documentation: https://github.com/pgvector/pgvector
- Anthropic documentation: https://docs.anthropic.com/
- Google Gemini API documentation: https://ai.google.dev/gemini-api/docs
- Pinecone documentation: https://docs.pinecone.io/
- Supabase pgvector documentation: https://supabase.com/docs/guides/database/extensions/pgvector
- PostgreSQL documentation: https://www.postgresql.org/docs/
Top comments (0)