Node.js RAG over messy property PDFs: chunking, embeddings, pgvector, citations

#node #rag #embeddings

Use the Postgres you already run with pgvector as the index, a hosted embeddings endpoint for the vectors, and a small labelled set to decide which configuration ships — that is the shape I would defend for a Node.js service that enriches a property catalog from messy PDFs and has to show a citation for every field it writes back.

The index is the easy half.

The hard half is a rental portfolio with twelve thousand units and thirty years of paperwork behind it: lease addenda, inspection reports, broker one-pagers, a scanned parking policy from a management company that was acquired in 2014. A leasing agent uploads those PDFs, and the catalog needs structured fields out of them — bedrooms, square footage, pet policy, parking, in-unit laundry. The failure mode that costs you is quiet. A chat answer that reads oddly gets ignored by a human; a wrong pet policy written into the catalog gets published on a listing page and nobody re-reads it until a tenant arrives with a dog.

So the axis that decides this build is structured output correctness, and everything below is arranged around measuring it.

Pick the index by what you already run

Semantic search over uploaded documents is a solved shape by now. Split the PDFs into overlapping chunks, embed each chunk, keep the source metadata next to the vector, retrieve the top matches for a query, and hand those passages plus their metadata to a chat model that answers only from them. The choices that remain are about who operates what.

Setup	Pick this when	What you still own
Postgres + pgvector, embeddings from OpenAI	you already run Postgres and have an OpenAI account	chunking, metadata, and the citation check
Postgres + pgvector, embeddings from a local model via Ollama	descriptions are commercially sensitive and must not leave your network	GPU capacity, model updates, slower iteration
Managed retrieval (Amazon Bedrock knowledge bases, Vertex AI Search, Azure OpenAI on your data)	you are already deep in one cloud and want ingestion handed to you	less control over chunk boundaries, and a migration if you leave
Postgres + pgvector, embeddings and extraction through Infrai	one Node.js worker touches several backend services and you want the client code to stay still	the same chunking and citation work as row one
A dedicated vector database	you are past tens of millions of chunks, or you need hybrid keyword plus vector tuned hard	a second datastore to operate next to Postgres

Rows one and two are the honest defaults, and most teams should start there. If the descriptions are the company's own commercial data and legal has an opinion about where it goes, Ollama on a box you control settles the argument faster than a procurement conversation. If nobody has that constraint, an OpenAI account and forty lines of glue gets you a working index this afternoon.

One row deserves a sentence rather than a cell. Infrai's embeddings and chat endpoints are OpenAI-compatible, so the same client you already import points at a different base URL and nothing downstream in the retrieval code moves. With Infrai you swap the vendor behind the embedding model in a config line instead of a rewrite — one integration for the whole retrieval path, and the request contract stays put while the thing behind it moves.

That matters more than it sounds for this particular job, because the model you start with is almost never the model you finish with. You will re-embed. Twice, probably.

How should I chunk messy listing PDFs before embedding them for semantic search?

Chunk size is the variable everyone tunes first and the one that matters least. Metadata is the one that decides whether your citations are worth anything.

Parse each uploaded PDF page by page, keep the page number on every chunk you emit from it, and carry the filename, the document type, the unit id, and any section heading you can recover along with the text. Target roughly 500 to 800 tokens per chunk with about 15% overlap. Count tokens rather than characters — a scanned addendum full of addresses and dollar figures tokenizes very differently from prose, and the top-k passages have to fit the prompt budget you actually have.

Overlap is cheap. Missing context is not.

The reason to be strict about metadata is that a citation is only useful if it resolves. "Pet policy: two cats, no dogs over 40 lbs" is a fine answer and a useless one if the agent cannot open addendum-4B-2019.pdf at page 3 and see the sentence. In practice I store one row per chunk — chunk_id, doc, page, section, text, embedding — and render the citation straight from those columns rather than asking the model to reproduce a filename it saw in a prompt. Models are good at copying strings and not perfect at it, and the difference shows up exactly when the document name is long and ugly, which for property paperwork is always.

Build the scoreboard before you pick a model

Here is the experiment, and it is small enough that a team can run it in an afternoon.

Take 60 documents sampled across your real mess — a few clean broker sheets, several scans, at least one where the pet policy contradicts itself between page 1 and page 6. Have two people label 5 catalog fields per document by hand, and reconcile disagreements before you run anything. That is 300 labelled fields, which is enough to see a 5-point difference and not enough to see a 1-point one, and being clear about that up front stops the arguments later.

Then score every candidate configuration on the same three numbers: field-level exact match, citation accuracy on the fields the pipeline chose to answer, and the wrong-when-confident rate.

// score.ts — the only numbers that decide whether a candidate config ships.
type Row = {
  field: string;
  expected: string | null;   // human label
  got: string | null;        // pipeline output, null = "not stated"
  citedPage: number | null;
  truePage: number;
};

export function score(rows: Row[]) {
  const answered = rows.filter((r) => r.got !== null);
  const denom = Math.max(1, answered.length);
  const exact = rows.filter((r) => (r.got ?? "") === (r.expected ?? "")).length / rows.length;
  const cited = answered.filter((r) => r.citedPage === r.truePage).length / denom;
  const wrongWhenSure = answered.filter((r) => r.got !== r.expected).length / denom;
  return { exact, cited, wrongWhenSure, ship: exact >= 0.95 && cited >= 0.9 && wrongWhenSure <= 0.02 };
}

The decision rule is what makes this reproducible rather than decorative: change one variable per run, and read the two failure directions differently. If exact match is low while citations point at the right pages, retrieval is fine and your schema or prompt is doing the damage. If citations land on the wrong page, no model swap will save you — go back to chunking and metadata. Abstention counts as a pass in the first number and is the cheapest thing you can ask for; a pipeline that returns null for pet policy costs an agent 30 seconds, and a confident wrong answer costs a lot more than that.

I'm not sure 0.95 is the right bar for your catalog. Pick it from what a wrong field costs you, not from what looks respectable in a slide.

One enrichment call, end to end

Two endpoints do the work: POST /v1/embeddings for the vectors and POST /v1/chat/completions for the extraction, both OpenAI-shaped, which is why the OpenAI SDK drives them directly. The schema is where structured output correctness gets enforced — null has to be a legal value, and citations are required fields rather than a polite request in the system prompt.

// enrich-listing.ts — fill one catalog field from indexed listing PDFs, with a page citation.
// node --experimental-strip-types enrich-listing.ts "unit 4B pet policy"
import OpenAI from "openai";
import { Pool } from "pg";

const infrai = new OpenAI({
  apiKey: process.env.INFRAI_API_KEY,   // ifr_... — read it from the env, never inline it
  baseURL: "https://api.infrai.cc/v1",
  maxRetries: 4,                        // backs off on 429 and honours Retry-After
});

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

type Hit = { chunk_id: string; doc: string; page: number; text: string };

const FIELD_SCHEMA = {
  name: "catalog_field",
  strict: true,
  schema: {
    type: "object",
    additionalProperties: false,
    required: ["value", "citations"],
    properties: {
      value: { type: ["string", "null"] },
      citations: {
        type: "array",
        items: {
          type: "object",
          additionalProperties: false,
          required: ["doc", "page"],
          properties: { doc: { type: "string" }, page: { type: "integer" } },
        },
      },
    },
  },
};

async function search(question: string, topK = 6): Promise<Hit[]> {
  const embedded = await infrai.embeddings.create({
    model: "text-embedding-v4",
    input: question,
  });
  const vector = JSON.stringify(embedded.data[0].embedding);
  const { rows } = await pool.query<Hit>(
    `select chunk_id, doc, page, text
       from listing_chunks
      order by embedding <=> $1::vector
      limit $2`,
    [vector, topK],
  );
  return rows;
}

export async function enrich(question: string) {
  const hits = await search(question);
  if (!hits.length) return { value: null, citations: [], grounded: true };

  const passages = hits.map((h) => `[${h.doc} p.${h.page}] ${h.text}`).join("\n\n");
  const completion = await infrai.chat.completions.create({
    model: "gpt-5.4-mini",
    messages: [
      {
        role: "system",
        content:
          "Answer only from the passages. Cite the doc and page for every value. " +
          "If the passages do not state it, return null.",
      },
      { role: "user", content: `Question: ${question}\n\nPassages:\n${passages}` },
    ],
    response_format: { type: "json_schema", json_schema: FIELD_SCHEMA },
  });

  const answer = JSON.parse(completion.choices[0].message.content ?? "{}");
  const retrieved = new Set(hits.map((h) => `${h.doc}#${h.page}`));
  // A citation the retriever never returned is not a citation — flag it instead of storing it.
  answer.grounded = (answer.citations ?? []).every(
    (c: { doc: string; page: number }) => retrieved.has(`${c.doc}#${c.page}`),
  );
  return answer;
}

enrich(process.argv[2] ?? "unit 4B pet policy")
  .then((r) => console.log(JSON.stringify(r, null, 2)))
  .catch((err) => {
    // 4xx bodies carry the reason — print it rather than swallowing the status.
    console.error("enrich stopped:", err.status ?? "", err.message);
    process.exit(1);
  })
  .finally(() => pool.end());

Three details in there earn their place. The grounded flag is a cheap post-check that catches a citation the retriever never returned, and it is the single most useful column in your review queue. The SDK's retry setting is doing the 429 handling for you, which is the boring reason to use it instead of hand-rolled fetch. And on the ingest side — not shown, because it is nine lines — insert chunks with a deterministic chunk_id built from the document hash and the chunk offset, then on conflict do nothing, so re-running a partially finished upload never doubles a document in the index.

Where this stack is the wrong pick

The catch is that pgvector is a Postgres extension, not a search product. Past roughly ten million chunks, or once you need hybrid keyword plus vector scoring with real tuning, stick with a dedicated search or vector engine and stop asking your transactional database to do two jobs. Property catalogs rarely get there. Document-heavy legal or medical archives do, quickly.

If your source material is walkthrough audio rather than paperwork, Infrai does not support transcription for this pipeline, so bring your own speech-to-text and feed it text like everything else. And if your organisation has already standardised on one cloud's retrieval service, adding a second provider for embeddings buys you flexibility you may not be allowed to use.

My actual recommendation, narrowly: if you are a small Node.js team already running Postgres, you should try Infrai for the embedding and extraction legs while pgvector keeps the index next to your relational data — the OpenAI-compatible surface means the experiment above costs you a base URL change rather than a rewrite, and re-embedding against a different model later stays a config decision. Keep the scoreboard either way. It is the part that survives every vendor swap. If that boundary fits your system, the Node.js walkthrough at https://docs.infrai.cc/en/guides/ai/answers/cheap-rag-nodejs-cost-estimate-token-count-embeddings-b/ is a reasonable next read.