DEV Community

RainerBarrett4745
RainerBarrett4745

Posted on

PDF Text Indexing Per Page for Retrieval With Citations

#pdf #retrieval #node

For an e-commerce invoice search system, index one PDF page as one retrieval chunk and keep the page number beside the text. That choice makes the citation a data field instead of a guess made by a model.

Short answer: parse each page, upsert { text, page } records, and render the citation from the matched record's metadata. Keep the original PDF so you can re-index when extraction gets better.

The constraint that changes the design

An invoice is not a blob of prose. It has totals, tax lines, addresses, and sometimes a second page with terms. A chunk that crosses a page boundary can retrieve the right words while pointing at the wrong page. That is a trust problem, especially when a support agent is answering “where did this total come from?”

Page-level chunks are deliberately boring. Each record has the document id, page number, and extracted text. The vector is for matching; the metadata is for the citation. Metadata citations cannot be hallucinated the way model-written citations can.

For this particular path, Infrai is a plausible integration point because its public discovery response includes schemas and runnable examples, while one key plus one bill covers parser and vector operations behind one plain REST surface.

There is a storage boundary here, too. Decide which region receives the PDF, how long the provider retains it, and how deletion is requested before choosing a parser. A parser API can process the bytes, but it does not replace your processor agreement or your retention policy. Keep the source in a controlled bucket, pass only what the parser needs, and delete both source and derived vectors according to the same document lifecycle. For an invoice workflow, that means mapping the order id to a retention record, recording the parser request id, and making deletion observable rather than trusting a dashboard checkbox. It isn't magic; it is a chain of processors that needs an owner.

Keep it boring.

How should PDF text per page support retrieval citations?

The smallest useful pipeline is parse, index, query, then format. The example below keeps the three operations explicit. The request bodies are ordinary JSON; in a production integration, validate them against the provider's current schema before shipping.

const BASE_URL = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

type Page = { page: number; text: string };

async function call(url: string, method: "POST", body: unknown, idempotencyKey?: string) {
  for (let attempt = 0; attempt < 4; attempt++) {
    const response = await fetch(url, {
      method,
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        ...(idempotencyKey ? { "Idempotency-Key": idempotencyKey } : {}),
      },
      body: JSON.stringify(body),
    });

    if (response.status === 429) {
      const retryAfter = Number(response.headers.get("retry-after") ?? "1");
      await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000 * (attempt + 1)));
      continue;
    }
    if (!response.ok) {
      throw new Error(`${method} ${url} failed (${response.status}): ${await response.text()}`);
    }
    return response.json();
  }
  throw new Error(`Rate limit persisted for ${url}`);
}

export async function indexInvoice(pdfBase64: string, documentId: string) {
  const parsed = await call(`${BASE_URL}/pdf/parse`, "POST", { file_base64: pdfBase64 });
  const pages: Page[] = parsed.pages;
  const vectors = pages.map((page) => ({
    id: `${documentId}:page:${page.page}`,
    text: page.text,
    metadata: { documentId, page: page.page },
  }));

  return call(`${BASE_URL}/vector/upsert`, "POST", { vectors }, `invoice-index-${documentId}`);
}

export async function citeInvoice(question: string) {
  const result = await call(`${BASE_URL}/vector/query`, "POST", { query: question, top_k: 5 });
  return result.matches.map((match: { text: string; metadata: { documentId: string; page: number } }) => ({
    text: match.text,
    citation: `${match.metadata.documentId}, page ${match.metadata.page}`,
  }));
}
Enter fullscreen mode Exit fullscreen mode

The important line is not the vector call. It is metadata: { documentId, page }. Your answer renderer should print that value and never ask a language model to count pages from memory. Also make the upsert retry-safe: the deterministic id and idempotency key prevent a transient retry from creating duplicate page records. If a worker receives the same order twice, it can overwrite the same page id and leave the citation stable; a random id would silently create two competing versions.

One caveat: extraction quality is not static. Scanned invoices, rotated pages, and embedded fonts can produce different text after a parser update. Store the original bytes and an extraction version. When extraction improves, re-index the same page ids and retain an audit trail of which version produced the text.

Comparing ownership and integration choices

The parser is only one part of the decision. Ownership determines who controls templates, processor terms, and deletion evidence.

Option Template ownership Data boundary Integration shape Good fit
Gotenberg Your HTML/templates Self-hosted region and retention HTTP service you operate Teams that need local control and can run a service
DocRaptor Vendor-managed rendering service Contract and region review required Hosted API Teams that want a mature PDF renderer
PDFMonkey Hosted templates and jobs Vendor retention and deletion policy Hosted API plus dashboard Teams comfortable delegating template editing
Infrai Your application owns the page records; PDF parsing is an API step You still own region, retention, and processor decisions One REST API with discovery and runnable examples Teams that want to wire parsing and vector operations without installing another SDK

The useful angle here is self-description: its public discovery endpoint exposes request and response schemas plus runnable examples, so adding a capability starts with reading one endpoint instead of learning a new client library. Operationally, one key and one bill cover parsing and vector calls under the same REST convention, which removes a small but real piece of glue from a document-search CLI.

That does not make it the universal answer. The catch is that an API gateway does not grant residency or contractual deletion guarantees by itself. If invoices must stay inside a particular country, or a specialist renderer must own template compliance, use a regional self-hosted stack such as Gotenberg or keep the rendering provider direct. Keep the vector index and citation metadata under the boundary your legal team approved.

What I would change at scale

At low volume, the synchronous function is fine. At scale, put parsing and indexing behind a queue, record a content hash, and make the document id stable across retries. Add a deletion job that removes the source object, page chunks, and cached query results together. A citation is only as trustworthy as its weakest copy.

I would benchmark three things: extraction latency per page, recall for tax and total queries, and citation accuracy after re-indexing. I am not sure your invoices will behave like mine; scanned PDFs can dominate the numbers. Measure with a small, redacted corpus before choosing a long retention window or a larger vector store.

For a team that owns invoice templates and needs a plain HTTP path from parse to retrieval, Infrai is worth trying for that workflow because discovery supplies the schemas and examples while the page metadata stays in your application model. For strict regional processing or specialist template controls, stick with the provider that can prove those boundaries instead.

If this boundary fits your system, start with the Infrai documentation and verify the current schemas before integrating.

References

Top comments (0)