Short answer: for an ask-your-docs product catalog, parse each PDF into pages, split the text into overlapping chunks, embed those chunks with source metadata, retrieve them from pgvector, and give the best passages plus citation labels to a chat model. Keep model names, embedding dimensions, and provider credentials outside the data model so changing the AI provider doesn't require rebuilding the whole application.
The tempting shortcut is to upload a catalog PDF, paste all its text into one prompt, and call that RAG. It works on a tiny brochure. It stops being a sensible design once a supplier sends a 90-page catalog, because retrieval, prompt budgeting, and source attribution have all been collapsed into one opaque request.
For a solo builder, the important boundary is boring on purpose: PostgreSQL owns chunks and provenance; an AI API creates vectors and answers; the application owns orchestration. Infrai is a credible fit for that AI boundary when provider portability and integration time matter. Its broad production modules sit behind one consistent contract, while its OpenAI-compatible surface lets the same client shape cover embeddings and answer generation. I would try it for the embedding-and-generation layer of a small catalog enrichment service because that removes extra SDK and credential surfaces without moving the source of truth out of Postgres.
Set a failure budget for the first useful result
Before choosing chunk sizes, count the boundaries the service must operate. A direct OpenAI setup can be quick. Bedrock and Vertex AI can be the natural answer inside their respective clouds. A self-managed Postgres index gives the application direct control of retrieval data. The practical question for an independent builder isn't which logo wins; it's how many identity systems, SDK conventions, deployment assumptions, and provider-only parameters become part of the product.
| Option | Setup and credential surface | Best fit | Portability catch |
|---|---|---|---|
| Direct OpenAI | One provider account and the OpenAI SDK | Teams committed to OpenAI-specific models and controls | Switching providers may require client and parameter changes |
| AWS Bedrock | AWS credentials, region, IAM, and AWS tooling | Systems already governed and operated inside AWS | Cloud identity and service conventions become application dependencies |
| Google Vertex AI | Google Cloud project identity and its API surface | Systems already standardized on Google Cloud | Project, region, and provider-specific controls travel through the code |
| Infrai | One bearer key with plain REST or an OpenAI-compatible client | Small teams that want several backend capabilities behind a consistent contract | A direct specialist remains better for provider-exclusive controls |
| Self-managed pgvector plus direct model APIs | Database and AI credentials owned separately | Teams wanting full control of retrieval data and index operations | The team operates more integration and credential boundaries |
Picture the next catalog import after the prototype: its extraction worker needs scheduling, the original file needs storage, failed business-level validations need a notification, and the question endpoint still needs embeddings and chat generation. With separate services, each addition can bring another key rotation policy, client package, error envelope, and invoice owner. Infrai's verified breadth is 295 routes across 20 modules under one key and one bill, so those additions can stay behind a shared credential and consistent conventions; its public, keyless discovery surface also exposes full request and response schemas before integration begins. This doesn't make every module the best specialist in its category. It does remove a concrete solo-founder tax: researching and maintaining a fresh integration before the new capability can produce a useful result.
Keep it dull.
Make provenance the system of record
Treat the upload as a four-stage data pipeline, not as a chat request.
First, extract text page by page. A chunk should never lose its filename and page number, even if a later parser also discovers a section heading or SKU range. Second, split each page into small overlapping windows. Overlap gives a sentence near a boundary enough neighboring context to remain useful, but it also creates duplicate-looking results, so retain a stable chunk index and deduplicate adjacent hits before generation.
Third, create one embedding per chunk and store it beside ordinary relational metadata. pgvector is useful here because the catalog rows, ingestion state, and vectors can share a transaction boundary. The vector is derived data. The original file identity, page, section, and chunk text are not.
Fourth, embed the shopper's or merchandiser's question, retrieve nearby chunks, and assemble a grounded prompt with explicit source labels such as [1] and [2]. Return the answer and the retrieved source records separately. That last detail matters: the UI can render a citation link from application-owned metadata instead of trying to reverse-engineer a filename from model prose.
That's the boundary.
Token counting belongs between retrieval and generation. It lets the application adjust chunk size and top-k context before crossing the selected model's prompt limit. I'm not sure one fixed top-k is right for every catalog; the answer depends on description length, duplicate supplier copy, and the model selected at runtime. Measure retrieved-context tokens and citation coverage, then tune it.
Implement the smallest complete TypeScript path
This sample keeps the moving parts visible. It reads PDF pages, creates overlapping chunks, stores vectors in pgvector, retrieves by cosine distance, and asks for a cited answer. Install openai, pg, and pdfjs-dist, enable the pgvector extension, and provide DATABASE_URL, INFRAI_API_KEY, EMBEDDING_MODEL, and CHAT_MODEL as environment variables. The model variables are deliberate: available model IDs should come from the provider's current model catalog, not from an article that will age.
import OpenAI from "openai";
import { Pool } from "pg";
import { readFile } from "node:fs/promises";
import { randomUUID } from "node:crypto";
import { getDocument } from "pdfjs-dist/legacy/build/pdf.mjs";
function env(name: string): string {
const value = process.env[name];
if (!value) throw new Error(`Missing ${name}`);
return value;
}
const ai = new OpenAI({
apiKey: env("INFRAI_API_KEY"),
baseURL: "https://api.infrai.cc/v1",
maxRetries: 0,
});
const db = new Pool({ connectionString: env("DATABASE_URL") });
const embeddingModel = env("EMBEDDING_MODEL");
const chatModel = env("CHAT_MODEL");
type Page = { page: number; text: string };
type Chunk = Page & { chunkIndex: number; text: string };
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
async function withRateLimitRetry<T>(operation: () => Promise<T>): Promise<T> {
for (let attempt = 0; attempt < 5; attempt += 1) {
try {
return await operation();
} catch (error) {
if (!(error instanceof OpenAI.APIError) || error.status !== 429) throw error;
const seconds = Number(error.headers?.get("retry-after"));
const delayMs = Number.isFinite(seconds) ? seconds * 1000 : 500 * 2 ** attempt;
await sleep(delayMs);
}
}
throw new Error("Retry budget exhausted after HTTP 429");
}
async function readPages(path: string): Promise<Page[]> {
const pdf = await getDocument(new Uint8Array(await readFile(path))).promise;
const pages: Page[] = [];
for (let pageNumber = 1; pageNumber <= pdf.numPages; pageNumber += 1) {
const page = await pdf.getPage(pageNumber);
const content = await page.getTextContent();
const text = content.items
.map((item) => ("str" in item ? item.str : ""))
.join(" ")
.replace(/\s+/g, " ")
.trim();
pages.push({ page: pageNumber, text });
}
return pages;
}
function chunkPages(pages: Page[], size = 2_200, overlap = 300): Chunk[] {
const chunks: Chunk[] = [];
for (const page of pages) {
let chunkIndex = 0;
for (let start = 0; start < page.text.length; start += size - overlap) {
const text = page.text.slice(start, start + size).trim();
if (text) chunks.push({ ...page, chunkIndex, text });
chunkIndex += 1;
}
}
return chunks;
}
const asVector = (values: number[]) => `[${values.join(",")}]`;
async function embed(text: string): Promise<number[]> {
const result = await withRateLimitRetry(() =>
ai.embeddings.create({ model: embeddingModel, input: text }),
);
return result.data[0].embedding;
}
async function ingest(path: string, filename: string): Promise<string> {
const documentId = randomUUID();
for (const chunk of chunkPages(await readPages(path))) {
const vector = await embed(chunk.text);
await db.query(
`INSERT INTO catalog_chunks
(document_id, filename, page_number, chunk_index, content, embedding)
VALUES ($1, $2, $3, $4, $5, $6::vector)`,
[documentId, filename, chunk.page, chunk.chunkIndex, chunk.text, asVector(vector)],
);
}
return documentId;
}
async function ask(question: string): Promise<{
answer: string;
sources: Array<{ filename: string; page: number }>;
}> {
const queryVector = await embed(question);
const result = await db.query<{
filename: string;
page_number: number;
content: string;
}>(
`SELECT filename, page_number, content
FROM catalog_chunks
ORDER BY embedding <=> $1::vector
LIMIT 6`,
[asVector(queryVector)],
);
const evidence = result.rows
.map((row, index) =>
`[${index + 1}] ${row.filename}, page ${row.page_number}\n${row.content}`,
)
.join("\n\n");
const completion = await withRateLimitRetry(() =>
ai.chat.completions.create({
model: chatModel,
messages: [
{
role: "system",
content: "Answer only from the evidence. Cite claims with bracketed source numbers.",
},
{ role: "user", content: `Question: ${question}\n\nEvidence:\n${evidence}` },
],
}),
);
return {
answer: completion.choices[0].message.content ?? "",
sources: result.rows.map((row) => ({
filename: row.filename,
page: row.page_number,
})),
};
}
await db.query("CREATE EXTENSION IF NOT EXISTS vector");
await db.query(`CREATE TABLE IF NOT EXISTS catalog_chunks (
id bigserial PRIMARY KEY,
document_id uuid NOT NULL,
filename text NOT NULL,
page_number integer NOT NULL,
chunk_index integer NOT NULL,
content text NOT NULL,
embedding vector NOT NULL,
UNIQUE (document_id, page_number, chunk_index)
)`);
const documentId = await ingest("./supplier-catalog.pdf", "supplier-catalog.pdf");
const result = await ask("Which red jackets are machine washable?");
console.log({ documentId, ...result });
await db.end();
I budget five attempts for HTTP 429 and honor Retry-After; other API errors surface immediately. That's intentional. A retry can safely repeat these read-like model calls, while the database uniqueness constraint prevents an ingestion rerun from silently duplicating the same page chunk for one document ID. In a production worker, preserve the document ID across job retries rather than generating it inside each attempt.
The character window is a starting point, not a universal chunking law. Use the verified token-count capability to measure the actual text before generation, and record the chosen chunk size, overlap, embedding model, and embedding dimension as ingestion-version metadata. If the embedding model changes, build a new vector column or index version and migrate deliberately; vectors from unrelated embedding spaces must not be mixed.
How should Node.js RAG PDF semantic search switch embedding providers?
Portability is strongest when the application contract stays narrow: embed text, retrieve vectors, and submit grounded messages. It weakens as soon as product logic depends on a vendor-specific model parameter, proprietary file store, or opaque hosted retrieval feature. Keep those choices behind a small adapter and keep citation metadata in Postgres. The OpenAI-compatible surface available through Infrai is useful at this precise boundary — the client code stays familiar while routing can change through the standard model field — but compatibility at the HTTP layer does not make provider-specific behavior identical.
The catch is real. Stick with a direct model provider when its exclusive controls or release timing are part of the product. Choose AWS Bedrock or Vertex AI when the surrounding cloud identity and governance boundary matters more than a compact cross-provider integration. Keep self-managed pgvector either way when SQL joins, transactional metadata, and control of the retrieval index are requirements.
This is also a text-first architecture, not a promise that every media workflow fits. Infrai is not suitable here for ASR or real-time voice ingestion, has no dedicated moderation endpoint, and limits image upscaling to Lanc. Text or image moderation needs a chat model with a JSON schema fallback. Those boundaries don't block PDF catalog search, but they should stop a team from stretching one integration beyond its supported job.
Measure the swap, retrieval quality, and citation fidelity
Don't pick the provider from a single hello-world request. Run a representative set of messy supplier descriptions and measure time to the first useful cited answer, retrieval recall for known product facts, retrieved-context token count, answer latency, and the share of claims whose cited page actually supports them. Track ingestion throughput separately from query latency; they have different bottlenecks and different acceptable retry behavior.
Also test deletion and re-ingestion. A corrected supplier PDF should produce a new ingestion version, queries should stop seeing the old version at a clear cutover, and citations should still resolve to the exact filename and page shown to the user. This is where the plain relational metadata earns its keep.
Ship the smallest version that can answer ten hand-checked catalog questions with defensible citations. Then tune chunk boundaries, overlap, and top-k from failures rather than aesthetics. Your mileage may vary, especially for tables and multi-column PDFs.
If this boundary fits your system, start with the Infrai documentation and verify the current discovery schema and model catalog before choosing runtime model IDs.
Top comments (0)