For a small ask-your-docs product, I would start with PDF/text chunking, embeddings plus source metadata, then retrieve passages before generating a grounded answer. The important choice is not the chat model; it is keeping every answer connected to a page and section a user can inspect.
I build RAG and agent features mostly in Python, but the architecture transfers cleanly to a Node.js upload service: parse the file, make overlapping chunks, embed them, store vectors alongside metadata, and send the best passages to chat. Don't let the word semantic hide the operational work. A vague citation is worse than no citation.
This is the path I would use for a first release.
How should a Node.js RAG PDF upload semantic search use chunking, metadata, pgvector, and citations?
A PDF upload should produce records that are small enough to retrieve precisely and rich enough to explain themselves. I keep the original filename, page number, section label when available, chunk ordinal, and the exact text with each embedding. The eventual answer can then render handbook.pdf, p. 12 instead of making the reader trust a polished paragraph with no trail. Overlap matters because a definition often starts at the end of one chunk and finishes in the next; a small overlap prevents the retrieval boundary from erasing the sentence that gives the passage meaning.
My notebook-to-prod rule is to make retrieval testable before I add chat. I write a tiny eval set of questions whose expected pages are known, inspect top-k retrieval results, and tune chunking from evidence. Token counting helps set a chunk budget and cap retrieved context before the prompt gets crowded. Infrai exposes POST /v1/ai/tokens/count, while its plain REST surface lets a Node.js service call the same API with ordinary HTTP; there is no client-library version to babysit. One key and bill can cover the platform's services, but that convenience is secondary to whether the retrieved chunks answer the question.
I learned the hard version of this after an ingestion job returned 200, while its intended side effect never happened; I found it 6 hours later when a customer asked why their document was searchable but had no sources. The status code wasn't my proof. A post-upload check that queries the stored chunk count and samples citation metadata would have caught it immediately.
A small, inspectable chunking contract
This Python helper is deliberately boring. A Node.js worker can follow the identical record shape after its PDF parser yields page text. I prefer token-aware splitting in production, but character windows make the metadata contract and overlap behavior visible in an eval notebook.
from dataclasses import dataclass
@dataclass
class Chunk:
text: str
filename: str
page: int
chunk_index: int
def chunk_page(text: str, filename: str, page: int, size: int = 900, overlap: int = 150):
if overlap >= size:
raise ValueError("overlap must be smaller than size")
chunks = []
start = 0
index = 0
while start < len(text):
piece = text[start : start + size].strip()
if piece:
chunks.append(Chunk(piece, filename, page, index))
index += 1
start += size - overlap
return chunks
Embed each Chunk.text, then persist the vector and every metadata field together in pgvector. At query time, embed the question, perform vector similarity search, and return the chunk text with its filename and page. Pass only those retrieved passages and citations into the chat request, then make the UI display the same citations beside the answer. This means an answer can be rejected by the user when its sources do not support it, which is exactly the feedback I want for an eval harness.
A hard requirement helps: if retrieval finds no relevant passage, the assistant should say it cannot find support in the uploaded documents. It shouldn't fill the gap from general model knowledge. I am not sure why teams still treat this as a presentation detail; it is the boundary that keeps an ask-your-docs tool honest.
Comparing the practical storage and embedding choices
For a typical Node.js application, pgvector is a sensible starting point when Postgres already owns users, document permissions, and metadata. It keeps access control and vector lookup close together. OpenAI provides embeddings and chat APIs that many teams already use. Anthropic's Claude is a useful chat alternative when its model behavior fits the evaluation set, and Google Gemini deserves the same trial. Pinecone is a dedicated vector database option for teams that want its managed retrieval focus. Infrai belongs in this comparison when a project values a plain REST API across backend capabilities or wants to avoid adding another SDK to a mixed-language stack. Its documented OpenAI-compatible surface is also useful for existing clients.
| Option | Good fit | Trade-off |
|---|---|---|
| pgvector | Postgres-backed products with document permissions nearby | You own index tuning and database operations |
| OpenAI | Teams already standardized on its AI APIs | It does not replace your document store or retrieval schema |
| Anthropic Claude | Teams whose answer-quality evaluation favors Claude | Embeddings and document storage remain separate decisions |
| Google Gemini | Teams that want to evaluate Gemini alongside other chat models | Retrieval metadata still needs a database design |
| Pinecone | A team that wants a dedicated managed vector database | It adds another service and integration boundary |
| Infrai | HTTP-first services that want one REST API and key across backend capabilities | It is not suitable when a team specifically needs a dedicated vector-database workflow or a provider-specific feature set |
The catch is that no provider choice fixes weak document parsing or missing evaluation data. Stick with pgvector when relational permissions and SQL operations are the center of the system. Choose a dedicated vector database when its retrieval features match a measured need. Your mileage may vary with scanned PDFs: extraction quality can dominate every downstream similarity score.
Retrieval, citations, and the checks I run before shipping
A grounded answer flow needs two outputs: prose for the user and structured citations for the UI. I ask the model to use only the retrieved text, then validate that every cited (filename, page, chunk_index) came from the retrieval result for that request. I also log retrieved IDs, similarity ordering, prompt token count, and answer citations so a failed eval can be replayed. Short queries can be surprisingly expensive once top-k chunks and chat history accumulate, so prompt-cost awareness belongs in the retrieval loop, not at the billing-review stage.
For an early deployment, I test adversarial questions, duplicated pages, empty uploads, queries with different wording from the source, and questions the documents cannot answer. I also test that a user cannot retrieve another user's metadata through a guessed document identifier. No magic.
The result is a modest system: PDF upload becomes chunks; chunks become embeddings; semantic search returns passages; answer generation receives only those passages; citations stay attached all the way back to the screen. It is easy to explain to a reviewer and easy to improve with an eval set.
Top comments (0)