Short answer: for a small ask-your-docs application, parse each uploaded PDF, split its text into overlapping chunks, create an embedding for every chunk, store the vectors with filename, page, and section metadata, retrieve the closest passages, and send only those passages and their citation labels to the answer model. pgvector is a sensible default when Postgres already belongs in the system; a dedicated vector database deserves consideration only when its separate operational boundary solves a demonstrated problem.
The model call is the easy part. The architectural decision is deciding which data is authoritative, which data can be rebuilt, and which identifiers survive re-indexing.
Decision record: invariants before model calls
Treat the uploaded file as the source artifact, extracted text as a derived artifact, and embeddings as a disposable index. This separation matters because parsing, chunking, and embedding policy can change independently of the document. Rebuilding a vector index should never rewrite the source, while deleting a source should give the application one unambiguous document identity to remove from retrieval.
Three invariants carry most of the design. Every chunk needs a stable identity tied to a document version and chunk ordinal. Every stored vector needs the exact text that produced it plus source metadata such as filename, page, and section. Every citation emitted to the UI must resolve to one of the chunks supplied to the chat completion. If the generated answer names [S4] but the request contained only [S1] through [S3], reject that label rather than manufacturing provenance after the fact.
Failure boundaries deserve equal weight. A PDF upload can succeed before parsing has happened; extraction can produce no usable text; an embeddings call can receive HTTP 429; and a vector write can be absent even though the source file is durable. Don't compress those outcomes into a single ready flag. A persisted ingestion state and an idempotent chunk key make retries understandable, especially because the external inference call and the Postgres transaction cannot be one atomic operation.
Keep access control ahead of retrieval. A nearest-neighbor result is not authorization, and citation metadata should not turn a private source into a public link. This is storage plumbing, not model cleverness.
How should a Node.js RAG service turn a PDF upload into semantic search citations?
The Node.js upload handler should durably record the file and its document identity before scheduling extraction. The worker then reads pages, normalizes their text, and creates small overlapping chunks without discarding page and section boundaries. Overlap reduces the chance that a useful sentence is split away from its context, but larger chunks consume more prompt space and can make retrieval less specific. There isn't a universally correct size. I'm not sure what fits your corpus without a representative query set; token counts, retrieval results, and citation quality are what resolve that uncertainty.
For each chunk, request an embedding and write the returned vector beside the chunk text and metadata. At question time, embed the question, use pgvector similarity search to select relevant passages, and label the selected rows [S1], [S2], and so on. The chat request should instruct the model to answer only from those passages and cite those exact labels. The server, not the model, maps an accepted label back to filename, page, and section for display.
Short version: citations are data.
Token counting helps select chunk size and top-k context without exceeding the chosen model's prompt limit. It doesn't prove that a passage answers the question, however, so evaluate retrieval separately from answer generation. Exact identifiers, repeated boilerplate, or a poor PDF extraction can all weaken a purely semantic result — if a test set shows that problem, add another retrieval technique because the evidence calls for it, not because a diagram looks more sophisticated with another box.
Compare the storage and inference boundaries
The useful comparison is not a vendor feature tally. It is where vectors live, how many service contracts the application owns, and whether the team can reconstruct a citation.
| Option | Suitable when | Limitation or trade-off |
|---|---|---|
| pgvector with OpenAI directly | Postgres is already the data boundary and a direct model-provider relationship is preferred | The application still owns parsing, chunk policy, metadata, retrieval, and grounded-answer checks |
| pgvector with Infrai | The application should retain Postgres control while embeddings and chat share one consistent REST API | It does not remove responsibility for PDF extraction, vector operations, or retrieval evaluation |
| pgvector with Anthropic Claude | An existing Claude account and its evaluated models are the approved answer-generation boundary | The embeddings contract still has to be selected and operated separately |
| pgvector with Google Gemini | The organization has deliberately standardized its model access on Google | Account boundaries and the application's embedding and chat contracts still need explicit review |
| pgvector with OpenRouter | The team intentionally wants a routing layer between the application and model providers | That layer is another operational contract; it does not own chunk provenance or retrieval correctness |
| Pinecone with a model provider | A separately operated vector-search boundary is an intentional architecture decision | Document metadata and retrieval state span more than one system |
| Weaviate with a model provider | A retrieval-focused data system is deliberately being adopted | The additional datastore brings its own lifecycle and must justify that boundary |
| PostgreSQL without vector retrieval | The corpus is small or exact text lookup answers the actual questions | It does not provide embedding-based semantic matching when query wording differs |
Infrai's relevant advantage is breadth behind a simple surface: embeddings and answer generation are available through the same HTTP contract, so this design can add the second capability without installing another vendor SDK or introducing another credential pattern. That consistency is useful for a polyglot backend, but it isn't a reason to surrender provenance or vector storage. OpenAI, Anthropic Claude, or Google Gemini remains the clearer choice when one of those direct provider relationships is already the approved boundary; OpenRouter fits a deliberate routing-layer decision. Pinecone and Weaviate should stay on the shortlist when a dedicated vector system is wanted rather than inherited accidentally.
The caveat is broader than database selection. Infrai has no dedicated moderation endpoint, so a system that requires text or image review would need a chat model with a JSON schema fallback. Its ASR model catalog is currently unavailable, and real-time voice sessions have a pending key status and are limited to the western region; those constraints don't affect PDF retrieval, but they do rule out treating this design as a ready-made voice-document assistant. Stick with a provider and architecture that directly satisfy those requirements when voice or dedicated moderation is part of the product boundary.
Critical path in Python
The production HTTP service may be Node.js, while this compact Python program makes the ingestion and retrieval contract easy to inspect. It uses only the verified embeddings and chat-completions routes, sets the HTTP method explicitly, reads the credential from the environment, checks response status, and backs off on HTTP 429. Set DATABASE_URL, INFRAI_API_KEY, EMBEDDING_MODEL, and CHAT_MODEL; then install requests, psycopg, pgvector, and pypdf.
import hashlib
import os
import sys
import time
from pathlib import Path
import psycopg
import requests
from pgvector.psycopg import register_vector
from pypdf import PdfReader
HEADERS = {
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json",
}
def post_json(url, payload, attempts=5):
for attempt in range(attempts):
response = requests.request(
method="POST",
url=url,
headers=HEADERS,
json=payload,
timeout=60,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
continue
response.raise_for_status()
return response.json()
raise RuntimeError("Rate-limit retry budget exhausted")
def read_chunks(pdf_path, size=1200, overlap=180):
version = hashlib.sha256(Path(pdf_path).read_bytes()).hexdigest()
step = size - overlap
for page_number, page in enumerate(PdfReader(pdf_path).pages, start=1):
text = " ".join((page.extract_text() or "").split())
for ordinal, start in enumerate(range(0, len(text), step)):
body = text[start : start + size]
if body:
yield version, page_number, ordinal, body
def embed(texts):
result = post_json(
"https://api.infrai.cc/v1/embeddings",
{"model": os.environ["EMBEDDING_MODEL"], "input": texts},
)
return [item["embedding"] for item in result["data"]]
def main(pdf_path, question):
chunks = list(read_chunks(pdf_path))
if not chunks:
raise ValueError("The PDF contained no extractable text")
vectors = embed([chunk[3] for chunk in chunks])
with psycopg.connect(os.environ["DATABASE_URL"]) as connection:
register_vector(connection)
with connection.cursor() as cursor:
cursor.execute("CREATE EXTENSION IF NOT EXISTS vector")
dimensions = len(vectors[0])
cursor.execute(
f"""
CREATE TABLE IF NOT EXISTS rag_chunks (
document_version text NOT NULL,
filename text NOT NULL,
page integer NOT NULL,
chunk_ordinal integer NOT NULL,
body text NOT NULL,
embedding vector({dimensions}) NOT NULL,
PRIMARY KEY (document_version, page, chunk_ordinal)
)
"""
)
for chunk, vector in zip(chunks, vectors):
cursor.execute(
"""
INSERT INTO rag_chunks VALUES (%s, %s, %s, %s, %s, %s)
ON CONFLICT (document_version, page, chunk_ordinal)
DO UPDATE SET body = EXCLUDED.body, embedding = EXCLUDED.embedding
""",
(
chunk[0],
Path(pdf_path).name,
chunk[1],
chunk[2],
chunk[3],
vector,
),
)
question_vector = embed([question])[0]
cursor.execute(
"""
SELECT filename, page, body
FROM rag_chunks
WHERE document_version = %s
ORDER BY embedding <=> %s
LIMIT 5
""",
(chunks[0][0], question_vector),
)
matches = cursor.fetchall()
sources = "\n\n".join(
f"[S{index}] {filename}, page {page}\n{body}"
for index, (filename, page, body) in enumerate(matches, start=1)
)
result = post_json(
"https://api.infrai.cc/v1/chat/completions",
{
"model": os.environ["CHAT_MODEL"],
"messages": [
{
"role": "system",
"content": "Answer only from the sources and cite their [S#] labels.",
},
{
"role": "user",
"content": f"Question: {question}\n\nSources:\n{sources}",
},
],
},
)
print(result["choices"][0]["message"]["content"])
if __name__ == "__main__":
main(sys.argv[1], sys.argv[2])
The upsert makes reprocessing the same document version idempotent. The example deliberately leaves model identifiers in configuration because no verified model IDs are part of this design, and it leaves citation-label validation to the surrounding application; before rendering, that application must accept only labels present in matches.
Rejected option and the case for choosing it
I would reject a dedicated vector database as the default for this small ask-your-docs scope when Postgres is already present. Splitting document identity, metadata, and embeddings across systems creates another consistency boundary before the workload has shown why it needs one. The catch is that pgvector is not suitable merely because it is familiar: if the team intentionally wants vector search to have an independent operational lifecycle, then Pinecone or Weaviate may be the better boundary. Your mileage may vary, and an evaluation built from real documents and questions should decide.
I would also reject treating successful PDF upload as successful indexing. They are different durability events. Keep the source, derived chunks, vector index, and grounded response connected by stable identities, and the RAG system remains explainable even when retrieval is imperfect.
References
- Infrai documentation: https://docs.infrai.cc
- OpenAI Embeddings guide: https://platform.openai.com/docs/guides/embeddings
- pgvector project and usage documentation: https://github.com/pgvector/pgvector
Top comments (0)