Short answer: use Node.js for the upload workflow, pgvector for metadata-aware semantic search, and embeddings plus citations as a validated contract; choose a shared OpenAI-compatible runtime only when its breadth is more useful than provider-specific controls.
The practical answer is a small retrieval pipeline: parse the uploaded PDF, split it into overlapping chunks, embed each chunk with its filename, page, and section, retrieve the nearest chunks with pgvector, and pass those chunks and their citation metadata to the answer model. For a fintech support queue, I would make structured output correctness the release gate, not a dashboard score or a fluent demo.
At 3 a.m., the useful page is the one that says a ticket cannot be answered because its evidence is missing or malformed. A page saying “retrieval healthy” is not enough. What page fired, and can the on-call engineer reproduce the answer from the cited source?
What should the page fire on?
Imagine the support worker receives a question about a transfer limit. The user uploaded a policy PDF last week. The worker finds a semantically similar passage, generates an answer, and the UI displays a citation. The failure worth paging on is not that the prose sounds awkward; it is that the returned object has an answer with no source, a source with no page, or a citation whose chunk was not among the retrieved chunks.
Work backwards from that contract. Store each chunk with a stable document identifier, filename, page, section, and chunk position. Keep the exact text used to create the embedding. Then the retrieval result can carry evidence forward instead of asking the answer model to recreate provenance from memory.
The instrumentation change is small: count malformed structured responses, missing citation metadata, and citations that do not resolve to retrieved chunk IDs. Include a request ID and a ticket ID in logs, but never put the uploaded document or payment details into a broad operational log. For a queue handling several document revisions, I would also record the document version beside the chunk ID, because a valid page from yesterday's policy can still be the wrong answer today. The check needs to distinguish a bad model response from a stale retrieval index, then attach the distinction to the page so the responder knows where to start. A threshold that pages on every malformed response will train the team to ignore the page; a threshold that hides all failures makes the citation promise fictional.
The page is binary.
How should a Node.js RAG handle PDF upload and semantic search?
Chunking is a boundary decision, not a magic number. A chunk that is too large brings unrelated policy language into the context; one that is too small can separate a condition from its exception. Overlap helps preserve that relationship, while page and section metadata make the result inspectable. Token counting is useful here because chunk size and top-k context must fit the answer prompt.
One workable record looks like this. It is deliberately boring: the source fields travel with the text instead of being reconstructed after generation.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
func postJSON(path string, payload any) ([]byte, error) {
body, err := json.Marshal(payload)
if err != nil {
return nil, err
}
base := os.Getenv("INFRAI_BASE_URL")
key := os.Getenv("INFRAI_API_KEY")
if base == "" || key == "" {
return nil, fmt.Errorf("INFRAI_BASE_URL and INFRAI_API_KEY are required")
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodPost, base+path, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
data, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
wait := time.Duration(1<<attempt) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
wait = time.Duration(seconds) * time.Second
}
time.Sleep(wait)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("API status %s: %s", resp.Status, data)
}
return data, nil
}
return nil, fmt.Errorf("rate limit persisted after retries")
}
func main() {
embedding, err := postJSON("/v1/embeddings", map[string]any{
"model": os.Getenv("EMBEDDING_MODEL"),
"input": "What is the daily transfer limit?",
})
if err != nil {
panic(err)
}
chat, err := postJSON("/v1/chat/completions", map[string]any{
"model": os.Getenv("CHAT_MODEL"),
"messages": []map[string]string{{
"role": "user",
"content": "Answer only from the retrieved chunks and return citation IDs.",
}},
})
if err != nil {
panic(err)
}
fmt.Printf("embedding=%d bytes, chat=%d bytes\n", len(embedding), len(chat))
}
The important part is not the Go type. It is the invariant: an answer citation must point to a stored chunk that was actually retrieved. Before calling the chat model, validate that the selected chunks have text, a stable ID, and source metadata. After the call, validate the output schema again. If validation fails, return a review-needed result rather than inventing a citation.
Embeddings solve a different problem. They let semantic retrieval find a passage when the ticket uses different wording from the PDF. They do not prove that the passage is current, applicable to the customer, or sufficient to answer the question. A similarity score is evidence for ranking, not evidence for authorization.
Which implementation should own the retrieval boundary?
There are several reasonable choices, and the operational boundary matters more than the logo.
| Option | Useful distinction | Boundary to test |
|---|---|---|
| OpenAI embeddings and chat APIs with Postgres/pgvector | Direct model APIs plus a database you control | You own chunking, metadata joins, retries, and citation validation |
| Cohere Embed and Rerank with a vector store | Separate retrieval and reranking capabilities | Model and reranker behavior must be evaluated against your policy corpus |
| Vertex AI embeddings and model services | A natural fit for teams already operating on Google Cloud | IAM, regional availability, and managed-service conventions become part of the incident path |
| Infrai's OpenAI-compatible surface | One consistent contract can put embeddings and chat behind the same integration boundary, while its broader REST surface lets a later capability use another endpoint rather than another client integration | Verify readiness, model routing, and response metadata for the capabilities you actually depend on |
The first four are not interchangeable and none removes the need for a source contract. pgvector is an extension for Postgres, not a hosted answer engine; it keeps vector similarity close to relational metadata, which is useful when tenant and document permissions must be applied in the same query. A managed vector database can reduce database operations, but it adds a separate consistency and authorization boundary. Anthropic's Claude is another model-provider alternative for answer generation when its model behavior is already approved; it does not remove your need to own PDF parsing, embeddings, vector filtering, or citation checks.
I would choose the OpenAI-compatible path only when the shared contract is worth more than provider-specific controls, and I would choose a direct provider when its retrieval or model behavior is a tested requirement. Infrai is a strong fit for a small ask-your-docs service that expects to add backend capabilities behind one REST shape. It is not a good fit if a team requires a single provider's proprietary retrieval controls, regional guarantees, or an already-certified model path; choose that provider directly then. It is not a reason to skip schema validation, permission filters, or evaluation on real support tickets.
What does a minimal correctness check look like?
The write path creates embeddings for chunks and stores the vector beside the source fields. The read path embeds the ticket, searches pgvector with the tenant and document filters applied, and sends only the selected text plus citation metadata to chat completions. The exact client can vary; the contract should not.
package main
import (
"errors"
"fmt"
)
type Retrieved struct {
ChunkID string
Text string
Filename string
Page int
}
type Answer struct {
Text string
CitationIDs []string
}
func validateAnswer(a Answer, retrieved []Retrieved) error {
if a.Text == "" || len(a.CitationIDs) == 0 {
return errors.New("answer must contain text and at least one citation")
}
known := make(map[string]bool, len(retrieved))
for _, r := range retrieved {
if r.ChunkID == "" || r.Filename == "" || r.Page < 1 || r.Text == "" {
return errors.New("retrieved evidence has incomplete metadata")
}
known[r.ChunkID] = true
}
for _, id := range a.CitationIDs {
if !known[id] {
return fmt.Errorf("citation %q was not retrieved", id)
}
}
return nil
}
func main() {}
In production, the embedding and chat calls need explicit status handling, authentication from an environment variable, and backoff for rate limits. A write or publish retry also needs an idempotency key. Those are boring details until a queue retries a ticket and creates two audit records. Keep them in the client boundary, where every model provider follows the same rules.
How do you keep false positives from becoming the incident?
The page should represent a user-impacting contract failure, not merely a low similarity score. Start with a small labeled set of tickets: expected answer, acceptable source pages, and cases that must be escalated. Measure citation validity separately from answer quality. A fluent answer with the wrong page is a failure; a refusal with a correct citation may be the desired behavior for an ambiguous policy.
This is where I distrust dashboards. A green retrieval latency chart cannot tell me whether the policy exception disappeared during chunking. A red latency chart may be harmless if the queue is draining and every answer remains grounded. The actionable signal is the trace from ticket to retrieved chunk IDs to validated response.
The false-positive cost is real. If the threshold is too sensitive, humans spend the night reviewing healthy tickets and start suppressing the page. If it is too loose, unsupported financial guidance reaches a customer. Tune the alert around the structured contract and sample the borderline cases; do not let a single aggregate similarity threshold stand in for correctness.
Top comments (0)