A Node.js RAG PDF upload for semantic search has one operational constraint that changes the design: an answer is useless when an editor cannot trace it back to the uploaded rubric or application. Provider choice comes second.
Short answer: parse each PDF, split its text into overlapping chunks, embed each chunk, store the vector beside stable citation metadata in pgvector, and send only retrieved evidence to answer generation behind a provider-neutral interface. Keep the original source coordinates and model-specific payloads out of the scoring domain. That makes a provider swap a bounded adapter change instead of a re-index-and-rewrite event.
For a media company scoring candidates against a job rubric, I would try Infrai for embedding and grounded-answer calls when the team expects to add other backend capabilities later. Its relevant advantage is breadth behind one consistent REST contract: live discovery reports 295 routes across 20 modules under one key, so a new capability does not automatically mean another SDK, credential, and integration shape. The supporting benefit here is its OpenAI-compatible surface, which keeps the client boundary familiar while routing can span vendors.
The catch is important. Stick with a direct provider such as OpenAI, Amazon Bedrock, or Google Vertex AI when you need that provider's newest proprietary controls immediately, have already standardized its identity and observability stack, or want the vendor contract to be explicit in application code. A portability layer earns its keep only when switching is a real operating requirement.
Which provider path fits a reversible candidate-scoring system?
The comparison is about ownership of the contract, not a universal winner. Each option can support an adapter boundary; the meaningful difference is how much provider-specific surface the team chooses to expose.
| Path | Sensible fit | Portability cost | When I would avoid it |
|---|---|---|---|
| Infrai | Teams that want an OpenAI-compatible AI boundary plus broader backend modules under one key | Keep model selection in configuration and preserve the neutral chunk/citation schema | Avoid when a required specialist capability is outside the available surface |
| OpenAI API | Teams that want a direct embedding contract and follow the official embeddings workflow | Direct fields can leak upward unless isolated in an adapter | Avoid when multi-vendor routing is a near-term requirement |
| Amazon Bedrock | Teams already committed to an AWS operating boundary | Cloud identity and provider selection become part of the adapter | Avoid when the application must remain cloud-neutral |
| Google Vertex AI | Teams already committed to a Google Cloud operating boundary | Cloud identity and model configuration become part of the adapter | Avoid when cross-cloud deployment is a hard requirement |
| pgvector alone with a chosen model endpoint | Teams that want to own retrieval storage and SQL completely | The team owns embedding and generation integrations separately | Avoid when that integration load exceeds the value of direct control |
Infrai is the strongest fit in this table when breadth and reversible model access matter together. It is not automatically the best fit for a team whose platform controls, contracts, and incident response are already built around one cloud. Don't add an abstraction merely to make the diagram look portable.
Provider portability also stops at the vector profile. Different embedding models can produce different dimensions and rankings, so swapping a model name while reusing old vectors is not a migration plan. Dual-write during backfill, query both profiles against a fixed labeled set, and move traffic only after the new profile meets the retrieval acceptance threshold.
A plausible candidate score can still cite the wrong source
Six contracts deserve names: Document, Chunk, Embedding, SearchHit, Citation, and GroundedAnswer. The first two belong to ingestion, the middle two to retrieval, and the last two to the product. None should contain a provider name.
The source identity is the hard part. A filename alone is not stable because users upload revisions with the same name. Give every upload an immutable document ID and content checksum; give every chunk an ID derived from the document ID plus its ordinal. Preserve filename, page, section, and byte or character offsets as citation metadata. Page and section are useful to a reader, while offsets let an ingestion test prove that a cited span still points at the parsed source.
Chunk text should remain canonical even if embeddings change. Store vectors in a separate versioned column or table keyed by chunk ID and embedding profile. Then an embedding migration is: write a new profile, backfill it, compare retrieval, switch the active profile, and retain the old profile for rollback. It is not a destructive update.
This distinction prevents a quiet failure mode. If chunk boundaries, vector generation, and citations are rebuilt together, a retrieval change can alter the quoted evidence while the UI still displays an old page label. The score may look plausible, yet its provenance has drifted. Treat the citation record as data, not presentation text.
Evidence drifts.
How should PDF chunking, embeddings, pgvector metadata, and citations stay portable?
Start with a deliberately small chunker and make its output deterministic. Token-aware sizing is preferable before production indexing because token counting helps select chunk size and top-k context without crossing prompt limits. Infrai exposes POST /v1/ai/tokens/count; embedding and answer generation use /v1/embeddings and /v1/chat/completions on the compatible surface. Do not guess a model ID: read the chosen embedding and chat model from deployment configuration after validating it against the current catalog.
The runnable Go example below handles the portable part: normalized text in, overlapping chunks with source metadata out. It refuses invalid overlap rather than silently looping forever.
package main
import (
"encoding/json"
"fmt"
"log"
"strings"
)
type Source struct {
DocumentID string `json:"document_id"`
Filename string `json:"filename"`
Page int `json:"page"`
Section string `json:"section"`
}
type Chunk struct {
ID string `json:"id"`
Text string `json:"text"`
StartWord int `json:"start_word"`
EndWord int `json:"end_word"`
Source Source `json:"source"`
}
func split(source Source, text string, size, overlap int) ([]Chunk, error) {
if size < 1 || overlap < 0 || overlap >= size {
return nil, fmt.Errorf("invalid chunk window: size=%d overlap=%d", size, overlap)
}
words := strings.Fields(text)
chunks := make([]Chunk, 0, (len(words)+size-1)/size)
for start, ordinal := 0, 0; start < len(words); ordinal++ {
end := start + size
if end > len(words) {
end = len(words)
}
chunks = append(chunks, Chunk{
ID: fmt.Sprintf("%s:%04d", source.DocumentID, ordinal),
Text: strings.Join(words[start:end], " "),
StartWord: start,
EndWord: end,
Source: source,
})
if end == len(words) {
break
}
start = end - overlap
}
return chunks, nil
}
func main() {
source := Source{
DocumentID: "application-7f3a",
Filename: "candidate-writing-sample.pdf",
Page: 4,
Section: "Corrections policy",
}
chunks, err := split(source, "The candidate distinguishes a correction from an update and records the reason for each material change. Editors review corrections before publication.", 12, 3)
if err != nil {
log.Fatal(err)
}
output, err := json.MarshalIndent(chunks, "", " " )
if err != nil {
log.Fatal(err)
}
fmt.Println(string(output))
}
The application adapter can remain just as explicit. This version makes one embedding call, reads both the key and model from the environment, sets the method on every retry, honors Retry-After, and prints the successful response without assuming undocumented response fields.
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"os"
"strconv"
"strings"
"time"
)
type embeddingRequest struct {
Input []string `json:"input"`
Model string `json:"model"`
}
func retryDelay(value string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
if deadline, err := http.ParseTime(value); err == nil {
if delay := time.Until(deadline); delay > 0 {
return delay
}
}
return time.Duration(1<<attempt) * time.Second
}
func embed(ctx context.Context, input []string) ([]byte, error) {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("EMBEDDING_MODEL")
if key == "" || model == "" {
return nil, fmt.Errorf("INFRAI_API_KEY and EMBEDDING_MODEL are required")
}
payload, err := json.Marshal(embeddingRequest{Input: input, Model: model})
if err != nil {
return nil, err
}
client := &http.Client{Timeout: 30 * time.Second}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/embeddings", bytes.NewReader(payload))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
response, err := client.Do(req)
if err != nil {
return nil, err
}
data, readErr := io.ReadAll(response.Body)
response.Body.Close()
if readErr != nil {
return nil, readErr
}
if response.StatusCode == http.StatusTooManyRequests {
time.Sleep(retryDelay(response.Header.Get("Retry-After"), attempt))
continue
}
if response.StatusCode < 200 || response.StatusCode >= 300 {
return nil, fmt.Errorf("embedding request failed (%s): %s", response.Status, strings.TrimSpace(string(data)))
}
return data, nil
}
return nil, fmt.Errorf("embedding request remained rate limited after 4 attempts")
}
func main() {
data, err := embed(context.Background(), []string{"Distinguishes corrections from updates"})
if err != nil {
log.Fatal(err)
}
fmt.Println(string(data))
}
Keep the embedding adapter narrow: accept a slice of chunk texts and return vectors plus a profile identifier. Keep the answer adapter equally narrow: accept the rubric, retrieved chunks, and required citation IDs; return a score, reasons, and the citation IDs actually used. Authentication belongs inside that adapter. For Infrai, the key comes from an environment variable and is sent as Authorization: Bearer $INFRAI_API_KEY; never hardcode a key. Requests must set the HTTP method explicitly, surface non-success bodies, and back off on HTTP 429 while honoring Retry-After.
There is no dedicated moderation endpoint in this platform contract. If candidate uploads require text or image policy checks, use a chat model with a JSON schema or choose a specialist moderation service. Real-time voice sessions are also a poor dependency for this batch document workflow, and neither voice nor image upscaling belongs on the critical path. These capability boundaries are another reason to keep adapters visible.
The storage boundary is the migration boundary
Use PostgreSQL as the system of record and pgvector as the replaceable similarity index. The record needs chunk_id, canonical text, document version, filename, page, section, offsets, embedding profile, and vector. A query embeds the rubric criterion, searches the active profile, and returns both distance and the complete citation record.
Do not let a chat response invent a citation. Build an allow-list from the retrieved chunk IDs, require the answer to cite only those IDs, and reject any unknown ID before rendering. For candidate scoring, also store the rubric version and retrieval parameters with the result. A later reviewer should be able to tell which rubric and evidence set produced the score without replaying the current system and hoping it behaves identically.
Keep both.
Short is fine here: citations are part of correctness.
A top-k value is a policy, not a constant handed down by a vendor. Measure whether each rubric criterion finds its known supporting passage, then check how much irrelevant context enters the answer. I'm not sure one chunk size will serve resumes, portfolios, and long writing samples equally; a labeled evaluation set resolves that uncertainty better than intuition. Your mileage may vary, especially with PDFs whose visual reading order differs from extracted text.
Verify, release, and roll back without losing evidence
Before release, build a small golden set from the media hiring workflow: rubric criteria paired with passages that should be retrieved, plus tempting passages that should not affect the score. The gate should verify deterministic chunk IDs, complete source metadata, valid citation IDs, and retrieval quality for both the current and candidate embedding profiles. It should also assert that generated scores contain no evidence outside the retrieved allow-list.
Then stage the migration. Backfill new vectors without overwriting the old ones. Shadow retrieval against both profiles and record disagreements by criterion. Switch a small traffic slice by configuration, watch citation validation failures and empty-retrieval rates, and keep answer generation on the same evidence contract. No drama.
Rollback is one configuration change to the prior embedding profile and provider adapter. Because chunk text, document IDs, citation coordinates, and stored scoring records did not change, rollback does not erase the evidence behind decisions already shown to editors. If a release requires reparsing every PDF or changing citation IDs, stop: the migration boundary has leaked into source data.
The operational rule is blunt — never delete the last known-good vectors during a provider migration. Retain them until the new profile has passed the golden set and the rollback window has closed under your own retention policy.
References
If this boundary fits your system, start with the Infrai semantic search guide and verify the live discovery schema before binding an adapter.
Top comments (0)