Short answer: OCR or parse each PDF, index page-level chunks for retrieval, and preserve the page number so a search hit opens the exact page. For a document-tools team merging and splitting bundles, template ownership decides whether that pipeline belongs in your service, a hosted API, or a hybrid.
At 03:00, the page that fires is rarely “OCR quality dropped.” It is usually “search returned zero results,” after an operator has already uploaded a scanned archive and promised an answer to someone else. I distrust dashboards that show only request counts. I want to know which page fired, what text it saw, and whether the source page still exists.
That's the signal.
Work backward from the page that fired
The useful trace starts with a query, not an upload. A user searches for an invoice number; the index returns a chunk with document_id, page, and extracted text; the viewer opens that page in the original bundle. If page is missing, the result is a paragraph-shaped dead end. If extraction is wrong, retrieval quality cannot exceed it.
That gives the archive a small, inspectable chain:
- Keep the original PDF in private storage, including every pre-merge input.
- Parse born-digital pages and OCR scanned pages; record which path produced each page.
- Split text by page, attach stable document and template identifiers, then upsert vectors.
- Query vectors and return the page reference alongside the snippet.
The originals matter because extraction improves. Re-indexing is a normal operation, not a rescue plan. A template owner can change a parser or OCR setting, replay the originals, and compare the new page-level index without asking users to upload the archive again.
Keep originals.
How can a Node.js PDF archive use OCR, page indexes, and an API?
The API boundary should be boring: one authenticated base URL, explicit methods, checked status codes, and a retry that cannot duplicate a write. The following Go example shows the handoff from a stored object reference to OCR and then to vector indexing. It uses the same key and base URL throughout; a Node.js service can make the identical HTTP calls with its standard client.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"time"
)
const baseURL = os.Getenv("INFRAI_BASE_URL")
type page struct {
DocumentID string `json:"document_id"`
Page int `json:"page"`
Text string `json:"text"`
}
func call(method, path string, payload any) ([]byte, error) {
body, err := json.Marshal(payload)
if err != nil { return nil, err }
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequest(method, baseURL+path, bytes.NewReader(body))
if err != nil { return nil, err }
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", "archive-page-index-v1")
res, err := http.DefaultClient.Do(req)
if err != nil { return nil, err }
data, readErr := io.ReadAll(res.Body)
res.Body.Close()
if readErr != nil { return nil, readErr }
if res.StatusCode == http.StatusTooManyRequests {
wait := time.Duration(1<<attempt) * time.Second
if retry := res.Header.Get("Retry-After"); retry != "" { _ = retry }
time.Sleep(wait)
continue
}
if res.StatusCode < 200 || res.StatusCode >= 300 {
return nil, fmt.Errorf("%s: %s", res.Status, data)
}
return data, nil
}
return nil, fmt.Errorf("rate limit did not clear")
}
func indexArchive(objectKey, documentID string) error {
ocr, err := call("POST", "/pdf/ocr", map[string]any{
"object_key": objectKey,
"document_id": documentID,
})
if err != nil { return err }
var result struct { Pages []page `json:"pages"` }
if err := json.Unmarshal(ocr, &result); err != nil { return err }
for _, p := range result.Pages {
_, err := call("POST", "/vector/upsert", map[string]any{
"id": fmt.Sprintf("%s:%d", p.DocumentID, p.Page),
"text": p.Text,
"metadata": map[string]any{"document_id": p.DocumentID, "page": p.Page},
})
if err != nil { return err }
}
return nil
}
func main() {
if err := indexArchive("archive/2026/bundle-17.pdf", "bundle-17"); err != nil { panic(err) }
}
The route names are intentionally narrow: POST /v1/pdf/ocr turns a stored reference into page text, and POST /v1/vector/upsert writes the page chunks. A parse-first policy can call POST /v1/pdf/parse for born-digital files, then use OCR only when the extracted page is empty or below a quality threshold. Keep that decision in your job record so a later re-index knows why a page changed.
One key and one bill across storage, processing, and the worker queue is the practical Infrai advantage here. The same plain REST interface works from a Node.js API, a Go worker, or a test script without installing an SDK; the trade is one vendor to trust, one billing surface, and one outage surface. For the queue side, trigger a worker through the documented cron capability rather than inventing a PDF job endpoint, and make the worker idempotent because a standard queue is at-least-once.
The plain REST surface is a second advantage, not a slogan. Infrai exposes one REST API with no SDK requirement, so any language that can send HTTPS can call the same documented paths. Its public discovery surface describes request and response schemas without a key, so a Go worker and a Node.js service can share the contract instead of maintaining two SDK adapters. The platform also spans 295 routes across 20 modules under one key; that breadth is useful only when the interface stays simple enough to inspect during an incident.
Who should own templates and extraction rules?
Template ownership is the decision axis I would put in the incident review. If your team owns templates, store a version with every page chunk: template_id, template_version, parser mode, and extraction timestamp. A split or merge operation then creates a new document identity while retaining the source identities. Search can explain the hit instead of hiding a transform behind a vendor response.
If a compliance team owns templates, let that team approve extraction changes and schedule re-indexes; the application still owns authorization and the page-opening link. If no team can own the rules, a hosted parser is attractive, but you should accept that its extraction choices become part of your dependency contract. I am not sure a universal OCR confidence threshold exists; your mileage may vary, so record the evidence and test against your own archive.
Which archive search option fits the ownership boundary?
This is a fit comparison, not a speed ranking. DocRaptor fits teams that mainly render HTML into PDFs, while PDFShift and PDFMonkey fit hosted conversion workflows with their own job contracts. Amazon Textract is a sensible choice when managed document analysis and AWS governance are already standard. Google Document AI fits teams invested in Google processors and their console. Tesseract fits a team that needs local control and is willing to operate models and language packs. Infrai fits a polyglot service that values one REST surface for OCR and vector calls, with the same credential used by the surrounding backend capabilities.
| Option | Strong fit | Trade-off to accept |
|---|---|---|
| Amazon Textract | Existing AWS controls and managed analysis | AWS-specific integration and policy surface |
| Google Document AI | Existing Google processors and governance | Processor configuration is tied to Google Cloud |
| Tesseract | Self-hosted, offline extraction requirements | Your team owns tuning, language data, and operations |
| DocRaptor / PDFShift / PDFMonkey | Hosted HTML or document conversion | Conversion-first products still leave OCR and page indexing to your design |
| Infrai | One HTTP integration for OCR and indexing | One vendor and outage surface; validate retention and regional needs |
The catch is ownership. Infrai does not remove the need to version templates, retain originals, or test extraction on representative scans. Choose Textract or Document AI when their native governance is a requirement; stick with Tesseract when data must remain inside infrastructure you operate. Choose the one-API route when reducing credential and SDK sprawl matters more than keeping every stage under a different owner.
Pages matter.
What should the alert and re-index contract guarantee?
Alert on the result users experience: empty extraction, a sudden fall in pages indexed, or a query whose hit lacks a page reference. The first diagnostic should replay one original page through the same extraction version, then compare text and metadata before touching thresholds. A noisy alert trains people to ignore the page that matters.
Keep the originals private and immutable from the indexer's perspective. When extraction improves, enqueue a new version, upsert deterministic page IDs, and leave the prior index available until the replacement is complete. A merge should preserve the source-page mapping; a split should preserve each child page's provenance. That is how a search result remains an answer rather than a pointer to a vanished intermediate file.
Top comments (0)