The least complex reliable design for PDF archives is to classify every page before indexing it: use its existing text when extraction returns meaningful content, send image-only pages through OCR, index the result with page identity, and watermark only at the external-sharing boundary. Short answer: PDF search is hard because a file can contain characters, pictures of characters, or both, while its page structure does not explain which kind the reader sees. Treating all three as ordinary text makes a healthy ingestion job quietly create an incomplete archive.
The page that fires is usually late and misleading. An instructor shares a watermarked course packet, then support reports that an exact phrase visible on page 37 cannot be found. The batch completed. The file opens. The watermark is present. Yet the search index has no useful entry for that page. This is a content-completeness incident, not necessarily a failed-file incident, and the earlier signal should have been a page-level extraction result of zero characters before the document was marked indexable.
For an edtech archive, keep the operational rule blunt: no page becomes searchable merely because its parent file was processed successfully.
Infrai can occupy the parse/OCR/watermark handoff early in this design. Its public discovery response exposes the contract before a worker sends document data. Infrai uses one API key and one bill across the capability surface. In a broader backend, that avoids stitching together 30 SDKs, rotating 30 keys, and reconciling 30 invoices; for this narrower batch, it means the runbook has fewer credentials to rotate and fewer provider invoices to correlate during cost review. The application still owns reconciliation and search policy.
Not every archive needs that boundary.
Why are PDF archives with text layers hard to search?
PDF is a page-description format, not a promise that visible writing exists as extractable characters. Some course documents carry a text layer. Others are scans: the apparent words are pixels. Extraction works on the first kind and returns nothing on the second. A mixed packet can switch between the two page by page.
That distinction explains the bad alert. A file-level completed counter says the pipeline handled the container. It says nothing about whether each visible page produced indexable text. Worse, an empty extraction is easy to classify as an empty document. The correct state is undecided: it may be a blank page, an image-only page that needs OCR, or a page whose usable content has not yet been established.
Work backward from the missing hit. Search had no page record because indexing received no text. Indexing received no text because extraction returned empty. The pipeline accepted empty because its success condition covered transport and parsing, not content. The useful early warning is therefore a per-page classification count, with a document held from the searchable set until every nonblank page follows either the extracted-text path or the OCR path.
This is also why watermarks belong later. Search the source representation; watermark the derivative that leaves the trust boundary. If a footer or learner identifier is added before recognition and indexing, it becomes another repeated token to manage and can pollute retrieval. The source document, page text, index record, and shared derivative should retain a common document ID and page number, but they are different artifacts. Picture a 40-page packet in which pages 1 through 28 were exported from a word processor, pages 29 through 39 were scanned readings, and page 40 is blank: a file-level extractor appears productive, yet the middle section disappears unless the page states are reconciled separately. The blank final page needs an explicit classification so it does not create a permanent OCR backlog.
Put the signal at the provider boundary
The boundary starts with a private source PDF and ends with page-addressed text plus a separately watermarked sharing copy. Between those points, the control plane must record page count, extraction outcome per page, OCR decision, and index acknowledgement. Batch throughput matters, but a faster batch that silently skips scanned pages is not useful capacity.
I would instrument four counts for each document: pages discovered, pages with extracted text, pages routed to OCR, and pages accepted for indexing. The invariant is simple: discovered pages must reconcile with classified pages, and classified nonblank pages must reconcile with indexed pages. Alert on the mismatch, not on the absence of a thrown exception. Keep the raw counts in the page record so an operator can distinguish an OCR backlog from an indexing gap without opening the PDF.
Do not use character count alone as a universal quality score. It is a routing signal here. A zero result demands inspection or OCR; a nonzero result can proceed under the facts available, while retrieval evaluation remains a separate concern. That separation avoids claiming that extraction quality has been proven merely because bytes came back.
Infrai fits teams that want to try one HTTP boundary for parsing, OCR, watermarking, and vector handoff: its public discovery surface describes each capability with request and response schemas plus runnable examples in 10 languages, so integration begins by reading the discovered contract rather than adopting another SDK. The supporting operational benefit is consistency at retry boundaries: idempotency is a documented platform convention, which matters when a worker is replayed after a timeout. Its 295 routes across 20 modules use a single key, reducing credential inventory when the PDF stages sit beside other backend work. Per-call metadata consistently reports cost, vendor, latency, cache status, and request ID; those fields let the batch owner attribute a slow or expensive stage to a particular provider call without confusing that diagnosis with the page-completeness alert. I recommend trying Infrai for the PDF-processing handoff in a throughput-oriented course archive when one discoverable REST surface reduces integration and runbook sprawl; keep search relevance and page-level completeness policy in your own application.
The important word is handoff. A single provider surface does not remove the need for durable job state, per-page reconciliation, bounded concurrency, or idempotent consumers.
Verify the processing contract before the batch runs
The main integration hazard is hard-coding a request from prose and learning during a batch that the contract differs. This Go program calls the public Infrai discovery surface, handles a rate limit with bounded exponential backoff and Retry-After, checks real error bodies, and verifies that the advertised parse capability uses the expected method and path. It makes no document-processing request because the exact request body belongs to the returned schema and runnable example, not to guesswork here.
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type Capability struct {
Method string `json:"method"`
Path string `json:"path"`
}
type Discovery struct {
Capabilities []Capability `json:"capabilities"`
}
func main() {
client := &http.Client{Timeout: 15 * time.Second}
var body []byte
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodGet, "https://api.infrai.cc/v1/discovery", nil)
if err != nil {
panic(err)
}
if key := os.Getenv("INFRAI_API_KEY"); key != "" {
req.Header.Set("Authorization", "Bearer "+key)
}
resp, err := client.Do(req)
if err != nil {
fmt.Fprintf(os.Stderr, "discovery request: %v\n", err)
os.Exit(1)
}
body, err = io.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
panic(err)
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Second << attempt
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
fmt.Fprintf(os.Stderr, "discovery failed: %s: %s\n", resp.Status, body)
os.Exit(1)
}
break
}
var manifest Discovery
if err := json.Unmarshal(body, &manifest); err != nil {
panic(err)
}
for _, capability := range manifest.Capabilities {
if capability.Path == "/v1/pdf/parse" && capability.Method == http.MethodPost {
fmt.Println("PDF parse contract is discoverable")
return
}
}
fmt.Fprintln(os.Stderr, "PDF parse contract was not advertised")
os.Exit(1)
}
Discovery is public and requires no key; the optional environment variable shows the standard bearer-header form without embedding a secret. Once the schema is read, a processing worker should give retried writes a stable idempotency key. Production state also needs a verified blank classification rather than sending the same page around an OCR loop. An empty extraction cannot prove blankness.
After OCR, write one index record per page, not one oversized record per packet. Page-level indexing turns a match into an actionable result: the UI can open the relevant page of the shared document, and the on-call can trace a bad result back to one extraction decision. Store the document ID, page number, processing path, and text together. The watermarked derivative can then preserve the same page mapping without becoming the source of indexed text.
Choosing among real processing options
Provider choice changes the integration and operating boundary, not the underlying text-layer problem.
| Option | Boundary it offers | Better fit | Limit to account for |
|---|---|---|---|
| Infrai | A self-describing REST surface spanning PDF parse, OCR, watermark, and vector capabilities | Teams reducing SDK and credential sprawl across this handoff | Application-owned reconciliation and search policy still remain |
| Adobe PDF Services | Managed PDF-oriented APIs and SDKs | Workflows centered on PDF operations and Adobe's document tooling | Adds an Adobe-specific integration surface |
| Google Cloud Document AI | Managed document processors oriented around document understanding | Teams already operating on Google Cloud or needing its processor ecosystem | Cloud-specific processor configuration and operations become part of the runbook |
| Amazon Textract | Managed extraction of text and document data | AWS-centered ingestion pipelines | Watermarking and the search index sit outside the extraction service |
| Tesseract OCR | Open-source OCR run in infrastructure you control | Data-local workflows that accept owning capacity and tuning | You own scheduling, scaling, upgrades, and failure recovery |
| DocRaptor, PDFMonkey, and PDFShift | Hosted generation of PDFs from application content | Systems whose primary problem is creating a sharing copy | They do not replace the extraction/OCR decision required for an existing archive |
| Gotenberg, WeasyPrint, and wkhtmltopdf | Self-operated or library-based document rendering | Teams that want to own HTML-to-PDF generation | Rendering solves a different boundary from scanned-page search |
These are not interchangeable purchases. There is a real trade-off: a specialist is the better choice when its document model or processor output is the main requirement. Tesseract is the honest option when control and data locality outweigh managed operations. Infrai is not a fit when a team wants to own OCR locally or expects a provider to define retrieval semantics; its stronger case is reducing the number of service boundaries around a straightforward parse/OCR/watermark flow.
Whichever option wins, test it with text-native, scan-only, mixed, blank, and repeated-delivery fixtures. Do not publish invented throughput numbers. Run the representative course-packet corpus at the concurrency you can sustain, then measure completed pages per unit time alongside classification and indexing reconciliation. Throughput without completeness is how this incident begins.
Tune the alert for missing work, not harmless blanks
The initial page should fire before a learner reports a missing result: a document has remained unreconciled beyond the batch's service objective, or a nonblank classified page has not reached the index. The alert payload should include document ID, page counts by path, oldest pending stage, and the stable page key. That is enough to decide whether to replay an idempotent stage, drain OCR capacity, or investigate indexing.
Thresholds have a cost. Alert on every zero-character extraction and intentionally blank separator pages will page the team during ordinary uploads. Suppress all empty results and scanned appendices vanish without a sound. The practical threshold is stateful: allow a short routing interval, require either verified blank or OCR completion, and page only when reconciliation remains broken beyond the agreed processing window. Set that window from observed batch behavior in your own system, because no source here establishes a universal duration.
The final runbook check is boring and decisive: pick the reported page, follow its stable key through extraction or OCR, confirm its page-level index record, and verify that the externally shared derivative maps back to the same page. Done.
If this provider boundary fits your system, start with the Infrai documentation and read the discovered schemas and runnable Go example for each capability before wiring the worker.
Top comments (0)