Short answer: implement branded document delivery as an asynchronous Node.js job with idempotent retries, strict validation, private temporary files, and a queue-age budget that protects latency under load.
For an e-commerce warehouse, the reliable way to deliver searchable text from scanned documents is to accept quickly, process asynchronously, and spend the latency budget on bounded batches. A Node.js API should create a durable job, while workers handle OCR, validation, and publication behind explicit queue-age limits. Secure temporary files and idempotent retries keep a slow afternoon from turning into duplicate invoices.
The first signal to watch is not average OCR time. It is the age of the oldest accepted scan.
Watch that number.
Set the throughput contract before tuning OCR
Write down the contract in terms an operator can page on: maximum accepted upload size, target pages per minute, oldest-job threshold, and the freshness promised to the search index. A request that returns in 40 ms can still represent a document waiting 40 seconds, so one HTTP latency percentile is an incomplete health check.
For a returns pipeline, split interactive traffic from warehouse batches. Reserve worker capacity for the interactive lane and let the batch lane consume the remainder. Measure work in pages or megapixels, not queue messages; a 600-dpi customs scan can consume more memory than dozens of small receipts. Admission control should stop accepting new batch work when the oldest-job threshold is crossed and return a stable job identifier instead of holding a connection open.
I once started with a worker-count increase because the API dashboard looked green. The queue-age panel showed the real problem: preprocessing had saturated CPU while OCR duration stayed flat. The fix was a separate conversion pool and a page-weighted queue. The incident took longer than it should have because our alert measured request latency, not waiting work; by the time a warehouse lead reported missing search results, several hundred scans were already behind the same hot partition. We drained that partition, replayed the untouched jobs, and changed the alert to page on oldest age. That is a useful runbook distinction: queue age points to capacity or admission, while processing time points to the worker path.
Keep four timestamps on every job: accepted_at, started_at, ocr_finished_at, and published_at. Their differences expose queue wait, compute, and validation/storage time independently. Alert on the oldest job and on the rate at which the budget is being consumed, not only on an end-to-end average.
How should a Node.js service coordinate asynchronous jobs, retries, and validation?
Use a durable state machine such as queued, preparing, ocr, validating, stored, and ready. rejected and failed are terminal states with a reason code. Persist an idempotency key scoped to the merchant and input checksum. A repeated upload can then return the original job ID; a filename cannot, because many suppliers name every file invoice.pdf.
Keep the state boring.
Every transition must be safe to repeat. A worker can write searchable text and die before acknowledging the queue. The replacement worker should find the checksum and publish the same logical artifact, not create a second result. Retry only transient conditions such as a connection reset, rate limit, or temporary storage timeout. Bad headers, unsupported encryption, and schema rejection need an actionable terminal response, not five more attempts. Exponential backoff with jitter prevents a storage hiccup from becoming a synchronized retry storm. Exhausted jobs belong in a dead-letter stream with a redacted reason and correlation ID.
The HTTP layer can stay small while CPU-heavy image conversion runs in a worker process. This Go sketch shows the transaction boundary I expect to see in a runbook:
type Job struct {
ID string
InputHash string
Attempt int
}
func handle(ctx context.Context, j Job) error {
if alreadyPublished(ctx, j.InputHash) {
return nil
}
if err := validateInput(ctx, j); err != nil {
return permanent("input", err)
}
path, err := stagePrivate(ctx, j)
if err != nil {
return transient("staging", err)
}
defer removeAfter(path, 20*time.Minute)
text, err := runOCR(ctx, path)
if err != nil {
return classify(err)
}
if err := validateOutput(text); err != nil {
return permanent("output", err)
}
if err := publishOnce(ctx, j.InputHash, text); err != nil {
return transient("store", err)
}
return markReady(ctx, j.ID)
}
publishOnce and markReady are separate on purpose. Searchable bytes and their checksum must be durable before the status says ready. Queue-level delivery guarantees do not replace that application-level boundary. The Node.js producer should return 202 Accepted with the job ID and a status location; clients poll with backoff or subscribe to a documented completion event.
Keep temporary files private, bounded, and boring
Stage uploads below a directory the public web server cannot read. Generate random names, set restrictive permissions, enforce per-job byte limits, and cap each merchant's aggregate temporary bytes. Never put an order number in a path or in a log line. A deferred delete handles the normal path; a sweeper based on modification time handles worker crashes and host reboots.
The sweeper needs its own budget. Delete oldest eligible files in small batches, and alert on both file count and total bytes. A full volume is a capacity incident, not a reason to silently discard a queued document. Preserve the job's checksum and state so an operator can replay it after capacity is restored.
Browser previews have a different lifecycle. A Blob is an immutable byte container; it does not grant authorization or enforce expiry. The server must check the merchant and job state when a short-lived download URL is redeemed. Five minutes might fit a one-page invoice. Your mileage may vary for a 200-page customs packet on a slow connection, so set expiry from observed transfer time and revoke access when the job is withdrawn.
Verify load behavior and define a rollback
Load tests should use the real distribution of page counts, image dimensions, and branded templates. Inject a worker exit after OCR, a publish timeout, duplicate enqueue requests, a full staging volume, and a revoked merchant. The invariant is simple: one logical text artifact, one checksum, and one stable status after replay.
Instrument queue age, oldest job, attempt number, OCR duration, validation rejection rate, dead-letter count, and temporary bytes. Trace spans carry the job ID, never document contents. Keep the parser and preprocessing recipe versioned. During a rollout, retain the previous version until old jobs drain; if rejection rises sharply, pause the new consumer, preserve authorized source scans, and replay only after the cause is understood.
This pattern is not suitable when checkout must display OCR text inside a measured sub-second budget and the document cannot be preprocessed. Precompute common forms or return an acknowledgment with later enrichment. Stick with a synchronous path when the payload is tiny and deterministic. The queue earns its operational cost when burst tolerance, auditability, and controlled retries matter more than immediate pixels.
Top comments (0)