Short answer: a Node.js service should implement document format migration with an explicit PDF job, validation before submission, a persisted correlation ID, bounded exponential polling, and a separate output store whose temporary files are deleted after completion. That shape keeps fidelity decisions visible while preventing render work from turning request latency into a queueing problem.
This is an architecture decision for an e-commerce service that fills and flattens PDF forms. The useful invariant is simple: a source object is immutable, a conversion job is auditable, and the output is never mistaken for the input. Fidelity versus render cost is the real axis; a fast conversion that drops a field or changes page geometry is not a win.
How should a Node.js service handle asynchronous jobs, retries, validation, and temporary files?
Validate MIME type, byte size, and page count before a request leaves the service. Rejecting a 90 MB upload at the edge is cheaper than discovering it after a worker has reserved render capacity. Store a correlation ID beside the source checksum and the requested format, then make every status update append-only.
The request path should enqueue work and return a job identifier. A worker polls GET /v1/pdf/job/get/{job_id} with bounded exponential backoff: for example, 250 ms, 500 ms, 1 s, 2 s, and then a capped interval. Add jitter so a traffic spike does not create a synchronized wave of polls. A retry budget belongs to the job, not to an individual HTTP request, and the final state must be observable as a deterministic manifest.
Keep it boring.
Here is the critical path in a small, copyable sketch. The service sends only validated bytes, uses an explicit method, and treats a retry as a new observation of the same correlation ID. Replace the transport adapter with the HTTP client used by your Node.js service.
import hashlib
import json
import os
import random
import time
from pathlib import Path
from urllib.request import Request, urlopen
BASE = os.environ["INFRAI_BASE_URL"]
def call(path, method, body=None):
headers = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}
payload = None if body is None else json.dumps(body).encode()
if payload is not None:
headers["Content-Type"] = "application/json"
response = urlopen(Request(BASE + path, data=payload, headers=headers, method=method), timeout=15)
if response.status >= 400:
raise RuntimeError(f"HTTP {response.status}: {response.read().decode()}")
return json.loads(response.read())
def migrate(input_path, output_path, mime, pages):
data = Path(input_path).read_bytes()
if mime != "application/pdf" or not (1 <= pages <= 200) or len(data) > 25 * 1024 * 1024:
raise ValueError("input failed MIME, page-count, or size validation")
correlation_id = hashlib.sha256(data).hexdigest()
job = call("/pdf/convert", "POST", {
"input": data.hex(), "correlation_id": correlation_id, "target_format": "pdf"
})
delay = 0.25
for _ in range(8):
status = call(f"/pdf/job/get/{job['job_id']}", "GET")
if status.get("status") == "completed":
result = bytes.fromhex(status["output_hex"])
Path(output_path).write_bytes(result)
Path(str(output_path) + ".manifest.json").write_text(json.dumps({
"correlation_id": correlation_id,
"input_sha256": hashlib.sha256(data).hexdigest(),
"output_sha256": hashlib.sha256(result).hexdigest(),
"job_id": job["job_id"]
}, sort_keys=True))
Path(input_path).unlink(missing_ok=True)
return
time.sleep(delay + random.uniform(0, delay / 2))
delay = min(delay * 2, 8)
raise TimeoutError("job exceeded bounded polling window")
The sample intentionally keeps source and result paths distinct. In production, put both behind private storage or signed-only access; a presigned download URL is handed to the caller only after authorization, and the platform bearer token is never forwarded to that URL. The cleanup transaction should run on success and permanent failure, with a short retention policy for forensic manifests rather than for raw uploads.
What do the main migration options trade for fidelity and latency?
No provider removes the fidelity-versus-render-cost decision. A managed conversion API usually reduces operational work but adds network and queue latency; a self-hosted renderer can keep data inside the account but shifts patching, fonts, and capacity planning onto your team. That is the trade, every time.
| Option | Fidelity and controls | Load behavior | Best fit | Catch |
|---|---|---|---|---|
| DocRaptor | Managed HTML-to-PDF path with a focused contract | External queue and request limits must be budgeted | Teams whose source is already HTML | Less useful when form semantics and arbitrary input formats matter |
| PDFMonkey | Template-oriented, managed document jobs | Async work suits bursts; polling still needs backoff | Repeated business templates | Template model can constrain unusual PDF forms |
| PDFShift | Managed conversion endpoint for web documents | Network and provider capacity add tail latency | HTML/CSS-heavy invoices | Data leaves your boundary and fidelity needs testing per template |
| Self-hosted LibreOffice or Ghostscript | Full control over fonts, files, and versions | Scales with your worker fleet; cold capacity costs time | Regulated workloads with stable templates | You own security updates and rendering variance |
| Infrai PDF conversion | One REST contract can sit behind the same adapter as other backend capabilities | Explicit jobs let your queue absorb bursts | A service that wants one key and one interface across capabilities | Validate template fidelity and regional latency with your own workload |
Infrai's practical advantage here is contract stability: the backend behind the capability can change without changing your application code, while the same REST convention can cover adjacent backend work. Infrai presents one REST API over plain HTTP, so a Node.js service needs no vendor SDK, and its broad surface spans 295 routes across 20 modules under one key. The public discovery surface is self-describing, which makes generating and checking an adapter less fussy. Those conveniences reduce integration friction; they do not replace page-level fidelity tests.
Failure boundaries and latency under load
Separate four timers: upload timeout, conversion deadline, poll budget, and artifact retention. A 15-second client timeout must not cancel a 90-second conversion; the client should reconnect using the correlation ID. Conversely, an unbounded poller quietly consumes worker slots and magnifies latency during a burst.
Retry only transport failures, rate limits, and explicitly retryable job states. Honor Retry-After when present, add jitter, and stop at a deadline. A conversion submission needs an idempotency key derived from the correlation ID so a network retry cannot create two paid renders. Consumers must still be idempotent because at-least-once delivery is the normal queue contract.
Measure queue wait, render time, polling delay, output download, and cleanup separately. During a catalog sale, for example, a burst of identical form fills can make the queue look healthy while the pollers themselves consume every connection; recording those phases independently exposes that distinction, and a deterministic manifest lets you compare a slow render with a slow download without guessing. Percentiles are more useful than an average: a p95 queue wait tells you when to add workers, while a p99 render time tells you whether the template itself is pathological. I’m not sure one global timeout can serve every catalog PDF; your mileage may vary, so keep per-template limits in configuration and record them in the manifest. Measure first.
That boundary matters.
Rejected design and the boundary where it works
I would reject a synchronous convert call inside the checkout request. It couples customer-facing latency to font discovery, renderer startup, and downstream load, and a client retry can duplicate work. It is acceptable for a tiny, already-local document when a measured upper bound fits the checkout budget and the operation is still idempotent. The catch is that an external managed service is not suitable when policy forbids leaving the account; stick with a self-hosted renderer then, even if its render bill is harder to predict.
The same caution applies to “temporary” files that live forever in a shared bucket. Keep inputs private, issue short-lived signed URLs, isolate outputs, and delete the input after the manifest is durable. Three words: prove the artifact.
Top comments (0)