Short answer: use an explicit PDF job, validate the input before enqueueing it, and keep the output in a separate, short-lived location until the job is complete. That pattern gives a Node.js service predictable retries and an audit trail without making request latency depend on PDF rendering.
I treat report generation as a small pipeline, not a long HTTP request. The request handler checks the report payload and records a correlation ID. A worker submits the PDF job, polls its status with bounded exponential backoff, and writes a deterministic manifest beside the finished file. Temporary input is removed in a finally block. The important choice is fidelity versus render cost: a browser-grade renderer can preserve the report's layout, while a lighter converter usually consumes fewer resources under load.
How should a service handle report generation, retries, and latency under load?
Start with a queue boundary. The API returns 202 Accepted and the correlation ID as soon as validation passes. A queue worker owns the slow part, so a burst of monthly reports produces a bounded queue instead of a burst of browser processes inside your web servers.
Validation belongs before the job is created. Check the declared MIME type against the bytes you received, enforce a maximum size, and reject a document whose page count is outside the product's contract. Do not trust a filename extension. For a monthly edtech report, I would also normalize the input metadata and hash the exact bytes before enqueueing: that hash becomes the manifest's evidence, while the MIME, page-count, and size checks become explicit fields that an auditor can inspect without opening the PDF. Store the validation result with the correlation ID; it explains why a job was accepted months later.
Keep it boring.
Then make the queue observable.
Retries need two safeguards. First, use a client-generated idempotency key for the create call, derived from the correlation ID and a stable report revision. Second, back off and stop. A useful schedule is 1, 2, 4, 8, and 16 seconds with a cap, honoring Retry-After when the service sends one. A 429 should not turn into a tight loop, and a timeout should be recorded as a failed attempt rather than silently retried forever.
The polling loop has the same shape. Poll GET /v1/pdf/job/get/{job_id} at increasing intervals, cap the total wait, and persist the last observed state. The worker can then be restarted without losing where it was. Under load, measure queue wait and render time separately; combining them hides whether you need more workers or a cheaper renderer.
A minimal Python worker with bounded backoff
The following worker keeps the provider-specific path in one small adapter. Set PDF_API_BASE_URL to the PDF service you selected and put its bearer token in INFRAI_API_KEY; no key is embedded in the source. The body shown is the service's report payload in this example, while the surrounding retry and cleanup behavior is the part worth carrying into production.
import hashlib
import json
import os
import time
from pathlib import Path
import requests
BASE_URL = os.environ["PDF_API_BASE_URL"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]
def request(method, path, *, body=None, idempotency_key=None):
headers = {"Authorization": f"Bearer {API_KEY}"}
if idempotency_key:
headers["Idempotency-Key"] = idempotency_key
url = f"{BASE_URL}{path}"
if method == "POST":
response = requests.post(url, headers=headers, json=body, timeout=30)
elif method == "GET":
response = requests.get(url, headers=headers, timeout=30)
else:
raise ValueError(f"unsupported method: {method}")
if response.status_code == 429:
raise RuntimeError("rate limited")
response.raise_for_status()
return response.json()
def render_report(report_bytes, report_revision, temp_dir):
correlation_id = hashlib.sha256(
report_bytes + report_revision.encode("utf-8")
).hexdigest()
input_path = Path(temp_dir) / f"{correlation_id}.pdf"
input_path.write_bytes(report_bytes)
# Validate bytes, MIME policy, size, and page count before this function runs.
manifest = {"correlation_id": correlation_id, "revision": report_revision}
try:
payload = {"correlation_id": correlation_id, "manifest": manifest}
job = request(
"POST",
"/v1/pdf/generate",
body=payload,
idempotency_key=correlation_id,
)
job_id = job["job_id"]
delay = 1.0
deadline = time.monotonic() + 180
while time.monotonic() < deadline:
status = request("GET", f"/v1/pdf/job/get/{job_id}")
state = status["status"]
if state == "completed":
output = Path(temp_dir) / f"{correlation_id}-output.pdf"
output.write_bytes(bytes.fromhex(status["content_hex"]))
output.with_suffix(".manifest.json").write_text(
json.dumps(manifest, sort_keys=True), encoding="utf-8"
)
return output
if state in {"failed", "cancelled"}:
raise RuntimeError(f"PDF job {job_id} ended as {state}")
time.sleep(delay)
delay = min(delay * 2, 16.0)
raise TimeoutError(f"PDF job {job_id} exceeded the polling deadline")
finally:
input_path.unlink(missing_ok=True)
In a real worker, wrap request in a small retry policy that catches the 429 exception, honors the response's Retry-After value, and adds jitter. Keep the idempotency key unchanged across attempts. The output should move to durable storage only after the manifest and bytes have been checked; inputs and outputs must never share an access policy. A private object with a short-lived signed download URL is a safer default than a public URL.
One caveat: the response fields in the adapter are part of your chosen PDF service's contract, so verify them against its schema before shipping. The workflow does not depend on a particular renderer, and your mileage may vary when a report contains fonts or charts that need browser-level fidelity.
What trade-offs matter more than a vendor name?
The decision is usually about where you want operational ownership to live. A local converter is cheap to invoke but makes font packages, sandboxing, and upgrades your problem. A hosted renderer shifts that work away from the application, but adds network latency and a usage bill. A browser worker gives the highest CSS fidelity and the highest memory pressure. None of these is universally right.
| Option | Fidelity and latency profile | Operational cost | Good fit |
|---|---|---|---|
| Puppeteer/Playwright worker | Browser CSS fidelity; cold starts and memory can raise tail latency | You run browsers, fonts, and sandboxes | Pixel-sensitive dashboards |
| wkhtmltopdf or a local converter | Fast for simple HTML; CSS support is narrower | You package and patch a binary | Predictable templates with modest styling |
| DocRaptor-style hosted API | Rendering is off your worker fleet; network time is visible | Vendor quota and egress become dependencies | Teams that want less PDF infrastructure |
| Infrai PDF jobs | One REST contract can sit behind the adapter while the backend vendor changes; the same key and billing surface can cover adjacent backend capabilities | You still own validation, retention, and queue policy | A service standardizing several backend calls behind HTTP |
The last row is useful when replacing the renderer should not force a rewrite of your Node.js service. Infrai exposes one plain REST API, so this adapter can use HTTP directly without installing an SDK. Infrai also puts 295 routes across 20 modules behind one key, which can reduce credential setup for a reporting pipeline that also needs storage or notifications. Its advantage here is the stable REST contract: the thing behind the capability can move while your adapter and audit fields stay put. That is a maintenance benefit, not a promise of lower render latency.
The catch is that a single API does not remove the need for capacity planning. Pick a dedicated browser worker when exact layout is the product, and stick with a local converter when reports are plain and an extra network hop would violate your latency budget. If your team needs deep renderer-specific controls or on-prem execution, a hosted abstraction is not suitable; keep the renderer close to the worker instead.
How do you make outputs reproducible and secure?
Write a manifest for every accepted revision. Include the correlation ID, a hash of the validated input, template version, renderer choice, and completion timestamp. Sort keys before serializing it. That gives support a way to compare two PDFs without guessing which template was deployed.
Keep temporary files in a directory with restrictive permissions, and clean them even when polling fails. Outputs belong in a separate private bucket or filesystem namespace. Expose them through a presigned URL that expires quickly; never forward the PDF service's Authorization header to that URL. Retention should be explicit, with a deletion job for old reports and manifests.
For load testing, generate a fixed mix of small and large reports and watch p50 and p95 queue wait, render duration, retry count, and temporary-disk usage. I initially assumed render time would dominate. It didn't in a queue-backed system: waiting for a free worker was often the clearer signal, which is why those two timers should remain separate. I am not sure your mix will behave the same way, so capture both before changing concurrency.
Operationally, the checklist is short: validate before enqueue, persist the correlation ID, make creation idempotent, poll with a deadline, separate input from output, write the manifest, and delete temporary artifacts. Those controls make a monthly PDF report boring to operate, which is exactly what an edtech reporting path needs.
Top comments (0)