Short answer: In a Node.js service, poll a durable PDF job status record with capped jittered backoff, a per-request timeout, and one absolute deadline; archive only a validated result.
The least complex design that survives a busy month-end is a durable job record with one absolute deadline. A caller checks that record with capped, jittered waits; a worker renders once per idempotency key; the archive accepts a file only after byte-level and PDF-structure checks. Fidelity gets the budget it needs, while every other activity has a visible limit.
For a logistics operation, that means one report for each depot and accounting period. The PDF is an archival artifact, not a progress animation. I treat a dashboard preview and the retained document as separate quality tiers because paying archival render cost for every preview is a poor trade.
Where does the archive budget actually go?
Status requests are rarely the dominant term. Suppose 120 depots each produce one monthly report. Five render attempts per depot create 600 renderer runs, along with temporary files, validation, and queue occupancy. A client making ten small status reads per job is a much smaller load. The expensive change is usually an unnecessary rerender, not a missed polling interval.
I record render attempts, wall time, output bytes, source-data revision, and retention expiry in the job row. Those fields make a cost review concrete. They also expose a template that produces a 40 MB file when a 4 MB file was expected. The rule I use is blunt: retain the final PDF and a manifest; remove page images and failed outputs after the incident window. The cost is forensic convenience. Six months later, a disputed total may require regeneration from source data rather than opening an intermediate image set.
That retention policy is a limitation, not a promise of perfect replay. If source data can change without a revision lock, regeneration may not reproduce the archived page.
PDF validity belongs before retention. ISO 32000-2 describes the format; a parser or conformance check can catch a truncated stream or broken cross-reference table while the worker still has the source revision and error context.
How should a Node.js client poll PDF job status with backoff?
There are two clocks. Each HTTP request gets a short transport timeout, perhaps 5 seconds. The whole operation gets a caller deadline, perhaps 90 seconds for an interactive report screen. Backoff grows only while the state is pending, includes jitter, and stops at a cap. A 1, 2, 4, 8, 8 second schedule is predictable; an uncapped multiplier is not.
This Python sketch models the policy that a Node.js client can implement with its usual HTTP library. It distinguishes an explicit failed job from a temporary inability to read status, and it never sleeps beyond the remaining deadline.
import random
import time
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
def poll_until_done(status_url, deadline_s=90.0, request_timeout_s=5.0):
started = time.monotonic()
delay_s = 1.0
while True:
remaining_s = deadline_s - (time.monotonic() - started)
if remaining_s <= 0:
raise TimeoutError("report deadline exceeded")
payload = None
try:
request = Request(status_url, headers={"Accept": "application/json"})
with urlopen(request, timeout=min(request_timeout_s, remaining_s)) as response:
payload = response.read()
except (HTTPError, URLError, TimeoutError):
pass
if payload is not None:
state = parse_status(payload)
if state["state"] == "completed":
return state["download_url"]
if state["state"] == "failed":
raise RuntimeError(state.get("error", "render failed"))
wait_s = min(delay_s, remaining_s)
time.sleep(random.uniform(wait_s * 0.5, wait_s))
delay_s = min(delay_s * 2.0, 8.0)
The status contract needs only a stable identifier and states such as queued, rendering, completed, and failed. An accepted submission can return 202; a status read can return 200. Include a retry hint when the service knows a better interval, and use an entity validator such as ETag when the response grows beyond a few fields. Do not invent percentage progress unless the renderer can defend its meaning.
Keep the response boring.
Which boundary owns a retry?
The client owns patience, the queue owns leases, and the renderer owns execution time. Keeping those boundaries separate prevents a familiar failure: the browser gives up at 90 seconds, submits again, and two workers render the same depot. Derive an idempotency key from depot, period, and source revision, then enforce uniqueness in durable storage. A repeated submission returns the existing job. During a month-end surge, a worker can lose its lease while the rendering process continues consuming CPU; the queue then hands the same key to another worker. The uniqueness constraint prevents a second archive object, but it cannot reclaim wasted CPU. That is why the renderer deadline must be shorter than the lease, and why lease renewal should be tied to observed progress rather than a blind timer. These are separate controls with separate alarms.
Boundaries matter.
Classify before retrying. A dependency timeout or a temporary capacity response can consume one of a bounded number of attempts. Invalid input, denied access, and a PDF that fails validation should become terminal failures with an operator-readable reason. Retrying a malformed template five times multiplies the bill and hides the defect.
Express handlers should enqueue and return; they should not hold a request open for a multi-minute render. A status endpoint is easier to operate when it reports state and timestamps, while logs correlate with the opaque job ID rather than shipment names or personal data.
What is worth keeping for an audit?
Store the report period, depot ID, source revision, renderer build, byte length, checksum, validation result, completion time, and deletion time beside the object. Authorize downloads against the depot's reporting scope. Encrypt the archive and keep sensitive shipment fields out of ordinary logs.
Test the transition machine with a fake clock: enqueue, lease expiry, bounded retry, validation failure, completion, and retention deletion. A property test should prove that transient failures never extend the absolute deadline. Operational alerts on oldest queued age, validation failures by template revision, retry rate, and archive growth tell different stories; combine them before paging someone.
The decision rule is simple: allocate fidelity and CPU to the retained document, use a cheaper tier for previews, and stop retaining intermediates once the final checksum and manifest are durable. That deliberate loss of intermediate evidence is acceptable only when source data and renderer version remain reproducible.
Top comments (0)