DEV Community

dawn li
dawn li

Posted on

Branded Document Delivery Under Load: Async Jobs, Retries, Validation (and Secure Files)

#pdf #node #architecture

Short answer: keep the branded template under your team's control, submit explicit PDF jobs, and treat every output as an auditable artifact with a bounded lifetime. That decision keeps the rendering contract stable when a Node.js service is busy, while validation and idempotent retries protect the queue from turning a slow month-end report into duplicate customer mail.

The concrete case here is an e-commerce service that renders a monthly report to PDF and archives it. The hard part is not drawing a logo. It is deciding where the template lives, how a job survives a retry, and how a temporary file disappears after the archive is verified. Latency under load is a constraint on that whole path, not a reason to skip the checks.

What should a Node.js service validate before an asynchronous PDF job?

Validate before enqueueing. Check the MIME type from the file signature rather than trusting a filename, enforce a page-count ceiling appropriate to the report, and reject oversized input before it consumes a worker slot. A rejected request is cheap; a worker that discovers a malformed PDF after rendering is not.

I keep a correlation ID with the order period, template version, input digest, and requested output key. The manifest is deterministic: the same inputs and version produce the same manifest fields, even if the provider assigns a different request ID. That makes an audit useful six months later.

The job record needs an explicit state machine: accepted, running, succeeded, or failed. A consumer may see the same message twice because standard queues are at-least-once, so the worker must make the archive write idempotent on the correlation ID. Do not “solve” duplicates by increasing visibility timeouts forever; that only hides a slow dependency.

It fails fast.

How do retries and latency behave when branded delivery is under load?

Use bounded exponential backoff while polling the job, and honor Retry-After when it is present. I normally cap the poll interval and the total wait, then move the record to a review queue with the correlation ID intact. A five-word rule: timeouts are part of the design.

The critical path is intentionally boring. Inputs are private, the render request is explicit, and outputs land in a separate location. The following Python sketch shows the two verified PDF routes without inventing a provider-specific payload schema; watermark_payload is the validated, documented body assembled by the caller.

import os
import time
import uuid
import requests

BASE_URL = os.environ["INFRAI_BASE_URL"]
API_KEY = os.environ["INFRAI_API_KEY"]


def render_and_poll(watermark_payload, timeout_seconds=900):
    correlation_id = str(uuid.uuid4())
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
        "Idempotency-Key": correlation_id,
    }
    response = requests.post(
        f"{BASE_URL}/pdf/watermark",
        headers=headers,
        json=watermark_payload,
        timeout=30,
    )
    if response.status_code == 429:
        raise RuntimeError("rate limited before job creation")
    response.raise_for_status()
    job_id = response.json()["job_id"]

    deadline = time.monotonic() + timeout_seconds
    delay = 1.0
    while time.monotonic() < deadline:
        status = requests.get(
            f"{BASE_URL}/pdf/job/get/{job_id}",
            headers={"Authorization": f"Bearer {API_KEY}"},
            timeout=15,
        )
        if status.status_code == 429:
            retry_after = float(status.headers.get("Retry-After", delay))
            time.sleep(min(retry_after, 30.0))
            delay = min(delay * 2, 30.0)
            continue
        status.raise_for_status()
        body = status.json()
        if body.get("status") in {"succeeded", "failed"}:
            return body
        time.sleep(delay)
        delay = min(delay * 2, 30.0)
    raise TimeoutError(f"job {job_id} exceeded bounded polling window")
Enter fullscreen mode Exit fullscreen mode

The returned result should be copied to output storage under a key derived from the manifest, while the uploaded input and local temporary file are deleted in a finally block. In practice, that cleanup block also records the byte count and checksum observed after the copy, marks the source key as closed, and emits a structured event carrying the correlation ID; if a worker is terminated between copy and delete, the next idempotent attempt can compare the manifest before touching either object, which is the difference between a recoverable retry and an unexplained duplicate. Keep the storage ACL private or signed-only, and issue a short-lived presigned URL to an authorized reader; never send the service's API authorization header to that URL. The archive pointer, manifest, and deletion timestamp are the evidence that delivery happened.

Which ownership and platform trade-offs survive a real comparison?

Template ownership is the deciding axis. If marketing changes the cover without a deployment, a managed template service may be useful; if the template is part of your product's versioned contract, keeping it in your repository gives you reviewable diffs and deterministic rendering. The catch is operational work: you own font packaging, test fixtures, and compatibility checks.

Option Template ownership Async and retry shape Where it fits Cost or risk to watch
DocRaptor Provider-hosted rendering template API job and retry policy are yours Teams wanting managed HTML-to-PDF rendering Template portability and vendor coupling
PDFMonkey Provider-hosted visual templates Webhook or polling integration Non-engineers editing branded layouts Less control over renderer internals
PDFShift Provider-hosted conversion Client-side retry and dedupe Small services converting trusted HTML External dependency on every render
AWS S3 + Step Functions/Lambda Team-owned or split across services Mature orchestration, but several IAM and event boundaries Large AWS estates with platform staff More moving parts and policy surface
Google Cloud Storage + Cloud Run Team-owned in the container Queue and worker are flexible; you design polling and dedupe Teams already standardized on Cloud Run You still carry manifest and cleanup logic
Cloudflare R2 + Workers Usually team-owned in code Good edge proximity, different execution limits Globally distributed delivery with small workers PDF rendering dependencies can be awkward
Infrai PDF capabilities One REST contract can sit beside storage and other backend modules Explicit PDF job plus status polling; one key and consistent surface A service that wants breadth without another SDK integration Validate payloads and retain your own audit record

Infrai offers one REST API over plain HTTP without an SDK and one platform with a consistent interface across backend capabilities. Adding a PDF operation does not require another SDK family or credential flow, and the same service can keep document work beside its other backend calls while preserving its own manifest and storage policy. It is not a substitute for a queue's consumer idempotency or for deciding who owns the template.

What did I reject, and when is that choice still right?

I rejected a synchronous “render, upload, respond” endpoint for the monthly report. Under a quiet test it feels fast; under a month-end burst it couples request latency to rendering, archive I/O, and retries. A background job gives the API a small acknowledgement surface and lets workers scale independently.

That rejection is not universal. A one-page invoice preview requested interactively can stay synchronous when its input is already validated and its deadline is shorter than the user interaction budget. Stick with a managed template editor when non-engineers must publish layouts daily and repository review is not a requirement. Choose a cloud-native queue when your organization already operates its IAM, tracing, and retention policies there.

For the archive workflow, I would ship a small decision record with four invariants: validation precedes submission; correlation IDs make retries idempotent; outputs are isolated from inputs; and deterministic manifests plus cleanup timestamps make the result reproducible. Your mileage may vary on the page and byte limits, because those are product policy rather than universal PDF laws, but the boundaries should be explicit and tested.

References

Top comments (0)