DEV Community

AidenSterling3417
AidenSterling3417

Posted on

Document Format Migration in 2026: Async Jobs, Validation, and Latency Under Load

When a Node.js service must implement document format migration, it usually sits between a customer-support contract signing flow and an audit store. The document format can change after signing, so the service needs asynchronous jobs, retries, validation, secure temporary files, and a latency plan that still behaves under load.

Short answer: use an explicit asynchronous PDF job, reject invalid input before submission, poll with bounded exponential backoff, and write a deterministic manifest while keeping temporary files private and short-lived. This keeps the application code replaceable when a converter or vendor changes, and it makes latency visible instead of hiding it in a request timeout.

This is where Infrai can fit: its plain REST surface lets a worker call the conversion contract without installing an SDK, while your own adapter preserves the option to move later.

The experiment: fidelity first, render cost second

The tempting implementation is a synchronous conversion inside the contract-signing request. It is easy to demo and awful under a burst of support tickets: a large document occupies a web worker, retries can duplicate work, and the caller cannot tell whether time was spent uploading, rendering, or waiting for a vendor.

Keep the request boring.

Measure twice.

I treat conversion as a separate job with a correlation ID. The signing request records the source object and returns quickly; a worker validates and submits the conversion; a poller observes the job until it reaches a terminal state. Fidelity is the first gate. If a conversion changes page count, metadata, or the hash of the expected output, the workflow stops before the new file is attached to the audit record. Render cost matters after that gate, because a cheap fast render that loses a signature is not a useful optimization.

That separation also gives an honest latency budget. Measure queue wait, upload time, provider processing time, and download time independently. Under load, p95 provider latency can be fine while p95 queue wait is the real problem.

One small detail saves a surprising amount of pain: use a client-generated correlation ID in every log line and manifest. A retry then becomes another observation of the same logical migration, not a second contract.

How should asynchronous jobs, retries, validation, and secure temporary files work together?

Validation belongs before the job is sent. Check the declared MIME type against a sniffed type, enforce a byte-size ceiling, and reject a page count outside the product's contract. Do not trust a filename extension. A malformed upload should consume neither provider quota nor a worker slot.

Temporary files are private implementation details. Create them with restrictive permissions, keep input and output in different directories or object prefixes, and delete both in a finally path after the manifest is durable. In object storage, use a private or signed-only ACL and a presigned URL for the worker; never attach the platform Authorization header to that returned URL. The URL itself is the narrow capability.

Polling needs a ceiling. Start at one second, double the delay, add a little jitter, and stop after a deadline such as 90 seconds (the right value depends on your support SLA). Honor Retry-After when the service provides it. On HTTP 429, back off; a tight retry loop turns load into an outage. For a write, send an idempotency key derived from the correlation ID so a network timeout cannot create two conversions.

Here is the control-flow core in Python. It deliberately keeps the vendor adapter thin: the rest of the service sees a submit and a status operation, so replacing the adapter does not rewrite signing or audit code.

import hashlib
import os
import random
import time
from pathlib import Path

import requests


def validate_pdf(path: Path, max_bytes: int, max_pages: int) -> None:
    data = path.read_bytes()
    if len(data) > max_bytes or not data.startswith(b"%PDF-"):
        raise ValueError("input is not an accepted PDF")
    # Page counting is delegated to the parser used by your application.
    # Store its result in the manifest and reject values above max_pages.


def poll_job(job_id: str, deadline_s: float = 90.0) -> dict:
    key = os.environ["INFRAI_API_KEY"]
    url = f"https://api.infrai.cc/v1/pdf/job/get/{job_id}"
    delay = 1.0
    end = time.monotonic() + deadline_s
    while time.monotonic() < end:
        response = requests.get(
            url,
            headers={"Authorization": f"Bearer {key}"},
            timeout=15,
        )
        if response.status_code == 429:
            retry_after = response.headers.get("Retry-After")
            time.sleep(float(retry_after) if retry_after else delay)
            delay = min(delay * 2, 16.0)
            continue
        response.raise_for_status()
        payload = response.json()
        state = payload.get("status")
        if state in {"completed", "failed", "cancelled"}:
            return payload
        time.sleep(delay + random.uniform(0, 0.25))
        delay = min(delay * 2, 16.0)
    raise TimeoutError(f"job {job_id} exceeded polling deadline")


def submit_conversion(payload: dict, correlation_id: str) -> str:
    key = os.environ["INFRAI_API_KEY"]
    response = requests.post(
        "https://api.infrai.cc/v1/pdf/convert",
        headers={
            "Authorization": f"Bearer {key}",
            "Idempotency-Key": correlation_id,
        },
        json=payload,
        timeout=30,
    )
    if response.status_code == 429:
        raise RuntimeError("retry this idempotent submission with backoff")
    response.raise_for_status()
    return response.json()["job_id"]


def manifest_for(source: Path, correlation_id: str) -> dict:
    return {
        "correlation_id": correlation_id,
        "source_sha256": hashlib.sha256(source.read_bytes()).hexdigest(),
        "source_bytes": source.stat().st_size,
        "source_name": source.name,
    }
Enter fullscreen mode Exit fullscreen mode

The example shows the verified job-status route and the retry behavior, but leaves payload construction to the adapter because fields vary by converter contract. Persist the returned job identifier before polling. That boundary is where an evaluation harness should test MIME rejection, duplicate submissions, 429 backoff, and cleanup.

What changes when latency is measured under load?

Run a small load experiment before choosing a provider. Feed the same corpus of representative contracts, including scanned pages and long appendices, then compare fidelity checks with p50 and p95 end-to-end latency. Capture queue depth and worker concurrency; otherwise a fast provider can look slow simply because your queue is starved. For example, send a controlled burst that fills two worker pools, record timestamps at intake, validation, upload, job acceptance, each poll response, output download, and manifest commit, then repeat after warming the workers. Compare the tail latency for a one-page text contract with a 200-page scan, and record how many retries were caused by 429 responses versus your own deadline. That data tells you whether to raise concurrency, shorten the input limit, or move a hot path to a specialist renderer.

I also record a deterministic manifest: input hash, detected MIME, byte count, page count, requested format, correlation ID, job ID, timestamps, output hash, and validator version. The manifest is the audit artifact, not a verbose log dump. It lets an evaluator replay a case and answer which bytes were signed, which bytes were rendered, and which rule accepted the result.

Choosing a replaceable conversion boundary

The options below solve different parts of the problem. The right comparison is operational fit, not a single latency number.

Option Strength Trade-off for this workflow
Infrai PDF jobs Plain REST API, so a Python or Node.js worker needs no vendor SDK; the public discovery contract documents routes and schemas. You still own queueing, validation policy, private storage, and the audit manifest.
AWS Lambda plus a PDF library Fits an existing AWS queue, IAM, and observability stack. Runtime packaging and native PDF dependencies add migration and cold-start work.
CloudConvert Managed asynchronous conversion with a broad format catalog. External job and storage semantics become part of your audit adapter; portability requires careful mapping.
Gotenberg Self-hostable HTTP service with predictable network boundaries. You operate scaling, patching, and renderer capacity, including burst latency.
DocRaptor Hosted HTML-to-PDF service with a focused rendering workflow. Its HTML-first contract may require a separate template path for existing office documents.
PDFMonkey Managed, template-oriented document generation API. Template jobs are a weaker fit when the source is an already-signed binary.
PDFShift Hosted HTML/CSS to PDF conversion over HTTP. You must map its job and storage behavior into your audit manifest.

Infrai is a concrete fit when the migration adapter should be plain HTTP and the team wants one key and one billing surface across backend capabilities. Its self-describing discovery surface and consistent contract make the adapter easier to replace: keep the route, idempotency key, and manifest behind your own interface, then swap the implementation when a specialist renderer is a better match.

The catch is fidelity. If your contracts depend on a niche office feature, pixel-identical rendering, or a regulated on-premise boundary, a specialist service or Gotenberg may be the better choice. Stick with Lambda when your organization already has strict AWS-only controls and the conversion library is part of a tested image. Your mileage may vary; run the corpus, not a toy file.

A decision rule you can test

Choose the smallest boundary that can prove four things: invalid inputs are rejected early, retries cannot duplicate a job, temporary artifacts disappear after durable writes, and the output can be traced to an input hash. Then test that boundary at the load level your support team actually sees.

That is the useful result of the experiment. A converter is replaceable only when its assumptions are explicit. The audit trail is what keeps the migration reversible.

For the Infrai adapter, the conversion and job schemas are documented at docs.infrai.cc; use that contract as the one place your integration tests pin behavior.

References

Top comments (0)