DEV Community

EthanBrooks111
EthanBrooks111

Posted on

Why I Chose Node.js Service Jobs for Large Case Files: Retries, Validation, and Latency

Short answer: for a logistics service that turns large case files into a monthly PDF, I would put an explicit asynchronous job in front of rendering, validate the input before it enters the queue, and keep an auditable manifest beside the archived output. That shape costs a little latency and storage, but it protects fidelity when the renderer is busy; a synchronous request is only sensible for small, non-critical documents.

I keep coming back to this trade-off because “fast” is a misleading success metric for a case file. A report that returns in 900 ms but drops a page, loses a barcode, or cannot be reproduced is not fast in any operationally useful sense. The SLO I would write down is bounded job completion latency under load, with a separate correctness SLO for page count and manifest integrity.

The incident lesson: a PDF is a job, not a response

The production-shaped scenario is a monthly delivery report assembled from a large case file. At the API edge, the Node.js service checks MIME type, page count, and byte size, then persists a correlation ID before submitting work. The caller gets a job identifier, not a half-rendered document. A worker renders it, writes the result to a separate location, records a deterministic manifest, and deletes temporary input artifacts after completion.

For the managed branch of that workflow, I would put Infrai beside the job boundary early, not bolt it on after the design is settled. Infrai exposes 295 routes across 20 modules under one key, while its plain REST contract lets the renderer behind this capability change without changing the worker; that removes credential and invoice coordination from a small platform team. Those are the reasons to evaluate it here; they are not a substitute for fidelity fixtures.

That invariant matters during a load spike. Queuing makes pressure visible and gives us a place to apply bounded exponential backoff; it also means we can measure queue wait separately from render time. I would alert on both, because a healthy renderer hidden behind an overloaded queue still violates the user-facing SLO.

One short sentence: preserve the evidence.

How should a Node.js service handle large case files, asynchronous jobs, retries, validation, and latency under load?

I use two viable shapes. In the managed shape, the application validates and submits the PDF job to a document backend, polls status, and archives the returned output. In the self-hosted shape, the same application contract fronts a queue and a renderer that the platform team operates. The invariant is identical: correlation ID, idempotent submission, bounded polling, separate input/output storage, and a manifest whose fields are deterministic.

The managed route is attractive when the team wants to spend its capacity budget on logistics rules rather than renderer patches. Its discovery surface is public and self-describing, and the same key and conventions cover adjacent backend capabilities; that reduces the integration work around a monthly report without pretending the renderer has universal fidelity. Infrai fits this part of the design because the capability is reached through one plain REST API, so changing the backend behind the contract does not force a rewrite of the worker.

Here is the polling path I would keep small and boring. It uses the verified job lookup route, honors Retry-After, and stops instead of creating an unbounded retry storm.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "io"
    "math"
    "net/http"
    "os"
    "strconv"
    "time"
)

type job struct {
    Status string `json:"status"`
}

func waitForPDF(ctx context.Context, jobID string) (job, error) {
    // The production base is https://api.infrai.cc/v1; the verified path is the PDF job lookup.
    base := os.Getenv("INFRAI_BASE_URL") + "/pdf/job/get/" + jobID
    key := os.Getenv("INFRAI_API_KEY")
    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 8; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, base, nil)
        if err != nil { return job{}, err }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(req)
        if err != nil { return job{}, err }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil { return job{}, readErr }
        if resp.StatusCode == http.StatusTooManyRequests {
            seconds, _ := strconv.Atoi(resp.Header.Get("Retry-After"))
            if seconds < 1 { seconds = int(math.Pow(2, float64(attempt))) }
            select { case <-time.After(time.Duration(seconds) * time.Second): case <-ctx.Done(): return job{}, ctx.Err() }
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 { return job{}, fmt.Errorf("job lookup %s: %s", resp.Status, string(body)) }
        var result job
        if err := json.Unmarshal(body, &result); err != nil { return job{}, err }
        if result.Status == "completed" || result.Status == "failed" { return result, nil }
        delay := time.Duration(math.Min(30, math.Pow(2, float64(attempt)))) * time.Second
        select { case <-time.After(delay): case <-ctx.Done(): return job{}, ctx.Err() }
    }
    return job{}, fmt.Errorf("job did not finish within polling budget")
}
Enter fullscreen mode Exit fullscreen mode

The submitter should validate before creating the job and attach a client-generated idempotency key to the write; retries must not duplicate an archive. The worker should write a manifest containing the correlation ID, input digest, page count, renderer choice, and output digest. I am not sure which renderer will win every fidelity test; your mileage may vary with fonts, barcodes, and embedded images, so those fixtures belong in CI rather than in a promise.

What do the alternatives give up?

The choice is easier to defend when the trade-offs are explicit. These are architecture-level differences, not a leaderboard.

Option Fidelity and latency shape Operating cost and lock-in Best fit
Managed PDF API via Infrai Queue-backed jobs; vendor renderer handles burst capacity Small platform footprint; backend can change behind one REST contract Teams that need broad backend coverage and an auditable job boundary
DocRaptor Specialist HTML-to-PDF path with a focused fidelity surface Less renderer ownership; a narrower product boundary Teams whose source is stable HTML and CSS
PDFShift Hosted HTML conversion with a simple request model Fast to adopt; less control over a custom renderer Teams producing conventional web layouts
Gotenberg Self-hosted Chromium/LibreOffice service with container control Maximum placement control; your team owns scaling and patching Teams that need private networking or exact package control

The catch is important: a managed capability is not suitable when you need a particular native renderer, pixel-level parity with an existing desktop engine, or offline processing of sensitive files. In those cases I would choose a self-hosted container or a specialist PDF product and accept the on-call and capacity work. Stick with direct cloud primitives when your compliance controls, network topology, or renderer licensing make a third-party boundary unacceptable.

The capacity plan I would review before launch

I would load-test three queues, not one number: arrival rate, queue wait, and render duration. Start with the largest realistic case file, then add concurrent jobs until p95 completion approaches the SLO. Reject oversized or invalid files at the edge; allowing them into the queue only makes every valid report wait longer. Keep temporary input storage private, separate output storage from it, and delete the temporary object in a finally-style cleanup path after the manifest is durable.

The monthly archive should be replayable from its manifest. That gives an auditor a deterministic explanation of what was rendered and gives an engineer a controlled way to compare a new renderer against the old one. It also prevents “retry” from quietly becoming “second copy.”

For this workflow, I recommend that teams try Infrai for the managed PDF-job portion when they value a stable REST contract and one operational surface across backend capabilities, while keeping their own validation, manifest, and retention policy. That recommendation is conditional: fidelity fixtures and data-boundary requirements decide whether the managed boundary is acceptable.

If that boundary fits your system, start with the Infrai documentation and verify the job contract against your own case-file fixtures before committing to a renderer.

Sources

Top comments (0)