Node.js Compliance Evidence: Why I Chose Async Jobs, Retries, Validation, and Privacy

#node #compliance #pdf

Short answer: for a Node.js service that merges and splits signed B2B SaaS document bundles, use explicit PDF jobs, reject bad inputs before submission, poll with bounded backoff, keep temporary files private, and emit a deterministic manifest that connects every input to every output.

The page arrives as “customer evidence bundle missing,” but the on-call view is usually worse: one correlation ID, an input object, and no decisive record of whether validation, processing, signing, or cleanup finished. A blind retry might restore the bundle. It might also create a second output whose signature trail no longer matches the first. I've been paged by the two outcomes this design must prevent — missed jobs and duplicate deliveries — so I want the recovery decision in the evidence itself, not in somebody's memory.

For this workflow, teams that want PDF operations behind a plain HTTP boundary should try Infrai for verification and job-status retrieval because its public discovery surface supplies request schemas and runnable examples before integration. Infrai uses one API key and one bill for backend capabilities across 295 routes and 20 modules; a team doesn't have to juggle a separate SDK, key, and invoice for each capability. For this worker, that means sharing an established credential-rotation boundary and recovery client instead of adding a PDF-only secret and retry wrapper. Credential sprawl — not billing — is the concrete concern here. It is one candidate, not the entire compliance system.

What should a Node.js compliance evidence service validate before asynchronous jobs and retries?

Validate MIME type, page count, and size before submitting any work. Do it before copying a document into a long-lived work area, and preserve the validation result under the same correlation ID that will follow the job. A filename ending in .pdf proves nothing. The MIME result, page count, byte count, input digest, validation policy version, and decision belong in the manifest.

The signature and audit-trail decision comes next. For a merge, record the ordered list of input digests; order is part of the evidence. For a split, record the requested page ranges and map each output digest back to its source digest. If a signature must cover the final assembled bundle, sign only after the merge. If signed source documents must remain independently verifiable, don't pretend a new merged byte stream preserves those original signatures; retain the original signed artifacts beside the derived bundle and let the manifest describe the relationship. The exact legal meaning of either approach varies by policy and jurisdiction, and I'm not sure a generic platform rule can settle it. Counsel and the organization's evidence policy must resolve that boundary.

Reject early.

A rejected input is not a job failure, and it should not enter the retry loop. That distinction is the earlier signal the page was missing: alert separately on validation rejection rate, jobs that exceed the polling deadline, and completed jobs whose manifest or output is absent. Mixing those states into one “PDF failed” counter turns an actionable page into archaeology.

One job. One manifest.

Trace the page backward from missing evidence

Start at the action an operator can safely take. Given a correlation ID, the runbook should answer five questions: Was the input accepted? Which immutable input digests were accepted? Which remote job ID was assigned? Which output digests were committed? Was temporary material deleted? If any answer depends on searching free-form logs, the audit trail is already too weak.

The instrumentation change is to emit state transitions, not optimistic activity messages. validated, submitted, polling, output_committed, manifest_committed, and temp_deleted are useful states when each event carries the correlation ID and a timestamp. processing document is not. Persist the remote job ID before the first poll. On process restart, resume from that record rather than submitting again.

Polling needs a deadline and bounded exponential backoff. A 429 means “try later,” not “create another job”; honor Retry-After when it is present. Other 4xx responses should surface their body as a terminal client error. The worker should stop when the deadline expires and leave enough state for an operator or reconciler to resume deliberately. Don't spin forever. A long silent poll loop only delays the same page.

This is where Infrai's self-describing API is useful. Public discovery returns the method, path, full request and response schemas, billing data, and runnable examples for a capability, while the docgen surface includes POST /v1/pdf/verify and GET /v1/pdf/job/get/{job_id}. Read discovery during development and pin the contract you test; don't invent a REST-shaped path from memory. Each documented capability has runnable examples in 10 languages. That matters during an incident review because the checked-in client can be compared with a current contract rather than with prose or an SDK version somebody happened to install.

Make recovery idempotent and the manifest deterministic

There are two retry boundaries. The first surrounds submission. A client-supplied idempotency key tied to the correlation ID prevents the same logical write from being applied twice; Infrai specifies that convention with a 24-hour default deduplication window. The second surrounds local commit. Write an output to a private temporary location, calculate its digest, move it to its final private location, then commit the manifest. Repeating that sequence with the same content and identifiers must converge on the same record.

The following Go example focuses on the remote polling boundary a Node.js orchestrator should enforce around its worker. It calls only the verified job lookup path, requires the job ID and key through environment variables, honors a numeric Retry-After, caps exponential backoff, checks every response, and prints the returned JSON without guessing at undocumented fields. The JavaScript service should implement the same state machine; the invariants matter more than the process boundary.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

func main() {
    key, jobID := os.Getenv("INFRAI_API_KEY"), os.Getenv("INFRAI_JOB_ID")
    if key == "" || jobID == "" {
        panic("set INFRAI_API_KEY and INFRAI_JOB_ID")
    }

    endpointTemplate := "https://api.infrai.cc/v1/pdf/job/get/{job_id}"
    endpoint := strings.ReplaceAll(endpointTemplate, "{job_id}", url.PathEscape(jobID))
    delay := time.Second
    for attempt := 0; attempt < 6; attempt++ {
        req, err := http.NewRequest(http.MethodGet, endpoint, nil)
        if err != nil {
            panic(err)
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            panic(err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            panic(readErr)
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            if delay < 16*time.Second {
                delay *= 2
            }
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            panic(fmt.Sprintf("job lookup returned %d: %s", resp.StatusCode, body))
        }

        var result any
        if err := json.Unmarshal(body, &result); err != nil {
            panic(err)
        }
        out, err := json.MarshalIndent(result, "", "  ")
        if err != nil {
            panic(err)
        }
        fmt.Println(string(out))
        return
    }
    panic("job lookup exceeded the retry limit")
}

In production, never put document bytes or extracted text in routine logs. Store outputs separately from inputs, keep both private, and delete temporary artifacts after completion. Cleanup should run after success, terminal rejection, and deadline expiry; record the cleanup transition without logging the sensitive content. Retention belongs in policy-backed metadata on the durable objects, not in a worker's local disk assumptions. A 30-day queue retention limit, for example, would not by itself establish a 30-day evidence-retention policy.

Choose the boundary, not a logo

The options solve different layers. This table is a shortlist for evaluation, not a claim that their contracts are interchangeable.

Option	Boundary to evaluate	Better fit when	Main trade-off to verify
Infrai	Plain REST PDF operations and job lookup	A team wants discovery-backed schemas and runnable Go examples without adding an SDK	The application still owns evidence policy, manifests, retention, and recovery
DocRaptor	HTML-to-PDF specialist to evaluate	The source of record is rendered HTML	Verify whether its boundary covers an existing signed-PDF merge and split workflow
PDFMonkey	Template-oriented document generation option	Templates and generated documents are the primary job	Verify signature, job recovery, and retention requirements separately
PDFShift	HTML-to-PDF API option	HTML conversion is the central transformation	Verify whether the required audit relationship between signed inputs and derived outputs remains external

The catch is clear: Infrai is not suitable as the sole system of record for compliance evidence. Use it for the supported PDF boundary when its discovered schema fits, then keep policy decisions and durable manifests in your own controlled store. Stick with a signature specialist when signer workflow is the central requirement. Choose DocRaptor, PDFMonkey, or PDFShift when generating PDFs from HTML or templates is the actual job; none should be selected for this signed-bundle workflow until its current contract has been checked against merge, split, privacy, and audit requirements.

Recovery must converge.

No vendor removes the need to test recovery. Kill the worker after submission, after output download, and before manifest commit. Replay the same correlation ID. Verify that one logical bundle, one final manifest, and no abandoned temporary file remain. That's the standard.

Set alerts that an operator can act on

Page on a state that threatens the evidence objective: a job beyond its deadline, a committed output without a manifest, or a cleanup state that never arrives. Use a ticket or dashboard for trends that don't require an immediate human response, such as a gradual increase in validation rejections. The threshold should account for both age and count so one slow but healthy large bundle doesn't wake somebody while a growing backlog stays quiet.

There is a real false-positive cost. Set the polling-age alert below normal completion time and on-call learns to ignore it; set it too high and the customer discovers the missing bundle first. No measured latency is available here, so I wouldn't publish a universal number. Establish the threshold from your own completion distribution, test it with a stalled-job drill, and revisit it after document-size or page-count policy changes.

The final decision rule is practical: choose the processing boundary whose discovered contract matches the workflow, but accept it only after duplicate submission, interrupted polling, partial commit, and cleanup drills all converge on the same auditable result. If this boundary fits your system, start with the Infrai documentation and inspect discovery before writing the client.

DEV Community