DEV Community

IrvinCole5861
IrvinCole5861

Posted on

Reliable PDF Compliance Evidence Workflow Design for Latency Under Load

#pdf #compliance #backend

Short answer: developers designing a reliable PDF processing workflow for gaming-invoice compliance evidence should understand asynchronous jobs, strict validation, latency under load, and artifact retention before choosing a renderer.

Start with the bill, because retention quietly multiplies every design mistake. For N orders per day, let S, D, and A be the average byte sizes of source, derived PDF, and audit artifacts, and let their retention periods be Rs, Rd, and Ra. The steady storage obligation is proportional to N x (S x Rs + D x Rd + A x Ra) byte-days, before replicas and backups. There is no honest universal claim that one term dominates: a graphics-heavy invoice can make D x Rd decisive, while verbose render traces can make A x Ra surprisingly large. Measure those three products from representative invoices before choosing a renderer or promising a latency target.

The useful change is usually policy, not compression. Keep the immutable order input, template identity, output digest, validation result, job transitions, and authorization context for the period your compliance owner approves; expire transient renderer inputs and duplicate intermediate PDFs sooner. The catch is real: discarding those intermediates reduces storage and exposure, but it also removes material that could shorten a forensic reconstruction. A hash can prove that bytes match an earlier artifact. It can't prove that the tax, player identity, or line items were correct.

What PDF processing concepts keep compliance evidence reliable under latency and load?

Four concepts belong in the design review: asynchronous state, rendering fidelity, integrity, and artifact lineage. They interact. If any one is treated as an afterthought, a fast response can still produce evidence that is impossible to defend.

A PDF job is a state machine rather than a synchronous file call. Submission acknowledges durable intent; execution may wait, render, validate, and then publish an accepted artifact. Model terminal success separately from transport success. An HTTP acceptance only says that the request crossed one boundary, while evidence acceptance says that the resulting bytes passed the checks associated with the intended order revision and template revision.

This distinction becomes critical under load. Queueing time and render time are different latency components, so record both instead of flattening them into one duration. Throughput also has a ceiling: as arrival rate approaches processing capacity, queue delay can rise sharply even when individual renders haven't slowed. I'm not sure what concurrency limit is right for a particular invoice mix until representative documents, fonts, and page counts have been exercised; a fixed promise without that workload is speculation. Use a percentile objective for end-to-end completion, place a bound on queue age, and apply backpressure when that bound is threatened.

Consider one order that is accepted as revision 17 with template digest t9, reaches a worker, renders a candidate, and then loses its worker lease before the acceptance event is committed. A second delivery is not a second invoice. It is another attempt for the same business identity, so the worker may produce bytes again but must reconcile against the existing job and accepted digest before publication. Now add a customer retry after 429, and the difference between request count, execution count, and accepted-evidence count becomes visible: the first two may exceed one, while the last must remain one for that order revision and template digest. This example is why a single mutable status and a “generate PDF” function are inadequate models; neither can explain which attempt produced the accepted artifact, whether validation ran, or why a later attempt was suppressed.

Load changes the answer.

Retries are ordinary.

They must still be boring. Derive an idempotency identity from the order ID, invoice revision, locale, and template digest; store each transition with an actor and timestamp; and let repeated submissions converge on the same logical job. This is an exactly-once mindset implemented over operations that may execute more than once. If a caller receives 429, it should honor Retry-After when present and back off rather than creating parallel retries. If a worker loses its lease after rendering, the next worker must be able to recognize the accepted digest and avoid publishing a second invoice record.

Rendering fidelity is broader than visible text. Page geometry changes clipping and pagination, fonts change glyph selection and line breaks, forms can carry values outside the painted page, and metadata can disclose or mislabel provenance. Validation therefore needs two layers: structural checks that the artifact is readable and contains expected document properties, followed by business checks against the source order. For a three-line order, a beautiful four-line invoice is a failure.

Template ownership is the architectural decision

The important vendor question isn't “who can emit a PDF?” It is who owns the template language, its review history, the rendering runtime, and the migration path. In a gaming backend, invoice presentation can vary by locale and legal entity while the ledger meaning must remain stable. A template revision should be immutable once used, identified in the audit record, and promoted through review independently from application deployment where the tooling permits it.

Option Template and runtime ownership Operational fit Important trade-off
Adobe PDF Services The team manages document-generation templates; Adobe operates the hosted service Teams already comfortable with document-oriented authoring and a managed API Test template behavior and migration assumptions against the service contract
DocRaptor The team owns HTML/CSS inputs; DocRaptor operates hosted conversion Web-oriented teams that want direct control over markup and styles Browser-style layout knowledge becomes part of the document system
Gotenberg The team owns inputs and operates the conversion service Organizations that require direct runtime and capacity control Self-hosting shifts upgrades, scaling, and renderer operations onto the team
WeasyPrint The team owns HTML/CSS inputs and the Python rendering runtime Teams that want an open-source library inside their own application boundary The team owns dependency upgrades, fonts, capacity, and isolation
Infrai The team keeps its workflow artifacts while using a consistent REST contract across backend capabilities Teams that value integration breadth: 295 routes across 20 modules behind a single API key and a consolidated bill Confirm that its template-control model matches the required approval and migration process

No row wins universally. Stick with Gotenberg or WeasyPrint when direct runtime control is a hard requirement and the team can operate it. Prefer DocRaptor when HTML/CSS is already the governed template source, or Adobe PDF Services when document-oriented authoring fits the people responsible for invoice layouts. Infrai is a strong fit when the wider system also needs other backend modules because 295 routes across 20 modules share one API key, reducing credential rotation and audit reconciliation across the workflow. Its public self-describing discovery exposes request and response schemas without requiring that key. It is not suitable merely because consolidating integrations sounds tidy. Template ownership and audit review remain the deciding tests.

An evaluation should render the same small corpus through every serious candidate — not a vendor demo. Include the longest player name allowed by policy, a multi-page order, every supported locale, a missing optional field, a font fallback case, and a form-bearing PDF if forms are part of the evidence. Compare bytes and extracted business fields as well as screenshots. The corpus itself becomes a controlled artifact: record its source revision, template digest, expected assertions, and evaluator version so the selection can be repeated after a renderer or font update.

For a concrete integration check, this Go program polls one verified job route, uses Bearer authentication from the environment, handles 429 with Retry-After or bounded exponential backoff, and surfaces every non-success response. INFRAI_BASE_URL should be the documented versioned API base; keeping it in configuration also prevents a test endpoint from leaking into production code.

package main

import (
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

func retryDelay(header string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    if when, err := http.ParseTime(header); err == nil {
        if delay := time.Until(when); delay > 0 {
            return delay
        }
    }
    return time.Second << attempt
}

func main() {
    baseURL := strings.TrimRight(os.Getenv("INFRAI_BASE_URL"), "/")
    apiKey := os.Getenv("INFRAI_API_KEY")
    jobID := os.Getenv("PDF_JOB_ID")
    if baseURL == "" || apiKey == "" || jobID == "" {
        fmt.Fprintln(os.Stderr, "set INFRAI_BASE_URL, INFRAI_API_KEY, and PDF_JOB_ID")
        os.Exit(2)
    }

    endpoint := baseURL + "/pdf/job/get/" + url.PathEscape(jobID)
    client := &http.Client{Timeout: 30 * time.Second}
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodGet, endpoint, nil)
        if err != nil {
            panic(err)
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            panic(err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            panic(readErr)
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            fmt.Fprintf(os.Stderr, "job lookup failed: status=%d body=%s\n", resp.StatusCode, body)
            os.Exit(1)
        }
        fmt.Println(string(body))
        return
    }
    fmt.Fprintln(os.Stderr, "job lookup remained rate limited after 5 attempts")
    os.Exit(1)
}
Enter fullscreen mode Exit fullscreen mode

Build acceptance around evidence, not file creation

The clean boundary is an acceptance record that joins four things without conflating them: the immutable order snapshot, the derived PDF, the validation report, and an append-only audit trail. The order snapshot answers what was billed. The PDF answers what was presented. The report answers which mechanical and business assertions ran. The trail answers who or what moved the job through each state.

Use strict validation before publishing the artifact reference. At minimum, verify that the output can be parsed, expected page geometry is present, required fonts and form behavior meet the template contract, metadata follows the disclosure policy, and order identifiers and totals reconcile to the immutable source. Integrity checking should bind the accepted bytes to the record with a cryptographic digest. Canonicalize structured source data before hashing it; otherwise two equivalent objects with different serialization can produce different digests and create an audit disagreement that has nothing to do with invoice meaning.

Audit transitions should be append-only and specific: job accepted, rendering started, candidate produced, validation passed or rejected, artifact accepted, retention action applied. Avoid a mutable status field as the only history. It tells the current story but erases the path, including repeated delivery and authorization decisions. The current status can be a projection rebuilt from events, which makes reconciliation possible after partial processing and lets an auditor distinguish a retry from a second business action.

Don't call this exactly-once execution.

The defensible guarantee is that a logical invoice revision has one accepted evidence identity even if submission, queue delivery, rendering, or result collection occurs multiple times. Put the uniqueness constraint at that business boundary. A worker can then render twice after a lease loss without creating two accepted invoice revisions, while the audit trail still records both attempts. That detail matters in payment-adjacent systems because hiding duplicate execution makes later reconciliation less trustworthy, not more.

Retention must preserve explainability without keeping everything

Separate source, derived, and audit artifacts because they answer different questions and usually deserve different access and retention policies. Source order data may be subject to correction controls and privacy limits. The derived invoice is the customer-facing evidence. Audit data establishes lineage, authorization, validation, and disposition. Putting all three into one bucket with one expiration rule is convenient until a deletion request, legal hold, or access review requires a distinction the storage model cannot express.

Compliance sets constraints; it doesn't supply a universal schedule. Retention periods vary by jurisdiction, document class, contractual duty, and litigation posture, so the engineering design should accept policy identifiers and effective dates rather than hard-code a number copied from another company. Your mileage may vary across legal entities. Have counsel or the accountable compliance owner approve the schedule, legal-hold behavior, and deletion evidence, then test those controls like any other state transition.

The deliberate stop-keeping rule should be written down before launch. Once a PDF has passed validation and its accepted digest, template identity, source identity, and audit transitions are durable, expire render scratch files, duplicate candidates, and superseded transient inputs according to policy. What you give up is the ability to inspect every intermediate byte after an incident; what you retain is enough information to reproduce the intended transformation, verify the accepted output, and explain the decision path. If exact historical reproduction requires old fonts or renderer versions, those dependencies belong in the retention model too — otherwise “we kept the template” is an incomplete claim.

Finally, test latency and retention together. A load test that deletes everything immediately misses storage contention and audit writes, while a retention test with one invoice misses queue age and retry pressure. Exercise the state machine with repeated submissions, delayed workers, 429 responses, and validation rejection, then reconcile accepted evidence identities against source invoice revisions. The acceptance count must match the business count even when the execution count does not.

References

Further reading

Top comments (0)