DEV Community

NikitaChristensen2691
NikitaChristensen2691

Posted on

Document Rendering as a Background Job: Page Count Unpredictability and Audit Trails

Render invoice PDFs in a background job, return a job id, and let the caller poll for it. Use inline rendering only where you can bound the page count before the work starts — a one-page receipt off a fixed template, nothing beyond that. Document rendering belongs in a job for one reason: render time scales with content you don't control, so a synchronous endpoint inherits its timeout budget from someone else's document.

That is the whole recommendation. The rest is the evidence, plus the operational details that decide whether the job actually helps.

The system I'll use throughout is a property management billing run: order data in, monthly invoice PDFs out, each one signed, each one landing in an audit trail that an owner or a regulator can ask about three years later. Signature and audit trail are the axis that decides the design here. A render that dies halfway wrecks both of them.

What does page count unpredictability do to a synchronous document render?

Page count is an output of the data, not an input to the request. Three invoices, same template, same code path, roughly the same 4 KB of JSON on the wire:

  • a studio tenant with rent and one utility line: 1 page
  • a commercial tenant across 43 units with per-meter splits and a CAM reconciliation appendix: 60-odd pages
  • an owner's year-end bundle covering 210 properties and twelve monthly statements: several hundred pages, with an embedded photo per unit

Two orders of magnitude. Nothing in the request tells you which one you are about to get, because the fan-out lives in the order data, not in the payload. ISO 32000-2 describes the structure that comes out the other end: a page tree where every page carries its own content stream and its own resource dictionary. That structure explains why render cost tracks the document rather than the request.

Here is the concrete failure mode, and it's the one that pages you at 03:00 rather than the one that shows up in a load test. The proxy in front of your API gives up at 30s. The renderer keeps going — it finishes the 300-page bundle, signs it, writes it to storage, and has no idea anyone stopped listening. The client meanwhile saw a timeout and retried, because that is what clients do. Now there are two renders of the same invoice, two signatures over two documents that are not byte-identical, and two audit rows carrying different document hashes. The accountant receives both. Deciding which one is authoritative is a conversation you cannot win, and it started life as a timeout setting.

Inline rendering is safe in exactly one case: when you can prove the page count before you start. A fixed one-page receipt with no repeating sections qualifies. A monthly statement never does.

Shape the job so an invoice can't get stranded

The job id does two things at once. It makes the request fast — you write the id onto the invoice row and return 202 before any rendering happens — and it makes the work observable, which is the half people skip. An id you don't persist isn't observability. It's a receipt you threw away.

Terminal states matter more than the happy path. A job with a success state and no failure state is not a job; it's a row that sits at processing forever, and nobody notices until an owner calls in March asking where the year-end statement went. Give the state machine an explicit terminal failure, alert on anything that hasn't reached a terminal state in 15 minutes, and make the alert point at the invoice rather than at the job.

Make submission idempotent from the first line of code. The idempotency key comes from data you already have — invoice id plus template version — so a retry after a network blip re-reads the first job instead of starting a second render. That one header is what keeps duplicate deliveries out of the audit trail.

package main

import (
    "bytes"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const (
    generatePath = "/v1/pdf/generate"
    jobGetPath   = "/v1/pdf/job/get/{job_id}"
)

var (
    apiBase = os.Getenv("INFRAI_API_BASE") // API origin, from the provider's API reference
    apiKey  = os.Getenv("INFRAI_API_KEY")  // ifr_... — never a literal in source
)

type renderJob struct {
    JobID  string `json:"job_id"`
    Status string `json:"status"`
    URL    string `json:"url"`
}

// send performs one request, retrying on 429 with exponential backoff and
// honouring Retry-After when the response carries it.
func send(req *http.Request) (*http.Response, error) {
    for attempt := 0; attempt < 5; attempt++ {
        if req.GetBody != nil {
            body, err := req.GetBody()
            if err != nil {
                return nil, err
            }
            req.Body = body
        }
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return resp, nil
        }
        resp.Body.Close()
        wait := time.Duration(1<<attempt) * time.Second
        if after, convErr := strconv.Atoi(resp.Header.Get("Retry-After")); convErr == nil && after > 0 {
            wait = time.Duration(after) * time.Second
        }
        time.Sleep(wait)
    }
    return nil, errors.New("still rate limited after 5 attempts")
}

// submit queues one invoice render and returns the job id.
func submit(invoiceID, templateVersion, html string) (string, error) {
    payload, err := json.Marshal(map[string]string{"html": html})
    if err != nil {
        return "", err
    }
    req, err := http.NewRequest("POST", apiBase+generatePath, bytes.NewReader(payload))
    if err != nil {
        return "", err
    }
    req.Header.Set("Authorization", "Bearer "+apiKey)
    req.Header.Set("Content-Type", "application/json")
    // Derived from data we already have, so a retry re-reads the first job
    // instead of rendering and signing a second copy of the same invoice.
    req.Header.Set("Idempotency-Key", "invoice-"+invoiceID+"-"+templateVersion)

    resp, err := send(req)
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()
    raw, err := io.ReadAll(resp.Body)
    if err != nil {
        return "", err
    }
    if resp.StatusCode >= 400 {
        return "", fmt.Errorf("generate %s: %s", resp.Status, raw)
    }
    var job renderJob
    if err := json.Unmarshal(raw, &job); err != nil {
        return "", err
    }
    return job.JobID, nil
}

// await polls until the job reaches a terminal state or the deadline passes.
func await(jobID string, deadline time.Time) (renderJob, error) {
    url := apiBase + strings.Replace(jobGetPath, "{job_id}", jobID, 1)
    for time.Now().Before(deadline) {
        req, err := http.NewRequest("GET", url, nil)
        if err != nil {
            return renderJob{}, err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)
        resp, err := send(req)
        if err != nil {
            return renderJob{}, err
        }
        raw, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return renderJob{}, readErr
        }
        if resp.StatusCode >= 400 {
            return renderJob{}, fmt.Errorf("job get %s: %s", resp.Status, raw)
        }
        var job renderJob
        if err := json.Unmarshal(raw, &job); err != nil {
            return renderJob{}, err
        }
        if job.Status == "succeeded" || job.Status == "failed" {
            return job, nil
        }
        time.Sleep(2 * time.Second)
    }
    return renderJob{}, errors.New("no terminal state before the deadline")
}

func main() {
    const html = `<h1>Invoice INV-2026-00417</h1><table><tr><td>Rent</td><td>1450.00</td></tr></table>`

    jobID, err := submit("INV-2026-00417", "v7", html)
    if err != nil {
        log.Fatal(err)
    }
    // Persist jobID on the invoice row here, before polling: that row is what
    // the stuck-job alert and the audit trail both read from.
    fmt.Println("queued", jobID)

    job, err := await(jobID, time.Now().Add(10*time.Minute))
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(job.Status, job.URL)
}
Enter fullscreen mode Exit fullscreen mode

The other half of the runbook is the query you run when the billing dashboard looks wrong.

select invoice_id, job_id, status, created_at
from invoice_render
where status not in ('succeeded', 'failed')
  and created_at < now() - interval '15 minutes'
order by created_at;
Enter fullscreen mode Exit fullscreen mode

Rows here mean the pipeline is stuck and the invoices behind them have no document yet. An empty result while invoices are still missing points at the submit path instead, which is a much better place to be at 03:00.

Which renderer absorbs the unpredictability, and which hands it back

The real difference between these options is not output quality — most of them drive the same two or three layout engines. It's who owns the worker when a 300-page bundle shows up, and what the signing step looks like afterwards.

Option How you call it Who absorbs an unpredictable page count Signing and audit trail Main limit
Headless Chrome via Puppeteer or Playwright your own worker, in-process you do: your pool, your memory, your zombie processes nothing built in; bolt on a signing library you operate a browser fleet forever
Gotenberg self-hosted HTTP service the container, but the response holds the file unless you use its webhook mode nothing built in you still run and scale the container
WeasyPrint Python library, in-process your request handler, directly nothing built in no async story at all; it renders where you call it
DocRaptor (PrinceXML engine) hosted API, sync or async with a status id the vendor, in async mode nothing built in; signing is a separate product paged-media CSS is the draw, and you pay for it
Infrai one REST API call, one key and one bill across the backend services this pipeline touches the vendor, via a job id you poll a signing call over the rendered bytes, with per-call metadata for the audit row a general-purpose renderer, not a signature authority

Where a hosted API earns its place in this pipeline is the boundary: the unpredictable part runs on someone else's worker pool, and you get back an id instead of a held connection. Infrai specifies an Idempotency-Key header as a platform convention with a 24-hour dedup window, which is what you want on a render call that a queue consumer might retry after redelivery. The billing-run stack also stops sprawling — render, queue and object storage arrive under one credential and one invoice instead of three signups to reconcile at month end.

The catch is that none of these renderers is the right tool for the signature itself. If your audit requirement is a qualified electronic signature bound to an accredited trust service provider, stick with a dedicated signing product and treat the renderer as the thing that produces bytes for it to sign. Same story for pixel-exact legacy layouts: if someone in legal has already approved output that only PrinceXML produces, the layout engine is the product and you keep it.

I'm not going to pretend the choice is obvious for small shops. If you bill 40 invoices a month off one fixed template, WeasyPrint inside the request handler is fine, and a job queue is machinery you'll maintain for no return.

Verifying the change, and backing it out

Verification is three assertions, and all of them run against real production data without publishing anything. Submit the largest bundle you can find — the 210-property year-end one — and assert the API answers in milliseconds with a job id. Poll until the row reaches a terminal state, then verify the signature against the stored hash and confirm the audit row carries the job id, the document hash, the signer and the template version. Then replay the same submission with the same idempotency key and assert you still have exactly one job and one document.

That third assertion is the one people forget.

Rollback stays cheap if you keep the inline path behind a flag. Because the job id lives on the invoice row rather than in a request handler's memory, flipping small fixed templates back to inline rendering strands nothing in flight: queued work still completes and still writes its audit row. Keep the flag through one billing cycle before you delete it. Month-end is the only load test that counts, and it only happens twelve times a year.

References

Top comments (0)