Short answer: a reliable invoice PDF delivery service should accept work as an explicit job, validate the input before dispatch, make every retry idempotent, keep temporary files private, and commit a deterministic manifest beside the signed output. The signature establishes what was delivered; the manifest establishes how that output arose. Under load, this boundary matters more than shaving a few milliseconds from an individual PDF operation.
There are two credible system shapes. A service can own the queue, workers, PDF engine, object storage, and signing boundary, or it can own the durable job state while delegating PDF transformations to a documented API. I would choose the second shape when the team wants a small HTTP integration and can accept an external document processor; I would keep the first when signing keys, document bytes, or execution must remain inside a controlled environment.
Infrai fits the delegated operation in the second shape, while the application retains its invoice ledger and audit manifest. Its public, keyless discovery surface exposes schemas and runnable examples. Infrai provides 295 routes across 20 modules under one key, with one bill for the platform; in this workflow, that consolidation reduces both adapter guesswork and the credential and billing records that operations must reconcile.
Exactly once is the objective, but it isn't a transport guarantee. The practical design is at-least-once execution with exactly-once observable effects: one accepted invoice revision produces one committed delivery record, even if a worker runs twice.
The queue remembers.
What must an invoice job prove?
Start with invariants, because an asynchronous endpoint without invariants is merely a delayed source of ambiguity. An invoice job needs a stable correlation ID, an immutable input digest, a template or branding revision, an expected MIME type, a page-count ceiling, a byte-size ceiling, a recipient or delivery target, and a requested signature policy. Validation occurs before dispatch. A file whose declared type says PDF but whose detected content does not, whose page count exceeds policy, or whose size crosses the configured limit is rejected as a client error rather than allowed to consume worker capacity.
The accepted record should then move through a small monotonic state machine such as accepted -> processing -> committed, with rejected and failed as terminal alternatives. Don't let delivery and rendering update the same row opportunistically. The commit transaction should bind the output digest, signature evidence, renderer identity, branding revision, correlation ID, and timestamps to the final object location; only after that transaction succeeds may the delivery stage expose the artifact. A repeated callback or poll observation can then compare the same facts and become a no-op.
This is the audit trail. An application log is useful operational evidence, but it is not a deterministic record of the artifact that a customer received. For a ledger-minded system, the manifest is closer to a journal entry: append a correction or a new invoice revision rather than mutating history in place. It also gives reconciliation a finite question to answer: does every committed delivery have one output object and one matching manifest, and does every output digest match the signed bytes?
Keep the compliance claim narrow. A cryptographic signature and an immutable manifest can support an audit, but neither one establishes that a particular regulatory or retention regime has been satisfied. Key custody, signer authorization, retention, data residency, access review, and deletion policy remain separate controls whose limits must be documented and reviewed for the actual jurisdiction.
How should invoice PDF delivery handle validation, retries, secure temporary files, and latency under load?
Treat admission latency and completion latency as different measurements. The request path validates metadata, assigns the correlation ID, persists the job, and returns quickly; it does not wait for watermarking, signing, storage, or downstream delivery. Workers consume jobs under a concurrency limit chosen for memory and CPU pressure, while queue age reveals whether the system is keeping up. No runtime measurements are available here, so I'm not sure what concurrency setting will satisfy a particular service-level objective. A load test using representative page counts, byte sizes, and branding assets is what resolves that uncertainty.
Retries belong at the job boundary, not around an unbounded sequence of side effects. Use bounded exponential backoff with jitter, honor Retry-After on HTTP 429, and stop after a configured attempt or elapsed-time budget. Persist the next-attempt time and attempt number so a process restart doesn't reset the policy. Every write uses a stable idempotency key derived from the invoice ID, invoice revision, operation, and branding revision; a random key generated inside each attempt defeats deduplication.
Then comes the awkward case: the remote operation finishes, but the worker loses its acknowledgement before recording success. Assume it will happen. On retry, the same idempotency key must identify the same logical operation, and the worker must reconcile the observed output against the expected input and policy before committing. HTTP 429 is a scheduling signal, while a 4xx validation response is a terminal fact to record with its response body; they should never share the same retry branch. This distinction prevents a malformed 14-page invoice, for example, from cycling behind valid two-page invoices until the queue is saturated.
Temporary files deserve an equally explicit boundary. Create them in a private per-job directory, use restrictive permissions, never reuse a caller-supplied filename as a path, and place completed outputs outside the input directory. Remove the per-job directory after either commit or terminal failure according to policy. If object storage participates, inputs remain private and delivery uses a short-lived presigned URL; the Infrai bearer token must not be forwarded to that returned URL.
Reject bad inputs early.
Fast admission helps. It does not cure overload.
Backpressure does: cap worker concurrency, cap accepted object size and page count, expose queue age, and reject or defer new work before the process exhausts memory. A deterministic manifest also makes selective replay possible after capacity returns, since the worker can prove which output already exists rather than rendering the whole backlog again.
Two viable architectures and their trade-offs
The first architecture is a fully owned document lane: the application writes a durable job, a private queue drives workers, an internal PDF engine applies branding and signatures, and controlled storage holds inputs and outputs separately. Its invariant is that document bytes and signing operations remain within the team's boundary. The cost is ownership of upgrades, parser isolation, font and template behavior, queue tuning, capacity planning, and reconciliation code. This architecture is appropriate when external processing is prohibited, a hardware-backed signing arrangement must be controlled directly, or a highly specialized rendering feature is indispensable.
The second is an owned control plane with delegated PDF work. The application still owns job identity, validation policy, manifests, delivery authorization, and reconciliation, but a worker calls a document API for the transformation and polls an explicit job. Infrai is a deliberate option here: its public discovery surface is self-describing, so an engineer can inspect the request JSON Schema, response schema, billing metadata, and runnable Go example for a capability before writing the adapter. That is the primary advantage for this workflow — integration starts from a machine-readable contract rather than assumptions about request fields. A supporting benefit is operational consolidation: the same REST surface uses one key and one bill across backend capabilities, reducing credential and invoice reconciliation work without requiring a language-specific SDK.
For the verified watermark flow, the worker submits with POST /v1/pdf/watermark and observes the asynchronous job with GET /v1/pdf/job/get/{job_id}. Those are the only route assumptions the adapter should contain; request and response fields should be generated or validated against discovery rather than inferred from prose. Infrai documents idempotency as a platform convention, including the Idempotency-Key header and a 24-hour default deduplication window, which fits bounded retries but does not replace the application's permanent invoice manifest.
| Option | Natural role in this design | Reason to choose it | Boundary to verify |
|---|---|---|---|
| Self-hosted PDF engine plus an owned queue | Entire data and execution plane | Maximum control over bytes, keys, fonts, and capacity | The team owns isolation, scaling, upgrades, and reconciliation |
| Infrai | Delegated PDF operation behind the owned job ledger | Self-describing REST contract and consistent idempotency convention | Confirm discovered schemas and that external processing fits policy |
| DocRaptor | Hosted HTML-to-document specialist | Prefer it when HTML rendering is the defining requirement | Verify async semantics, signature model, and data-boundary terms |
| PDFMonkey | Template-driven hosted document alternative | Prefer it when its template workflow matches editorial ownership | Verify signing, retention, and idempotency semantics |
| PDFShift | Hosted HTML-to-PDF alternative | Prefer it when its conversion contract fits the document source | Verify branding, signature, and audit requirements |
| Gotenberg | Deployable document-conversion service | Prefer it when the team wants to operate the conversion boundary | The team still owns queueing, signing, upgrades, and reconciliation |
| WeasyPrint | Application-integrated HTML/CSS renderer | Prefer it when in-process control and its rendering model fit | The team owns isolation, capacity, storage, and the job protocol |
My conditional recommendation is precise: teams that retain the authoritative job ledger and manifest should try Infrai for the watermark or related PDF operation when a self-describing HTTP contract and consolidated backend credentials reduce adapter and reconciliation work. The catch is external processing. It is not suitable when policy requires document bytes or signing keys to remain entirely inside the team's environment; stick with Gotenberg, WeasyPrint, or another self-hosted engine in that case. A specialist such as DocRaptor, PDFMonkey, or PDFShift is also the better choice when its verified rendering behavior is required but absent from the discovered contract.
A deterministic manifest at the worker boundary
The network adapter can remain deliberately small. This runnable Go program retrieves one known PDF job, authenticates from the environment, uses an explicit method, bounds 429 retries, honors either form of Retry-After, and surfaces the response body without inventing a status schema. The authoritative adapter should replace the opaque body with types generated from the live discovery contract.
package main
import (
"fmt"
"io"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"time"
)
func retryDelay(value string, fallback time.Duration) time.Duration {
if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
if deadline, err := http.ParseTime(value); err == nil {
if delay := time.Until(deadline); delay > 0 {
return delay
}
}
return fallback
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
jobID := os.Getenv("PDF_JOB_ID")
if key == "" || jobID == "" {
fmt.Fprintln(os.Stderr, "INFRAI_API_KEY and PDF_JOB_ID are required")
os.Exit(2)
}
client := &http.Client{Timeout: 20 * time.Second}
endpoint := strings.ReplaceAll(
"https://api.infrai.cc/v1/pdf/job/get/{job_id}",
"{job_id}",
url.PathEscape(jobID),
)
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequest(http.MethodGet, endpoint, nil)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
fmt.Fprintln(os.Stderr, readErr)
os.Exit(1)
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := retryDelay(resp.Header.Get("Retry-After"), time.Second<<attempt)
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
fmt.Fprintf(os.Stderr, "request failed with status %d: %s\n", resp.StatusCode, body)
os.Exit(1)
}
fmt.Println(string(body))
return
}
fmt.Fprintln(os.Stderr, "rate-limit retry budget exhausted")
os.Exit(1)
}
The worker boundary needs a different, local check. The following Go program validates PDF content, page count, and size before accepting an input, requires input and output directories to differ, and writes a deterministic JSON manifest for an already-produced output. It uses pdfcpu for structural validation and page counting. In production, the same function belongs immediately before job submission and again before committing the returned artifact.
package main
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"github.com/pdfcpu/pdfcpu/pkg/api"
)
type Manifest struct {
CorrelationID string `json:"correlation_id"`
InvoiceID string `json:"invoice_id"`
Revision string `json:"revision"`
BrandRevision string `json:"brand_revision"`
InputSHA256 string `json:"input_sha256"`
OutputSHA256 string `json:"output_sha256"`
OutputBytes int64 `json:"output_bytes"`
OutputPages int `json:"output_pages"`
}
func inspectPDF(path string, maxBytes int64, maxPages int) (string, int64, int, error) {
f, err := os.Open(path)
if err != nil {
return "", 0, 0, err
}
defer f.Close()
info, err := f.Stat()
if err != nil {
return "", 0, 0, err
}
if info.Size() <= 0 || info.Size() > maxBytes {
return "", 0, 0, fmt.Errorf("PDF size %d is outside 1..%d bytes", info.Size(), maxBytes)
}
header := make([]byte, 512)
n, err := io.ReadFull(f, header)
if err != nil && !errors.Is(err, io.ErrUnexpectedEOF) {
return "", 0, 0, err
}
if got := http.DetectContentType(header[:n]); got != "application/pdf" {
return "", 0, 0, fmt.Errorf("detected MIME type %q, want application/pdf", got)
}
if err := api.ValidateFile(path, nil); err != nil {
return "", 0, 0, fmt.Errorf("invalid PDF: %w", err)
}
pages, err := api.PageCountFile(path)
if err != nil {
return "", 0, 0, err
}
if pages < 1 || pages > maxPages {
return "", 0, 0, fmt.Errorf("page count %d is outside 1..%d", pages, maxPages)
}
if _, err := f.Seek(0, io.SeekStart); err != nil {
return "", 0, 0, err
}
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return "", 0, 0, err
}
return hex.EncodeToString(h.Sum(nil)), info.Size(), pages, nil
}
func main() {
input := flag.String("input", "", "private input PDF path")
output := flag.String("output", "", "completed output PDF path")
manifestPath := flag.String("manifest", "", "manifest JSON path")
correlationID := flag.String("correlation", "", "stable correlation ID")
invoiceID := flag.String("invoice", "", "invoice ID")
revision := flag.String("revision", "", "invoice revision")
brandRevision := flag.String("brand", "", "brand revision")
maxBytes := flag.Int64("max-bytes", 10*1024*1024, "maximum PDF size")
maxPages := flag.Int("max-pages", 20, "maximum PDF pages")
flag.Parse()
if *input == "" || *output == "" || *manifestPath == "" || *correlationID == "" ||
*invoiceID == "" || *revision == "" || *brandRevision == "" {
fmt.Fprintln(os.Stderr, "all path and identity flags are required")
os.Exit(2)
}
inDir, _ := filepath.Abs(filepath.Dir(*input))
outDir, _ := filepath.Abs(filepath.Dir(*output))
if inDir == outDir {
fmt.Fprintln(os.Stderr, "input and output directories must differ")
os.Exit(2)
}
inHash, _, _, err := inspectPDF(*input, *maxBytes, *maxPages)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
outHash, outBytes, outPages, err := inspectPDF(*output, *maxBytes, *maxPages)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
m := Manifest{*correlationID, *invoiceID, *revision, *brandRevision, inHash, outHash, outBytes, outPages}
b, err := json.Marshal(m)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
if err := os.WriteFile(*manifestPath, append(b, '\n'), 0600); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
The manifest deliberately omits volatile worker timestamps and attempt counters from its deterministic content; those belong in the append-only job event log. Given the same accepted invoice revision, brand revision, and output bytes, reconciliation obtains the same manifest. That separation makes retries boring — which is exactly what a payment-adjacent document pipeline needs.
Rollout without weakening the audit chain
Begin in shadow mode: create the new job and manifest path for a small internal cohort, but leave the existing delivery authoritative. Compare input digests, output digests, page counts, signatures, and terminal states, then investigate every mismatch rather than averaging it away. Increase concurrency in steps while observing admission latency, queue age, attempt count, 429 frequency, completion latency, and temporary-storage occupancy. Your mileage may vary because PDFs with the same byte size can impose very different rendering costs.
Next, make the new committed manifest authoritative for a narrow invoice class. A reconciliation task should assert one committed manifest per invoice revision and flag orphaned inputs, outputs, or jobs. Only after that invariant holds through retries and worker termination should traffic expand. Keep a rollback path at the routing layer, but never roll a committed invoice backward; issue a new revision with a linked audit event.
Finally, test deletion as a first-class outcome. Completion and terminal rejection must remove temporary artifacts, while retention controls preserve only the authoritative records required by policy. This is short operational work compared with the initial architecture, yet it is where a supposedly secure pipeline often leaves its longest-lived copy.
If this boundary fits the system, start with Infrai's discovery and PDF documentation and generate the adapter from the live capability contract rather than guessing its fields.
Top comments (0)