Short answer: use an asynchronous job endpoint as the default for fintech PDF report generation, reserve a synchronous endpoint for small bounded documents, and expose a batch endpoint only when the workload can be admitted and scheduled without weakening per-tenant limits.
The decision is driven by the tail, not the demo. A single report may render quickly while a month-end burst fills every worker, delays redaction, and leaves callers retrying work that is already in progress. Fidelity, latency, and operational complexity therefore belong in one contract: accept immutable input, make duplicate submissions harmless, redact personal data before any shareable artifact exists, and return an explicit state rather than keeping an HTTP connection open for an uncertain render.
This is an architecture decision record for a US/EU SaaS producing customer account reports. It deliberately does not choose a PDF engine or vendor. The useful choice comes earlier: what the endpoint promises when load rises.
Decision record: invariants and failure boundaries
The primary invariant is blunt: an unredacted document must never enter the download path. Redaction is part of generation, not a cleanup step after upload. The render worker should receive a versioned template, a frozen data snapshot, a redaction policy identifier, locale and time-zone inputs, and a caller-supplied idempotency key. Its output becomes visible only after validation confirms that the artifact belongs to the expected tenant and policy version.
Keep the failure boundary narrow. Request validation and job admission happen before work is queued; rendering and redaction happen inside an isolated worker; publication happens only after validation. A failure in any stage leaves no partially approved URL to share. This separation also makes latency legible: queue delay, render time, validation time, and publication time are different measurements, and collapsing them into one average hides the part operators can actually fix.
There are four practical contract rules:
- Treat the report request as immutable. A correction creates a new job rather than changing an in-flight one.
- Scope idempotency to tenant, report definition, and input version. A retry should locate the original job, not consume another render slot.
- Apply admission control before enqueueing. Once a tenant reaches its concurrency allowance, return a retryable rejection with a bounded retry hint instead of accepting an unlimited backlog.
- Publish opaque artifact identifiers. Authorization is checked again on download; possession of a guessed identifier cannot stand in for tenant access.
The boundary is important because PDFs are binary artifacts, but the control plane is ordinary structured data. In a browser, the completed response can be represented as a Blob; MDN describes a Blob as an immutable, file-like object of raw data and documents creating an object URL for local use. Revoke that object URL after download or preview so the browser does not retain it longer than needed.
One more constraint deserves its own line.
Don't log report inputs.
Log identifiers, stage timings, template and policy versions, byte counts, and outcome classes. That is enough to debug throughput without copying personal data into a second system whose retention rules may differ from the report store.
How should a US/EU SaaS balance PDF fidelity, latency, and operational complexity?
Start with two service classes rather than one universal endpoint. The interactive class has a strict input-size and page-complexity envelope; the deferred class accepts everything within the product's documented limits. Fidelity stays constant across both classes. The system changes when the caller receives the result, not which fonts, redaction rules, or validation checks apply.
| Endpoint shape | Best fit | Latency behavior under load | Operational cost | Main limitation |
|---|---|---|---|---|
| Synchronous render | Small, bounded preview or one-off report | Caller waits for queueing and rendering inside one request budget | Lowest number of moving parts | Retries and connection deadlines can duplicate expensive work unless idempotency is enforced |
| Asynchronous job | Normal customer report generation | Admission is fast; queue delay and render time are observable separately | Requires job state, workers, artifact storage, and cleanup | More client states and authorization checks |
| Batch submission | Scheduled statements with many independent reports | Scheduler can smooth work and enforce tenant fairness | Requires chunking, partial-result semantics, and batch progress | A large batch can monopolize capacity without per-tenant quotas |
The catch is operational weight. An asynchronous path needs durable state, a worker fleet, cleanup for expired artifacts, and a client that understands pending, complete, rejected, and failed terminal outcomes. It is not suitable when every document is predictably tiny, generated rarely, and already fits comfortably inside the application's request deadline. Stick with a synchronous endpoint in that narrow case, but retain idempotency and the same redaction-before-publication invariant.
I'm not sure there is a universal page-count threshold, because a page containing embedded fonts, charts, and complex layout can cost more than several plain pages. Your mileage may vary. Resolve that uncertainty with replay tests built from sanitized production shapes, then define the interactive envelope from the slowest acceptable class rather than from an average document.
Fidelity should not silently degrade to protect latency. If the requested template is outside the synchronous envelope, route it to the deferred contract or reject it before rendering. Quietly dropping fonts, shrinking images, or changing pagination creates a report that is fast but cannot be trusted. For financial documents, deterministic inputs and a recorded template version are more useful than a renderer that makes opportunistic quality decisions.
Endpoint contracts for latency under load
Use three resources with deliberately small responsibilities. A synchronous render endpoint validates, renders, redacts, validates again, and returns the artifact only inside its published envelope. An asynchronous create endpoint returns a job representation immediately after admission. A job read endpoint reports state and, only after publication, an authorized artifact reference. Batch creation accepts a finite list of independent request objects and returns one batch identifier plus child job identifiers.
The endpoint names matter less than their semantics. For example, a team might expose POST /reports/render, POST /report-jobs, and GET /report-jobs/{job_id}. Batch work can be a mode on job creation rather than another public route if that keeps authorization and idempotency consistent. Avoid an endpoint that accepts arbitrary HTML and returns a permanent public URL; it couples untrusted presentation input, expensive execution, and publication into one boundary.
Backpressure begins at admission. Track queued work and active work by tenant, template class, and region; reject before enqueueing when a limit is reached. A retry hint should come from current queue policy, while the idempotency key ensures a caller following that hint does not create duplicate work. Fair scheduling matters more than raw worker count during a statement run: one tenant's 20,000-document batch should be chunked so interactive jobs and other tenants continue to advance.
Measure percentiles for each stage rather than publishing a single end-to-end average. At minimum, record admission duration, queue age at start, render duration, redaction and validation duration, artifact publication duration, total completion time, and rejected work by reason. Keep the labels bounded. Tenant identifiers and job identifiers belong in traces or structured fields, not metric labels that create an unbounded series count.
Retries need stage awareness. A worker may safely retry before publication if its output key is derived from the immutable job identifier and attempts cannot expose intermediate artifacts. After publication, the job record is the authority; repeating the request returns the completed representation. If validation rejects an artifact, keep it outside the download namespace and record a terminal reason that is useful to operators without echoing personal data to clients.
Short deadlines are healthy.
They force the API to distinguish admission from completion. They also expose a common mistake: letting an application server own the only copy of job state. A process restart must not turn accepted work into an unknowable outcome. Persist the transition before acknowledging admission, and make workers claim jobs with a lease so abandoned work can become eligible again without two successful publications.
Critical path in Python
The following sketch keeps framework details out of the decision. It shows the ordering that matters: authenticate, validate, deduplicate, admit, persist, and only then acknowledge. The repositories and queue are interfaces backed by components appropriate to the deployment.
from dataclasses import dataclass
from enum import StrEnum
from typing import Protocol
class JobState(StrEnum):
PENDING = "pending"
RUNNING = "running"
COMPLETE = "complete"
REJECTED = "rejected"
@dataclass(frozen=True)
class ReportRequest:
tenant_id: str
template_version: str
data_snapshot_id: str
redaction_policy_version: str
locale: str
idempotency_key: str
class Jobs(Protocol):
def find(self, tenant_id: str, key: str) -> dict | None: ...
def create_pending(self, request: ReportRequest) -> dict: ...
def mark_complete(self, job_id: str, artifact_id: str) -> None: ...
class Admission(Protocol):
def allowed(self, tenant_id: str, workload_class: str) -> bool: ...
class Queue(Protocol):
def enqueue(self, job_id: str) -> None: ...
def create_report_job(
request: ReportRequest,
jobs: Jobs,
admission: Admission,
queue: Queue,
) -> tuple[int, dict]:
existing = jobs.find(request.tenant_id, request.idempotency_key)
if existing is not None:
return 200, existing
if not admission.allowed(request.tenant_id, "deferred-report"):
return 429, {"state": "rejected", "retry_after_seconds": 15}
job = jobs.create_pending(request)
queue.enqueue(job["id"])
return 202, job
The worker must preserve the same sequence on the artifact side. Render into private temporary storage, apply the named redaction policy, validate the finished bytes, publish under an opaque artifact identifier, and atomically mark the job complete. The download handler then authorizes tenant and user access before returning bytes. It doesn't infer authorization from the job identifier.
Test the state machine, not just the happy-path PDF. Submit the same idempotency key concurrently and assert that one job exists. Stop a worker after rendering but before publication, reclaim its lease, and assert that only one artifact becomes visible. Saturate one tenant's quota and confirm another tenant still advances. Feed a report containing names, account identifiers, and free-form notes through the validation fixture, then assert that the shareable artifact contains only the allowed representation. These tests say more about production latency and privacy than a single warm render benchmark.
Rejected option and its valid use case
The rejected default is a synchronous-only API. It looks attractive because the client receives bytes from one call and the service avoids job storage. Under variable load, however, the request owns queueing, rendering, redaction, validation, and transfer time. Client deadlines then become an accidental scheduler, and retries arrive precisely when capacity is already constrained.
Still, don't delete the synchronous path on principle. It is the right fit for a bounded preview generated from already-redacted sample data, or for an internal tool with low concurrency and a measured document envelope. Give it a hard admission check and the same idempotency behavior as the deferred path. If it cannot start promptly, decline the synchronous request and let the caller create a job; do not accept the request and then lower fidelity to finish in time.
The batch endpoint has a similarly narrow use case: scheduled statement runs where every child report can succeed or fail independently. It should not be a way to bypass normal quotas. Chunk batches, expose child status, define whether cancellation applies only to work that has not started, and clean up artifacts according to the same retention policy as individually submitted jobs.
The final decision is intentionally conditional. Choose asynchronous jobs for the normal path when latency under load and redaction safety matter; keep synchronous rendering for a proven bounded class; add batching only after tenant-aware admission and partial-result semantics exist. The architecture earns its complexity by making overload explicit while keeping fidelity and the publication boundary unchanged.
Top comments (0)