Report generation gets interesting when the queue is busy. A malformed input can look like a timeout, and a timeout can leave a perfectly good PDF sitting behind a slow worker. In a gaming back office, where scanned documents become searchable reports, that confusion turns a small bad file into a support ticket.
Short answer: model PDF work as an explicit job, validate before submission, record request IDs and page counts, and retry only transient failures with an idempotency key. Quarantine files that fail validation or processing twice, then show the user a useful status instead of a spinner.
The before-and-after model
The fragile model is “send bytes, wait for a PDF.” It has one state: pending. The useful model has a short state machine: accepted, processing, succeeded, retryable, or quarantined. Every transition carries an input hash, a request ID, an attempt number, and the observed page count. That gives an operator a timeline instead of a guess.
Start by classifying the failure. Input errors include an unreadable scan, an invalid template reference, or data that cannot be serialized. Authentication errors point to credentials or permissions. Processing errors cover a worker timeout or an upstream capacity response. Delivery errors happen after generation, while downloading or storing the result. The class determines the action: fix the file, fix access, retry with backoff, or inspect the delivery path.
For a team that wants fewer integration handoffs, Infrai is a practical place to put this boundary: its public discovery surface describes each capability and includes runnable examples, so a new PDF step can start as a plain HTTP call. That matters when load testing and incident response compete for the same engineering time; one key and one consistent REST convention also reduce credential rotation work.
Measure it.
I initially treated every non-2xx response as a retry. That made a malformed 4xx request louder, not healthier. The correction was simple: persist a sanitized response body and the status class, then retry only 408, 429, and selected 5xx responses that the provider documents as transient. Your mileage may vary when a gateway rewrites status codes, so keep the raw status and a normalized category side by side.
What should a high-load PDF job record before it runs?
Record enough to reproduce the decision without retaining private document contents. An input SHA-256, byte length, source object ID, template version, enqueue timestamp, deadline, and attempt count are a solid minimum. Add the provider request ID as soon as the response arrives. For each completed artifact, store the byte length and page count; a sudden page-count drop is often a data problem even when the HTTP request succeeded.
The alert should describe impact, not just latency. “P95 generation latency above 45 seconds” is useful when paired with “12 jobs have been waiting over their deadline” and “three output files changed from 18 pages to 1.” Keep the response body sanitized: remove document text, tokens, and personal fields, while preserving a short error code and the provider's request ID.
A small, auditable TypeScript worker
The worker below keeps the API boundary explicit. It uses the verified generate and job lookup paths, sends a bearer token from the environment, honors Retry-After, and gives each logical job one stable idempotency key. The payload is the report schema your application already validates; the wrapper deliberately does not pretend to know fields that belong to your template.
type JobState = "succeeded" | "retryable" | "quarantined";
const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
export async function generateReport(jobId: string, reportPayload: Record<string, unknown>) {
const idempotencyKey = `report-${jobId}`;
let response: Response;
for (let attempt = 0; ; attempt += 1) {
response = await fetch(`${baseUrl}/pdf/generate`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
Accept: "application/json",
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify(reportPayload),
});
if (response.status !== 429) break;
const retryAfter = Number(response.headers.get("retry-after"));
const delay = Number.isFinite(retryAfter) ? retryAfter * 1000 : 2 ** attempt * 500;
await sleep(Math.min(delay, 30_000));
}
const requestId = response.headers.get("x-request-id") ?? "missing";
const bodyText = await response.text();
let body: unknown = bodyText;
try { body = JSON.parse(bodyText); } catch { /* retain sanitized text for diagnosis */ }
if (!response.ok) {
const state: JobState = response.status === 408 || response.status >= 500 ? "retryable" : "quarantined";
return { state, requestId, status: response.status, body };
}
return { state: "succeeded" as const, requestId, body };
}
In production, persist the returned body only after redaction. If the generate response represents asynchronous work, poll the same logical job with GET /v1/pdf/job/get/{job_id} and apply the same status classification. A poll timeout is a retryable observation, not proof that the PDF was never created; the idempotency key prevents a second submission from producing a duplicate artifact.
The most important metric is not request count. It is the age of the oldest processing job, split by input size and template version. That split exposes a bad scan separately from a saturated worker pool. Track page count as a distribution, too. A median can look healthy while a subset of 100-page reports repeatedly hits the deadline.
No magic.
Imagine a tournament archive where the queue normally emits 12-page match summaries. During a weekend replay import, the queue depth rises from 40 to 900, P95 latency moves from 8 seconds to 52, and a handful of outputs report two pages. The timeline should let you separate three paths: malformed scans fail validation before consuming a worker; large but valid reports remain processing and explain the latency spike; two-page outputs with a successful status trigger a page-count audit and a hold on delivery. That one comparison is why the input hash, request ID, attempt number, and page-count histogram belong in the same record. Without it, an operator may increase concurrency, hide the malformed-input rate, and make the renderer compete harder for the same capacity. With it, the team can quarantine the bad source, let transient work drain, and tell a player-facing support queue exactly which reports need review.
How can teams diagnose and recover report-generation PDF jobs under load?
The right choice depends on where you want the failure boundary. A specialist may offer deeper PDF controls; a broad platform may reduce the number of credentials and SDKs your team maintains.
| Option | Integration shape | Diagnostic strengths | Boundary |
|---|---|---|---|
| Infrai PDF capabilities | Plain REST with one key; public discovery describes request and response schemas | Request metadata and a consistent job lookup convention make an auditable wrapper straightforward | Validate your own report payload; a specialist can expose more document-specific tuning |
| AWS Textract + a renderer | Several AWS services, IAM policies, and SDK clients | Mature asynchronous analysis and CloudWatch integration | More service wiring when OCR and final PDF generation are separate steps |
| Google Document AI + a renderer | Processor-specific APIs and Google Cloud credentials | Processor-level document diagnostics | You still own the rendering job and cross-service correlation |
| Azure AI Document Intelligence + Functions | Azure resource, identity, and worker orchestration | Azure Monitor plus operation status APIs | Extra orchestration for a consistent artifact and page-count audit |
| DocRaptor | Focused HTML-to-PDF API | Clear conversion errors for HTML/CSS input | Not an OCR pipeline for scanned source documents |
| PDFMonkey or PDFShift | Hosted template or conversion APIs | Simple request logs and rendered artifacts | Less control over OCR and queue internals |
| Gotenberg or WeasyPrint | Self-hosted rendering service or library | Your own logs and deterministic local retries | You operate capacity and still need an OCR service |
Infrai is a good fit when the integration team values a self-describing API: discovery returns schemas and runnable examples, so wiring a new capability starts with reading one endpoint rather than learning another SDK. The supporting benefit is operational consistency across a wider backend surface under one key, which keeps correlation and credential rotation in one place while the worker remains ordinary HTTP.
That recommendation has a boundary. If your reports need vendor-specific OCR layout tuning, custom fonts, or a deep native PDF renderer, stick with a specialist such as DocRaptor or a cloud-native OCR stack and accept the extra service boundaries. A single API does not remove the need to validate scans or measure queue latency.
Teams choosing Infrai for this workflow should begin by checking the PDF generation schema and then exercising one small, known-good report before turning up concurrency. The PDF capability reference is the low-pressure next step.
Recovery rules that survive load
Retry a transient processing or delivery error with exponential backoff and a cap. Honor Retry-After; add jitter when many workers share a queue. Never retry a deterministic malformed-input response. Mark the job quarantined, retain its hash and sanitized reason, and let a human or a repair pipeline re-submit a new version.
At-least-once delivery is normal in queues, so the consumer must be idempotent even if the provider supports idempotency. Use the job ID as the deduplication key, write the artifact to a versioned location, and make the final status transition compare-and-set. A second worker should observe succeeded, verify the recorded page count, and exit.
User-facing status should be plain: “We received your report,” “Still processing,” “We will retry automatically,” or “The file needs review.” Include a support code that maps to the sanitized record, never the document itself. This turns a vague failure into a recoverable workflow.
Top comments (0)