Short answer: treat a PDF-to-image timeout as a workload-budget failure before treating it as a broken file. Read page geometry first, compute total output pixels at the requested resolution, and admit only the pages that fit a bounded batch. For property-management documents, render one page at a time, redact before anything leaves the trusted boundary, and publish a preview only after every expected page has succeeded.
A 40-page lease packet is not one unit of work. At 300 DPI, each page has four times the pixels it would have at 150 DPI because both dimensions double. That square-law effect is the first thing to check when a conversion crosses a deadline. Page count, page dimensions, requested DPI, concurrency, and the renderer's per-page timing belong in the same diagnostic record.
Count pixels first.
How should I debug PDF image conversion times by page?
Start with pixels, not file size. A compact PDF can contain many pages or complex page content, while a large PDF can mostly contain already-compressed images. PDF page dimensions are expressed in default user space units; ISO 32000-2 defines the document model, and 72 user-space units correspond to one inch unless the page establishes otherwise. A useful workload estimate for page i is ceil(width_i / 72 * dpi) * ceil(height_i / 72 * dpi).
Sum that value across the requested page range. This estimate does not predict exact wall-clock time: transparency, clipping paths, fonts, annotations, and embedded images still affect rendering. It does expose the multiplier that teams often miss. Ten pages at twice the DPI represent roughly four times the output pixels, before encoder cost and memory copies enter the picture. Record the media box and crop box separately. A renderer commonly uses the crop box for visible output, but the media box describes the physical page boundary. Rotation matters when checking width and height, even though it does not change the pixel product. Reject impossible or policy-breaking dimensions before allocating a bitmap. One trap is especially nasty in property workflows: a scanned inspection appendix may have ordinary page dimensions but expensive image decoding. Pixel budgeting catches output pressure, not every input-side cost. Keep actual per-page render duration beside the estimate so later admission limits are based on your own corpus rather than a borrowed benchmark. I initially reach for file size because it is cheap to obtain; the page geometry corrects that instinct before the scheduler makes an expensive promise.
Invariants and failure boundaries
The privacy invariant is strict: an unredacted page never reaches preview storage, logs, a retry payload, or a browser. The completeness invariant is equally important. A preview marked ready must contain exactly the admitted page range, in order, with a redaction result attached to each page. A partially rendered lease packet can hide the addendum a reviewer needs to inspect.
The execution boundary should be a page. The publication boundary should be the document.
Keep those boundaries separate.
Use explicit limits for maximum pages, maximum pixels per page, maximum total pixels per batch, and maximum in-flight bitmap bytes. These are policy values, not universal constants. Choose them from worker memory, deadline, representative lease and inspection documents, and the latency target of the preview screen. A request outside the interactive budget can enter a slower asynchronous lane or require a narrower page range; it should not keep consuming an interactive worker until a generic timeout kills it.
A useful failure record includes a pseudonymous document ID, page index, page boxes, DPI, estimated pixels, elapsed render time, attempt number, renderer version, and a coarse error class. Do not log extracted text, owner names, tenant names, addresses, phone numbers, or unredacted thumbnails. Delivery systems taught me to distrust identifiers in retry logs: data copied for debugging tends to outlive the original request. The same discipline applies here.
Compare the admission strategies
| Strategy | Throughput behavior | Failure isolation | Best fit | Main cost |
|---|---|---|---|---|
| Whole document in one worker call | A long packet occupies one slot until completion | Weak; one bad page obscures completed work | Small, tightly bounded inputs | Retries repeat successful pages |
| One queued job per page | Scheduler can distribute work across workers | Strong; page failures retry independently | Mixed page cost and asynchronous previews | Queue and state overhead |
| Pixel-budgeted page batches | Fewer scheduling operations while bounding each unit | Good; failed batches remain small | High batch throughput with varied packet lengths | Requires page inspection before admission |
For a property-management preview service where batch throughput is the decision axis, pixel-budgeted batches are the practical default. They avoid a queue message for every trivial page but stop a high-resolution scan-heavy packet from monopolizing a worker. The batch builder should preserve page order and stop adding pages before either the pixel or estimated-memory ceiling is crossed.
The trade-off is deliberate.
Do not confuse worker concurrency with throughput. Raising concurrency can reduce throughput when several rasterizations compete for memory and trigger paging or process termination. Measure completed pages per worker-second and peak resident memory under a representative mix. Queue delay, render duration, redaction duration, encode duration, and publish duration need separate histograms; one end-to-end timer cannot identify the limiting stage.
Critical path in Python
The renderer and redactor below are generic interfaces. The important part is the control flow: inspect, budget, render, redact, stage, verify, then publish atomically.
from dataclasses import dataclass
from math import ceil
from typing import Iterable, Protocol
@dataclass(frozen=True)
class PageSpec:
index: int
width_points: float
height_points: float
def pixel_count(self, dpi: int) -> int:
width = ceil(self.width_points * dpi / 72)
height = ceil(self.height_points * dpi / 72)
return width * height
class Renderer(Protocol):
def inspect(self, document: bytes) -> list[PageSpec]: ...
def render(self, document: bytes, page_index: int, dpi: int) -> bytes: ...
class Redactor(Protocol):
def redact(self, image: bytes, page_index: int) -> bytes: ...
def make_batches(
pages: Iterable[PageSpec], dpi: int, max_batch_pixels: int
) -> list[list[PageSpec]]:
batches: list[list[PageSpec]] = []
current: list[PageSpec] = []
current_pixels = 0
for page in pages:
pixels = page.pixel_count(dpi)
if pixels > max_batch_pixels:
raise ValueError(f"page {page.index} exceeds the pixel policy")
if current and current_pixels + pixels > max_batch_pixels:
batches.append(current)
current = []
current_pixels = 0
current.append(page)
current_pixels += pixels
if current:
batches.append(current)
return batches
def build_preview(document, dpi, max_pages, max_batch_pixels,
renderer, redactor, staging_store) -> None:
pages = renderer.inspect(document)
if not pages or len(pages) > max_pages:
raise ValueError("document page count is outside policy")
completed: set[int] = set()
for batch in make_batches(pages, dpi, max_batch_pixels):
for page in batch:
raster = renderer.render(document, page.index, dpi)
safe_raster = redactor.redact(raster, page.index)
staging_store.put_page(page.index, safe_raster)
completed.add(page.index)
expected = {page.index for page in pages}
if completed != expected:
raise RuntimeError("preview is incomplete")
staging_store.publish_atomically(page_count=len(pages), dpi=dpi)
The example intentionally does not calculate a timeout from pixels. A timeout is an operational deadline, while pixels are an admission signal. Calibrate their relationship from controlled load tests using the renderer version and document mix that production actually runs. Keep test documents synthetic or properly de-identified, and include ordinary text pages, large scans, rotated pages, unusual page boxes, embedded fonts, and transparency.
Retries need an idempotency key covering the document version, page index, DPI, and redaction-policy version. Without the policy version, a retry can accidentally reuse an image produced under older privacy rules. Cap retries and send deterministic failures, such as a page beyond the dimension policy, to review rather than cycling them through the queue. Apply exponential backoff only to failures classified as transient.
Why reject whole-document rendering?
The rejected option is a single conversion call that receives the full document and returns every image. It has a valid use case: small, trusted, tightly limited documents in a low-concurrency process, especially when the chosen renderer can reuse parsed document state efficiently. It also makes a prototype pleasantly small.
I would not use it as the default for mixed property packets. It ties retry scope to the largest input, hides the page that consumed the deadline, and can hold several full-resolution bitmaps at once. Worse, a caller may receive no useful diagnostic boundary when page 37 fails after 36 successful renders. Per-page execution or bounded page batches make that boundary explicit.
The trade-off is bookkeeping. You must track page state, clean abandoned staged objects, and ensure only one manifest becomes visible. That work is justified when throughput matters because it turns an unbounded document job into schedulable units. Keep the publication operation atomic so readers never see gaps.
The debugging sequence is short: inspect page count and boxes; compute pixels at the requested DPI; compare the request with admission limits; locate the slow page from timings; then separate rendering, redaction, encoding, and storage latency. Lowering DPI can be a valid preview decision, but only after verifying that names, account numbers, signatures, and redaction edges remain reviewable. Privacy review quality is the floor.
References
- ISO 32000-2, Portable Document Format: https://www.iso.org/standard/75839.html
- NIST Privacy Framework: https://www.nist.gov/privacy-framework
- OWASP Logging Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
- Python documentation,
math.ceil: https://docs.python.org/3/library/math.html#math.ceil
Top comments (0)