Short answer: use a cheap metadata endpoint for discovery, reserve full PDF rendering for the few templates that need visual fidelity, and make the audit record independent of either response. That split keeps latency predictable under load without pretending that one endpoint fits every contract.
Our e-commerce service signs vendor agreements and customer terms on the server. The PDF is evidence, but the evidence is the chain around it: schema version, signer identity, hash of the exact bytes, and an append-only event. Form discovery is an input to that chain, not the chain itself.
What the bill is actually made of
Retention dominates this workflow long after the signing request finishes. A 2 MB signed document stored for seven years is a very different obligation from a 40 KB schema snapshot, and storing every intermediate render multiplies the footprint without improving legal review. Keep the final signed bytes, the normalized field map, and the event records that explain who approved what. Drop transient raster previews and failed render artifacts after a short, documented window.
That deletion has a cost. When a merchant disputes a clause, you may have to regenerate a preview from the final PDF, and a changed rendering library can make that preview look different. The durable answer is a hash and a schema version, not an assumption that pixels will remain identical.
A practical retention table makes the trade visible:
| Artifact | Keep | Why | Failure cost if discarded |
|---|---|---|---|
| Signed PDF bytes | Contract term plus legal hold | Exact evidence | Cannot prove what was signed |
| Field schema snapshot | Same term as contract | Replays validation and mapping | Discovery may change later |
| Audit events | Same term as contract | Explains sequence and actor | Gaps in review timeline |
| Preview images | Days, unless policy says otherwise | Human review aid | Re-render may differ |
| Failed attempts | 7–30 days with redaction | Troubleshooting | Less context for incident analysis |
The bill is therefore a policy decision. Object storage is cheap compared with an investigation, but unlimited retention is not a design.
How should schema discovery balance fidelity, latency, and operations under load?
Treat endpoint choice as a two-stage probe. First call a schema or metadata route that returns field names, types, required flags, and the template revision. Only call a page-render or byte-producing route when a human needs to inspect layout or when a PDF parser cannot represent a feature. A discovery response should be small enough to stay in cache and explicit enough to invalidate safely.
Latency under load is mostly queueing. If every checkout worker renders a 20-page document just to learn that it has six text fields, the renderer becomes the bottleneck and retries amplify it. Put a bounded worker pool in front of rendering, cap concurrency per tenant, and record p50, p95, and p99 separately for discovery, render, signing, and storage. I would rather return a clear 429 with a retry-after value than let requests hang until the browser gives up.
Fidelity still matters. Flattening a field can remove a signature widget; converting a checkbox to a text field can alter the meaning of a form. Define a fixture set with rotated pages, Unicode names, timezone offsets, and overlapping widgets. Compare the normalized schema and a rendered checksum in CI. Your mileage may vary across PDF producers, so keep a small corpus from actual partners instead of trusting a single synthetic file.
Here is a deliberately boring Python shape for the boundary. It keeps transport, policy, and audit concerns separate:
from dataclasses import dataclass
from hashlib import sha256
@dataclass
class Discovery:
template_id: str
revision: str
fields: list[dict]
def audit_payload(pdf_bytes: bytes, discovery: Discovery, actor: str) -> dict:
return {
"template_id": discovery.template_id,
"revision": discovery.revision,
"field_count": len(discovery.fields),
"sha256": sha256(pdf_bytes).hexdigest(),
"actor": actor,
}
The endpoint client can be swapped, but the audit payload cannot silently change shape.
Failure modes that deserve a name
The first failure mode is stale discovery: a cached schema outlives the template revision and validation accepts a field that no longer exists. Key caches by template ID plus revision, and reject an unsigned document if the revision in the audit event differs from the one used for mapping.
The second is retry duplication. A timeout after signing does not prove that signing failed. Use an idempotency key derived from the contract ID and revision, then persist the provider response hash before acknowledging the order.
The third is regional drift. US and EU traffic may cross different storage or processing boundaries. Record region and clock source in each event, keep personal data out of logs, and verify that your subprocess or queue does not copy the PDF into a default global bucket.
I once assumed a fast metadata call made the whole path fast. It didn't. The queue behind the renderer was the real 99th-percentile problem, and a single unbounded retry loop turned a brief spike into a backlog. That correction changed our dashboard: queue age became a release gate, not a footnote.
Choosing an endpoint contract
Prefer contracts with explicit content types, deterministic revision identifiers, bounded payloads, and documented pagination for large field sets. A binary response should be handled as bytes, not coerced through text; the browser Blob API exists for this exact distinction. For server code, stream to a temporary file with a size limit, hash while reading, and delete the temporary file on both success and failure.
The catch is operational complexity. A metadata-only design is not suitable when reviewers must inspect exact appearance before signing; add rendering for that checkpoint. A render-first design is a poor fit for high-volume discovery; stick with metadata probes and sample renders instead. Teams with no capacity to operate queues may choose a managed conversion service, while teams with strict residency or custom PDF controls may accept running their own workers.
Do not make price the decision rule. The durable choice is the one whose latency, residency, and evidence behavior you can observe and explain during an audit.
References
- https://developer.mozilla.org/en-US/docs/Web/API/Blob
- https://www.rfc-editor.org/rfc/rfc9110
- https://www.rfc-editor.org/rfc/rfc8785
Top comments (0)