DEV Community

Rivenor85
Rivenor85

Posted on

PDF Bundles in Node.js: Page-Order Invariants for Marketplace Contract Batches Explained

#node #pdf #documentprocessing #marketplace

Short answer: use POST /v1/pdf/merge with an ordered manifest, validate each source digest before assembly, and poll GET /v1/pdf/job/get/{job_id} until the output digest is recorded in an append-only audit event. Keep the manifest's sequence number as the only authority for page order.

The least complex design is a deterministic bundle job: accept an ordered manifest, validate every input before merging, write one immutable audit event, and return a job identifier. A worker can then merge the bytes in that order and retry safely.

That approach matters in a marketplace. A contract packet may contain the agreement, disclosures, and a signature page from different services. A buyer can tolerate a slower packet; they cannot tolerate page 7 silently becoming page 3. Batch throughput is the primary axis here, so the design should make ordering cheap to verify while keeping telemetry small enough to retain.

What does “order” mean in a PDF bundle?

A PDF is not a bag of pages. Its page tree defines a traversal order, and the merged document must preserve the manifest order when those trees are combined. ISO 32000-2 describes the PDF object model; your API should turn that standard-level fact into an application invariant: every item has an integer position, positions are unique, and the set is contiguous from zero (or from one, if that is your contract). Pick one convention and reject the other at the boundary.

For example, this manifest is unambiguous:

{
  "bundle_id": "mkt-2026-00421",
  "items": [
    {"position": 0, "document_id": "agreement", "sha256": "..."},
    {"position": 1, "document_id": "disclosure", "sha256": "..."},
    {"position": 2, "document_id": "signature-page", "sha256": "..."}
  ]
}
Enter fullscreen mode Exit fullscreen mode

Do not derive order from upload completion, object names, or database row order. Those are incidental properties. Validate the digest of each source before assembly; otherwise a retried download can produce a valid-looking packet whose audit record describes different bytes.

A batch API with one observable state transition

The HTTP surface only needs a submission and a status read. The submit request carries the complete manifest, not a collection of loosely related uploads. That gives the worker a stable unit of work and gives the audit store one event to append.

curl -X POST https://api.example.test/v1/pdf/merge \
  -H 'content-type: application/json' \
  --data '{
    "bundle_id": "mkt-2026-00421",
    "items": [
      {"position": 0, "document_id": "agreement", "sha256": "..."},
      {"position": 1, "document_id": "disclosure", "sha256": "..."},
      {"position": 2, "document_id": "signature-page", "sha256": "..."}
    ],
    "idempotency_key": "checkout-8f1c"
  }'
Enter fullscreen mode Exit fullscreen mode

The service should persist accepted before a worker claims the job, then append assembling, complete, or failed events. Make the transition append-only and include the manifest digest, source digests, actor, and timestamps. A retry with the same idempotency key must address the same bundle record; it must not create a second audit trail.

A status response can remain compact:

curl https://api.example.test/v1/pdf/job/get/job-00421
Enter fullscreen mode Exit fullscreen mode

Return the state, ordered item count, output digest when complete, and a failure code when not. Avoid returning every worker log line from this endpoint.

Where batch throughput actually goes

Merging is usually bounded by input and output bytes, storage reads, and the number of object operations. Page count alone is a poor proxy: three large scans can cost more memory and I/O than twenty small text PDFs. In Node.js, stream source objects into the merger where the library permits it, and cap concurrent source reads. An unbounded Promise.all turns a large marketplace settlement into a storage thundering herd.

Use a queue with a fixed worker count, then measure three rates separately: manifests accepted per minute, source bytes read per second, and completed bundles per minute. The first tells you about admission; the second exposes storage pressure; the third is the business result. Keep these as low-cardinality metrics. bundle_id belongs in a trace or audit record, not a metric label.

I initially treated every event as useful evidence. The retention review changed my mind: a 2 KB log line emitted for each of 12 source files is 24 KB per bundle before retries, while one structured completion event can preserve the same decision evidence. The deliberate loss is the per-page narrative in long-term logs. During an incident, I can still reconstruct it from the immutable manifest and source digests; I cannot reconstruct a missing source byte from a verbose message that was sampled away.

How do you make retries and failures boring?

Separate validation failures from assembly failures. A duplicate position, missing digest, or non-PDF input should be rejected before queueing. A storage timeout or worker restart is retryable. A malformed object that fails validation after download is terminal for that version of the manifest.

Record the reason code, not an unbounded exception string. Keep the original error in short-lived diagnostics with access control, and put a stable code plus attempt number in the audit event. The output object should be written to a temporary key and promoted only after its digest is computed. That prevents consumers from observing a partially written bundle.

For batch fairness, reserve capacity for small bundles or use a weighted queue. Otherwise one large settlement packet can occupy every worker and make ordinary contract signing appear down. Test this with a fixture containing deliberately shuffled inputs, duplicate positions, a missing source, a digest mismatch, and a worker restart between write and promotion. The expected result is deterministic: either the same ordered digest or the same terminal reason.

The retention policy is part of that proof. Keep manifests, source digests, output digests, state transitions, and actor identity for the contract's legal retention period. Sample high-volume timing telemetry separately, and never sample the audit events that establish what was signed. You are choosing which evidence survives. Make that choice explicit. The trade-off is deliberate: this pattern is a poor fit for interactive, sub-second previews because queueing and digest verification add latency; a synchronous, bounded merger is simpler for that workload. More machinery is justified only when a measured queue, storage, or memory limit requires it.

Further reading

Top comments (0)