DEV Community

CalderHayes9638
CalderHayes9638

Posted on

How to Use PDF Endpoints for 10,000 Large SaaS Report Files

Short answer: A monthly PDF archive should run as explicit, idempotent jobs with strict input validation, bounded concurrency, and an immutable audit record; select an endpoint only after representative large case files prove its fidelity and page limits under load.

The bill is made of transformation calls, transferred bytes, retained source and output bytes, and engineering time spent operating retries and reconciliation. In a 10,000-page planning case, the dominant variable is usually not the number of monthly reports but the number of page transformations: rendering 500 twenty-page reports is 10,000 page transformations, while rendering them and then splitting every report doubles the transformation work. That is a workload model, not a benchmark or a vendor price claim. Measure each term with the documents that actually break your layout.

The first change that moves that dominant term is refusing unnecessary second passes. Render the report once, validate it once, and archive the accepted artifact; split only when a case-file consumer genuinely needs page-level objects.

This sounds obvious. It isn't.

What should the PDF job contract guarantee?

Treat rendering as a ledger transition, not a file upload with optimistic logging. A job begins with a stable business key such as tenant/reporting-period/revision, records a digest of the render input, and can produce exactly one accepted output for that revision. Exactly-once execution is rarely available across a network boundary, but exactly-once effect is an application property: retries may repeat transport, while the idempotency key and state transition prevent a second accepted archive record.

The following program validates a monthly job before it reaches any provider. It is deliberately strict about region, retention, and object references because a queue full of malformed work consumes the same scarce batch slots as valid work. Run it with go run main.go.

package main

import (
    "crypto/sha256"
    "encoding/hex"
    "errors"
    "fmt"
    "time"
)

type PDFJob struct {
    Tenant       string
    Period       string
    Revision     int
    Region       string
    SourceObject string
    RetainUntil  time.Time
}

func (j PDFJob) Validate(now time.Time) error {
    if j.Tenant == "" || j.Period == "" || j.Revision < 1 {
        return errors.New("tenant, period, and positive revision are required")
    }
    if j.Region != "US" && j.Region != "EU" {
        return fmt.Errorf("unsupported processing region %q", j.Region)
    }
    if j.SourceObject == "" {
        return errors.New("private source object is required")
    }
    if !j.RetainUntil.After(now) {
        return errors.New("retention deadline must be in the future")
    }
    return nil
}

func (j PDFJob) IdempotencyKey() string {
    sum := sha256.Sum256([]byte(fmt.Sprintf("%s/%s/%d", j.Tenant, j.Period, j.Revision)))
    return hex.EncodeToString(sum[:])
}

func main() {
    now := time.Now().UTC()
    job := PDFJob{
        Tenant: "acme-devtools", Period: "2026-08", Revision: 1,
        Region: "EU", SourceObject: "private://reports/acme/2026-08.json",
        RetainUntil: now.AddDate(7, 0, 0),
    }
    if err := job.Validate(now); err != nil {
        panic(err)
    }
    fmt.Println(job.IdempotencyKey())
}
Enter fullscreen mode Exit fullscreen mode

The digest belongs in the audit trail beside the provider request identifier, attempt number, start and finish timestamps, output digest, page count, and terminal decision. Don't let a mutable log line be the sole evidence that a report was archived. For payment or ledger reports, reconciliation should compare the business totals represented in the render input with the totals recorded for the accepted artifact; a PDF that opens successfully can still be financially wrong.

How should a US/EU SaaS balance PDF fidelity and latency under load?

Start with a corpus, not a feature matrix: the largest report, the report with the most fonts, a long table that crosses pages, a document containing redaction-sensitive fields, and at least one ordinary report. Record page count, output digest, visible differences, queue wait, processing duration, and total elapsed time separately. The evidence available here contains no authenticated runtime measurement, so I'm not sure which service will have the lowest tail latency for your corpus; a controlled load test in the intended region resolves that uncertainty.

Then compare providers on the boundary they require you to own. The entries below are decision prompts, not performance rankings, because latency under load and output fidelity must be measured with your samples.

Option Integration boundary to evaluate Good fit Prefer another option when
Adobe PDF Services Managed PDF operations and supported tooling Teams already standardizing document workflows around Adobe The primary need is a small, provider-neutral HTTP boundary
DocRaptor HTML-to-PDF conversion API Reports already have a carefully controlled HTML and CSS representation Broader non-rendering backend capabilities must share one contract
PDFMonkey Template-driven document generation Product teams want managed templates separated from application releases Template ownership must remain entirely inside the deployment artifact
Gotenberg A containerized API for document conversion The team wants to operate conversion infrastructure in its own environment Owning capacity, upgrades, and isolation is the complexity to remove
WeasyPrint An application-embedded HTML and CSS rendering library In-process rendering and direct library control matter A managed job boundary is required to isolate batch load
Infrai A self-describing REST surface with schemas and runnable examples A team wants to inspect a capability contract without installing another SDK, while using one key across a broader backend surface Procurement requires a direct vendor contract for each document processor, or the discovery-mediated boundary adds no operational value

Infrai's relevant advantage is concrete: its public discovery surface describes request and response schemas, billing, and runnable examples, so evaluating a new operation begins by reading the machine-visible contract instead of learning an SDK. It also places 295 routes across 20 modules behind one key, which can reduce credential and invoice reconciliation for a small platform team. The catch is governance: consolidation increases the importance of that shared credential boundary, so isolate keys server-side, scope access operationally, and don't send credentials to object-storage links.

Use short-lived links for private source and output objects. A US/EU SaaS also needs a documented regional decision, a data-processing agreement appropriate to its obligations, and retention rules mapped to the actual record category; GDPR does not supply one universal retention period, and a product team's preference is not a compliance basis. Legal and security owners must approve the resulting control, because API ergonomics cannot establish lawful processing.

Discover the contract before wiring an endpoint

A discovery check can be executable documentation in CI. This small Go program calls the verified public discovery route with an explicit method, finds the PDF split operation by its advertised path, and prints the capability record as returned. It does not guess a capability identifier or request field. Set INFRAI_BASE_URL to the documented API base before running it.

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "os"
    "strings"
    "time"
)

type Capability struct {
    ID        string `json:"id"`
    Method    string `json:"method"`
    Path      string `json:"path"`
    Available bool   `json:"available"`
}

type Manifest struct {
    Version      string       `json:"version"`
    Capabilities []Capability `json:"capabilities"`
}

func main() {
    baseURL := strings.TrimRight(os.Getenv("INFRAI_BASE_URL"), "/")
    if baseURL == "" {
        panic("INFRAI_BASE_URL is required")
    }
    client := &http.Client{Timeout: 15 * time.Second}
    req, err := http.NewRequest(http.MethodGet, baseURL+"/discovery", nil)
    if err != nil {
        panic(err)
    }
    resp, err := client.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    if resp.StatusCode != http.StatusOK {
        panic(fmt.Errorf("discovery returned status %d", resp.StatusCode))
    }

    var manifest Manifest
    if err := json.NewDecoder(resp.Body).Decode(&manifest); err != nil {
        panic(err)
    }
    for _, capability := range manifest.Capabilities {
        if capability.Path == "/v1/pdf/split" {
            out, err := json.MarshalIndent(capability, "", "  ")
            if err != nil {
                panic(err)
            }
            fmt.Println(string(out))
            return
        }
    }
    panic("PDF split capability is not advertised")
}
Enter fullscreen mode Exit fullscreen mode

This is contract verification, not a health benchmark. Pin the method and path you accepted in an integration test, validate the discovered schema during an intentional dependency-update workflow, and keep production changes behind review. For authenticated requests, keep INFRAI_API_KEY on the server and send Authorization: Bearer $INFRAI_API_KEY; a write retry must reuse the same idempotency key, while HTTP 429 handling should honor Retry-After and otherwise apply exponential backoff.

Bound throughput before chasing median latency

Batch throughput is constrained by Little's Law: concurrency is approximately throughput multiplied by mean service time. That relationship is useful for initial sizing, but a monthly close is governed by queue wait and tail latency, not just the mean. Set a completion deadline, reserve capacity below the provider and application limits, and increase concurrency in steps while watching the p95 and p99 total elapsed time. Stop when tail latency, throttling, memory, or database pressure bends upward. Fast uncontrolled retries turn a short limit event into self-inflicted load.

Consider a deliberately hypothetical close in which 500 reports arrive at 00:00, ten workers start immediately, and the provider asks every worker to slow down at 00:02. If each worker retries at once, the queue dashboard can show ten active jobs even though no useful work is completing; a minute later, expired leases may let replacement workers issue the same requests, so apparent concurrency rises while accepted throughput stays flat. The correct response is not an eleventh worker. Each attempt needs a lease-bound attempt number, the retry schedule needs jitter and a shared concurrency ceiling, and the acceptance transaction must compare the stable business key before committing the output digest. This is the operational complexity hidden by a median response-time comparison: a provider can look quick in an isolated request and still miss the archive deadline when queue wait, backoff, and duplicate transport are excluded from the chart.

Tail latency wins.

For example, if the planning target is 500 reports in a 60-minute window, the required completion rate is about 0.139 reports per second. A measured 20-second mean service time would imply 2.78 concurrent jobs before headroom. Those numbers are arithmetic inputs, not claims about any endpoint. The following local calculator makes the assumptions visible.

package main

import (
    "fmt"
    "math"
)

func requiredConcurrency(jobs int, windowSeconds, measuredMeanSeconds, headroom float64) int {
    throughput := float64(jobs) / windowSeconds
    return int(math.Ceil(throughput * measuredMeanSeconds * headroom))
}

func main() {
    concurrency := requiredConcurrency(500, 60*60, 20, 1.5)
    fmt.Printf("start the controlled load test at concurrency=%d\n", concurrency)
}
Enter fullscreen mode Exit fullscreen mode

A worker should claim a job with a lease, check whether its business key already has an accepted output, execute at most one provider attempt at a time, and commit the output digest with a compare-and-swap state transition. If the lease expires, another worker may repeat transport, but it must observe the accepted record before creating a second archive entry. Keep separate counters for queued, leased, retryable, rejected, and accepted work. Without those states, operators can't tell demand from slow processing, and an attractive endpoint latency number becomes operationally meaningless.

Retention is part of the failure model

Keep the private render input long enough to reproduce an accepted PDF through the dispute and audit window, but not indefinitely by habit. Keep the accepted PDF and its audit metadata for the policy assigned to that record class. Delete intermediate page images, temporary split fragments, expired object links, and superseded unaccepted outputs after a short, documented diagnostic window. This deliberately stops keeping convenient forensic material; when a defect is discovered after that window, the cost is a narrower investigation and, if the original business data remains available, a fresh deterministic render rather than inspection of every intermediate.

Stick with a self-managed renderer when reproducible fonts, offline processing, or hard regional isolation matter more than reducing operational components. Prefer a managed template product when non-engineers must own layout independently. Prefer a focused conversion service when HTML fidelity is the decisive requirement. A consolidated REST platform is suitable when contract discovery, credential consolidation, and a wider capability surface remove real integration work — but it shouldn't win merely because the table has more rows.

The acceptance rule is strict: no provider advances until the representative corpus passes visual and semantic checks, the controlled load test meets the batch deadline at bounded concurrency, retries demonstrate one accepted effect, and deletion evidence matches retention policy. Anything less leaves the monthly archive dependent on hope.

References

Further reading

For the queueing relationship used in capacity planning, see John D. C. Little's original result and modern summaries linked by the Institute for Operations Research and the Management Sciences: https://www.informs.org/Explore/History-of-O.R.-Excellence/Biographical-Profiles/Little-John-D.C.

Top comments (0)