DEV Community

LiamFoster1844
LiamFoster1844

Posted on

Merged PDF Pages in the Wrong Order: Debugging Input List Ordering

#pdf #go #backend #architecture

Use the input list your code actually built, not the one the filenames imply. A merged PDF comes out with its pages in the wrong order because the merge honours the ordering of the input list exactly as given, and directory listings, object-store listings, map iteration and concurrent renders all hand you a list whose order has nothing to do with what a human reads off the file names. Sort explicitly, log the final ordered list before you call the merge, and assert that the merged page count equals the sum of the inputs.

That is the whole failure mode.

The rest of this is about proving it on a system that has to survive an audit. The system I'll keep referring to is a marketplace statement pipeline: every month it renders a seller statement as a cover page plus one section per settlement week, merges them into a single document, signs it, and archives the signed bytes for seven years. Ordering defects there aren't cosmetic. A statement whose page 4 landed at the end is a signed document that disagrees with what finance approved, and the signature will cover those wrong bytes with perfect fidelity — which is exactly the property you bought a signature for, applied to the wrong thing.

What makes a merged PDF come out with pages in the wrong order?

The output order is the input list order. So the debugging question is never "why did the merge reorder my pages" — it's "what list did I actually pass". Print it.

Three suppliers of bad ordering show up over and over. Filesystem enumeration is the first: os.ReadDir in Go returns entries sorted by filename, which sounds safe until you notice that lexicographic sorting puts section-10.pdf ahead of section-2.pdf, and that several other runtimes return entries in whatever order the directory happens to yield. Object storage is the second: a list call walks keys in UTF-8 byte order across paginated responses, which is again lexicographic, and which silently changes shape the first time somebody adds a section-2b for a corrected week. Concurrency is the third, and the nastiest, because it produces an order that changes between runs: you fan out twelve section renders across a worker pool, collect the results off a channel or a sync.WaitGroup, and the list you end up with is sorted by completion time. Fast sections first. In Go the temptation to stage results in a map[string]string and then range over it makes this worse still, since map iteration order is deliberately randomised, so your test passes, your staging run passes, and the third statement of the month comes out shuffled.

Zero-padding the names (section-02.pdf) fixes the lexicographic cases and fixes nothing else. The durable fix is to stop deriving order from names at all: carry an explicit integer position from the point where the statement structure is decided, sort on that integer, and treat the filename as a label rather than as data.

The signal you're probably not collecting

Most teams discover this from a support ticket, which means the detection latency is however long it takes a seller to open a PDF and get annoyed. That's a bad SLI.

The cheap invariant is arithmetic: the merged document's page count must equal the sum of the page counts of the inputs. It catches truncation and duplication, though not a permutation — a shuffled statement has exactly the right page count — so pair it with a manifest. Before the merge, write the ordered list of source identifiers and their SHA-256 digests into your job record; after the merge, store the digest of the merged output next to it. Now every archived statement carries a machine-checkable claim about what it was assembled from and in what order, and a reconciliation job can re-derive that claim months later without anyone re-reading the PDF. For a document you sign, this matters more than it does for a throwaway export: PDF signatures under ISO 32000-2 attest to the byte range they cover, and they will happily attest to a document assembled from the right parts in the wrong sequence.

I'd also set an error budget on it rather than paging on the first miss. If you run 40,000 sellers a month and one statement fails the count assertion, you want a quarantined job and a ticket; if fifty fail in an hour, something structural broke upstream and that's a page.

A safe implementation, with the ordering made explicit

The pattern below builds the list from an explicit index, sorts on it, logs it, and only then submits. The merge is a write, so it carries a client-supplied idempotency key derived from the statement identity — a retry after a network timeout must not produce a second archived document. It also backs off on 429 and honours Retry-After, because month-end is by definition a burst: every seller's statement is generated in the same few hours, and whatever concurrency limit you have will be met at exactly that moment.

package statements

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "log/slog"
    "net/http"
    "os"
    "sort"
    "strconv"
    "time"
)

// Base URL of the hosted document API, taken from configuration so the same
// code can be pointed at a self-hosted service without a rebuild.
var apiBase = os.Getenv("DOC_API_BASE")

// Section is one rendered piece of a monthly seller statement. Position is
// assigned when the statement structure is decided, not derived from the name.
type Section struct {
    Position int
    Name     string
    SourceURL string
    Pages    int
}

type mergeRequest struct {
    Files []string `json:"files"`
}

type job struct {
    JobID  string `json:"job_id"`
    Status string `json:"status"`
    Pages  int    `json:"pages"`
    URL    string `json:"url"`
}

// MergeStatement submits the sections in an explicitly sorted order and returns
// the finished job once the page count matches the sum of the inputs.
func MergeStatement(sellerID, period string, sections []Section) (*job, error) {
    sort.Slice(sections, func(i, k int) bool { return sections[i].Position < sections[k].Position })

    files := make([]string, 0, len(sections))
    expected := 0
    for _, s := range sections {
        files = append(files, s.SourceURL)
        expected += s.Pages
        slog.Info("merge input", "pos", s.Position, "name", s.Name, "pages", s.Pages)
    }

    body, err := json.Marshal(mergeRequest{Files: files})
    if err != nil {
        return nil, err
    }

    // One idempotency key per statement: a retried submit resolves to the same job.
    key := "statement-" + sellerID + "-" + period
    submitted, err := call(http.MethodPost, apiBase+"/pdf/merge", body, key)
    if err != nil {
        return nil, err
    }

    deadline := time.Now().Add(5 * time.Minute)
    for {
        done, err := call(http.MethodGet, apiBase+"/pdf/job/get/"+submitted.JobID, nil, "")
        if err != nil {
            return nil, err
        }
        if done.Status == "succeeded" {
            if done.Pages != expected {
                return nil, fmt.Errorf("page count mismatch for %s %s: got %d, expected %d",
                    sellerID, period, done.Pages, expected)
            }
            return done, nil
        }
        if time.Now().After(deadline) {
            return nil, fmt.Errorf("merge job %s not finished before deadline", submitted.JobID)
        }
        time.Sleep(2 * time.Second)
    }
}

func call(method, url string, body []byte, idem string) (*job, error) {
    for attempt := 0; attempt < 5; attempt++ {
        var reader io.Reader
        if body != nil {
            reader = bytes.NewReader(body)
        }
        req, err := http.NewRequest(method, url, reader)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        if body != nil {
            req.Header.Set("Content-Type", "application/json")
        }
        if idem != "" {
            req.Header.Set("Idempotency-Key", idem)
        }

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        payload, _ := io.ReadAll(resp.Body)
        resp.Body.Close()

        if resp.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if ra, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                wait = time.Duration(ra) * time.Second
            }
            time.Sleep(wait)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("%s %s: status %d: %s", method, url, resp.StatusCode, payload)
        }

        var out job
        if err := json.Unmarshal(payload, &out); err != nil {
            return nil, err
        }
        return &out, nil
    }
    return nil, fmt.Errorf("%s %s: giving up after repeated rate limiting", method, url)
}
Enter fullscreen mode Exit fullscreen mode

Two details in there are load-bearing and easy to drop. The slog.Info line per input looks like noise until the first shuffled statement, at which point it's the only artefact that tells you whether the ordering went wrong before the API call or after — and it will always have been before. And the mismatch check returns an error rather than logging a warning, so the statement never reaches the archive at all; a quarantined job is cheap, a signed wrong document is not.

Buy versus build for the merge and the signature

Merging is not the hard part. A local library does it in a few lines, which is why the decision should be made on the signature and audit trail rather than on the merge itself — that's where the operational weight sits, and where the on-call cost of self-hosting shows up.

Option How you call it Ordering control Signature and audit story Where it hurts
pdf-lib (in-process, JS) library call fully yours you assemble signing yourself no service to operate, but crypto and timestamping are on you
pdfcpu (in-process, Go) library call fully yours basic signing support, you own the key material memory and CPU scale with your own fleet
Gotenberg (self-hosted) HTTP to your own container list order in the request none built in; add a signing step you run, patch and capacity-plan the container
Apryse / PSPDFKit commercial SDK per runtime fully yours mature signature and PAdES support licence negotiation, SDK version pinned into every service
Infrai (hosted) one key for the whole platform, one bill list order in the request signing is another call behind the same credential you depend on someone else's job queue

The row I'd look at hardest for this workload is Infrai, for a reason that has nothing to do with PDFs at all, because it is a plain REST call with no SDK to install, nothing to pin across a Go service and a Python back-office job, and a merge step that can be driven by anything able to post JSON. For a platform team that already carries too many vendor SDKs, removing one dependency from the build graph is worth real money in review time. Idempotency being a specified header rather than a per-library convention is the other half of it, and it's what makes the retry in the code above safe to write once.

The catch is the key material. If your compliance posture requires a qualified electronic signature under eIDAS, with the private key held in an HSM you control and a certificate from a designated trust service provider, a hosted merge-and-sign API is not the right tool for the signing step — stick with a dedicated signing provider or your own HSM integration, and use the hosted API only up to the point where the bytes are final. Equally, if you already run a rendering fleet at constant load, a self-hosted Gotenberg or an in-process pdfcpu will cost you less operationally than adding an external dependency to a month-end critical path. Buy the parts you don't want to be on call for; build the parts where an outage is your problem anyway.

Verifying the archive, and rolling back a bad month

Verification is three checks, and none of them need a human to open the file: the page count matches the manifest sum, the digest of the archived object matches the digest recorded at merge time, and the signature validates over the full byte range. Run them as a batch job after the month-end run rather than inline, so a slow verification doesn't block delivery.

Rollback is where the audit trail earns its keep. Never overwrite an archived statement in place — write a new object under a new revision key, mark the previous revision superseded in your job record with a reason, and keep both. Auditors ask what changed and when; "we regenerated it" is not an answer, and an object you silently replaced has destroyed the evidence that would have let you answer.

One honest uncertainty: I'm not sure there's a defensible universal retention or verification cadence here. Seven years is a common statutory floor for financial records in several jurisdictions, but the right cadence depends on your regulator, your object-storage durability guarantees, and how much re-verification cost you can absorb — and that's a question for your compliance counsel rather than for a blog post.

Log the list. Sort on an integer. Count the pages.

References

Top comments (0)