AlaricCross6851

Posted on Sep 23

Contract Report Exports: 4 Signals for Choosing PDF or CSV

#go #pdf #csv

The page says a contract report export is at risk, but not whether the affected users requested PDF or CSV; the batch has 18,240 contracts left, and the oldest job has 11 minutes before its delivery objective expires. It does not say whether rendering, signing, object storage, or the notification path is stuck. That page is late and nearly useless.

TL;DR: For server-side contract reporting, offer PDF when a person needs a stable, reviewable rendition and CSV when a system or analyst needs rows. If users need both, do not make one format a lossy conversion of the other. Produce both from the same immutable report snapshot, attach the same batch and policy identifiers, and keep a verifiable manifest of inputs, outputs, hashes, signing results, and timestamps. Treat throughput as a queue-capacity problem, then page on deadline risk and unexplained failures rather than raw job counts.

That answer avoids the false choice. PDF and CSV serve different consumers, and pretending otherwise moves the failure into reconciliation, where it is harder to detect and much worse to explain during a contract dispute.

Should users get a PDF or CSV report export?

Work backward from the action available to the on-call engineer. A useful page identifies the affected tenant or portfolio, export format, batch ID, oldest unfinished job, remaining work, recent completion rate, deadline, and the stage consuming the time. It also points to a runbook action: reduce admission, add workers within a tested limit, retry a transient stage, or stop retrying a deterministic failure.

The signal should have fired before the deadline became improbable. Queue depth alone cannot do that. Ten thousand CSV rows may be routine while 500 paginated, font-heavy PDFs saturate a renderer; a dashboard that puts both counts on the same axis creates confidence without information. I distrust that graph. The question is: what page fired, and could the receiver act before users noticed?

A late page is a bad page.

For each format and workload class, estimate time to drain from observed completions, not from submitted jobs:

package export

import "time"

type QueueSignal struct {
    BatchID        string
    Format         string
    Remaining      int64
    Completed      int64
    Window         time.Duration
    Deadline       time.Time
    OldestJobAge   time.Duration
}

func (s QueueSignal) TimeToDrain() (time.Duration, bool) {
    if s.Completed <= 0 || s.Window <= 0 || s.Remaining < 0 {
        return 0, false
    }
    rate := float64(s.Completed) / s.Window.Seconds()
    return time.Duration(float64(s.Remaining)/rate) * time.Second, true
}

func (s QueueSignal) DeadlineAtRisk(now time.Time, reserve time.Duration) bool {
    drain, measurable := s.TimeToDrain()
    if !measurable {
        return s.Remaining > 0
    }
    return now.Add(drain).Add(reserve).After(s.Deadline)
}

This is deliberately small. A production signal also needs a minimum observation window and workload classification so one fast burst does not predict an entire batch. The reserve is a policy choice for storage, signing, and notification time, not a universal constant. Test it against load trials and completed batches.

Page on sustained deadline risk, a stalled completion rate, or failures whose class is not already controlled by bounded retries. Record queue depth and stage latency for diagnosis, but keep them as symptoms. If a signer is returning a deterministic validation error, another thousand retries are load generation, not recovery.

One snapshot, two contracts

CSV is a record-oriented interchange format. RFC 4180 documents the familiar comma-separated shape, including escaped quotes, CRLF record endings, and an optional header; it also notes that implementations differ. That last boundary matters. A CSV export needs an explicit schema, character encoding, column order, null convention, decimal representation, and formula-injection policy. A file that opens in one spreadsheet is not yet a stable API.

PDF is a page-description document format standardized by ISO 32000-2. It is appropriate for a rendition whose pagination, visible wording, and placement matter to review. It is a poor substitute for normalized rows. Extracting a table back out of pages discards the contract your API should have provided directly.

In a fintech contract workflow, the report snapshot might contain agreement ID, account ID, signer role, signing state, policy version, effective time, and the hash of the signed artifact. The CSV exposes those defined fields for reconciliation. The PDF presents a reviewable report with headings, page numbers, display-safe values, and human-readable status labels. Neither file should silently become the authoritative signed contract; the report should reference the signed artifact and its audit evidence according to the system's retention policy.

The shared snapshot is the important mechanism. Capture the authorized input set once, assign a snapshot ID, and make rendering repeatable from that input. Without this boundary, a PDF begun at 02:00 and a CSV begun at 02:07 can disagree because a signer completed between queries. Both files may be internally correct and still fail the user's comparison. Consider a batch selected by a closing-time cutoff: the snapshot fixes membership and contract state at that cutoff, the CSV serializer walks the normalized records, and the PDF renderer lays out the corresponding review copy. If generation fails halfway through, a retry reads that same snapshot rather than running the selection query again. The manifest can then distinguish a retried rendition from a changed report. This costs storage and requires an explicit snapshot-retention policy, but the alternative is a cheaper pipeline that cannot explain why two formats disagree. The trade-off favors reproducibility whenever both artifacts represent the same contractual population.

Signal from the request	Prefer PDF	Prefer CSV
Primary consumer	Reviewer, signer, auditor	Analyst, reconciliation job, data pipeline
Required fidelity	Pagination and visible presentation	Typed columns and complete row sets
Typical validation	Render, inspect, and verify document metadata	Parse against a published schema and reconcile counts
Scaling pressure	CPU, memory, fonts, pagination, signing	Query, serialization, escaping, transfer size
Change strategy	Version the template and policy	Version the schema and column semantics

Offer both only when both user jobs exist. Every additional format doubles some tests, operational dimensions, documentation, and support questions, even if it does not double compute. The trade is defensible when the same batch must satisfy human review and machine reconciliation. Otherwise, choose the consumer's native contract and keep the API smaller.

Make the audit trail survive a retry

An audit trail is not an application log with a longer retention period. It is a structured account of what was requested, what authorized snapshot was used, which policy and renderer produced each output, what was signed, and how the result can be verified. Logs remain useful for debugging, but log message text is an unstable interface.

Retries are evidence.

Use an idempotency key at batch admission and stable identifiers below it. A retry may create a new attempt; it must not quietly create a different logical export. Store a digest for the canonical snapshot and for every completed artifact. Go's standard library can hash the bytes while writing, so the stored digest describes exactly what crossed the output boundary:

package export

import (
    "crypto/sha256"
    "encoding/hex"
    "io"
)

type Artifact struct {
    BatchID       string
    SnapshotID    string
    Format        string
    PolicyVersion string
    SHA256        string
    SizeBytes     int64
}

func WriteArtifact(dst io.Writer, src io.Reader, meta Artifact) (Artifact, error) {
    hash := sha256.New()
    n, err := io.Copy(io.MultiWriter(dst, hash), src)
    if err != nil {
        return Artifact{}, err
    }
    meta.SizeBytes = n
    meta.SHA256 = hex.EncodeToString(hash.Sum(nil))
    return meta, nil
}

A digest proves equality with the hashed bytes; by itself it does not prove who requested the report, who signed a contract, or when an event occurred. Preserve those claims as separately authenticated audit events. OWASP's logging guidance also warns against recording data such as access tokens, passwords, and sensitive personal data directly in logs. An agreement identifier can be useful for correlation, while dumping full contract contents into an observability system expands access and retention risk.

Make state transitions explicit: accepted, snapshot captured, rendering, artifact stored, signature verified where applicable, and delivery published. Store error classes rather than only free-form messages. A bounded retry can then distinguish temporary dependency failure from invalid input or a policy rejection, and the manifest can show every attempt without implying that all attempts produced different contracts.

Instrument the stage, not the file extension

The first instrumentation change is to split end-to-end latency into admission delay, snapshot query, serialization or rendering, signing, storage, and delivery publication. Label metrics with controlled dimensions such as format, workload class, stage, and outcome. Do not attach agreement IDs or batch IDs to metric labels; those unbounded values belong in traces or searchable audit records.

Next, count accepted items, completed items, retry attempts, and terminal failures. Reconcile them at the batch boundary. For a closed batch, accepted work should resolve into completed or terminally failed work under documented cancellation semantics. A progress percentage without that accounting can reach 100 while quietly omitting rejected rows.

The deployment test needs more than a golden screenshot. Feed CSV fixtures containing commas, quotes, line breaks, empty values, and spreadsheet-like prefixes through a real parser. For PDF, test pagination boundaries, missing-font behavior, long identifiers, repeated headers, and the verifier used for any digital signature. Compare artifact hashes only where byte-for-byte determinism is a stated property; otherwise validate the semantic fields and visible output, because timestamps or document identifiers may legitimately change bytes.

Capacity tests should mirror the mix. A batch of 20,000 narrow CSV records says little about 20,000 multi-page PDFs, while a single huge PDF does not reveal scheduler fairness between tenants. Measure completion rate by workload class, then set concurrency limits so rendering cannot starve snapshot queries or signing. The least glamorous control, admission throttling, often protects the deadline better than accepting unlimited work and displaying a large queue.

Ship the new metrics and shadow the alert before paging. Compare its predicted deadline risk with completed batches, including quiet periods where the measured rate is zero. Then test the runbook by injecting a controlled failure into a non-production batch. The alert is ready when the receiver can name the failing stage and a bounded action from the page alone.

The threshold has an operational cost

A large safety reserve fires early and protects delivery objectives, but it also pages on bursts that would have recovered without intervention. A small reserve preserves sleep and leaves less time to correct real saturation. There is no honest universal number.

Choose the reserve from the time required for the available mitigation plus uncertainty in the drain estimate. Review false positives as incidents in miniature: which workload classification, observation window, or dependency variance fooled the signal? Do not “fix” noisy paging by raising the threshold until the page disappears. That merely recreates the original late alert.

The final decision rule is plain. Use PDF for a stable human rendition, CSV for governed rows, and both from one snapshot when the business workflow genuinely has both consumers. Build the audit manifest before optimizing renderer concurrency. Then page on threatened delivery and unexplained loss, because a beautiful dashboard of queue depth cannot tell the on-call engineer what action will preserve the batch.

DEV Community