DEV Community

RhettFletcher9678
RhettFletcher9678

Posted on

Invoice Text Protection: True Redaction Versus Drawing Black Boxes Before Signing

#pdf #redaction #fintech

Short answer: treat redaction as a destructive data transformation, then sign the transformed invoice and retain an auditable link to the approved source. A black rectangle is only presentation. If account details, customer notes, or internal order identifiers remain in a PDF content stream, annotation, attachment, metadata field, or OCR text layer, the invoice is not redacted even when a viewer shows an opaque box.

For a fintech invoice pipeline, the decision rule is blunt: remove sensitive objects before the final signature, then prove both absence and document integrity. Keep the unredacted source in a separately controlled evidence store under its own retention policy. Never depend on the published PDF as both the public artifact and the rollback copy.

Should invoice text use true redaction or drawing black boxes?

PDF is a structured container, not a screenshot. ISO 32000-2 defines pages in terms of document objects and content streams, while annotations are separate objects associated with a page. Drawing a filled rectangle can change what a human sees without deleting the text underneath. Copy and paste, text extraction, object inspection, layer changes, or removal of an annotation may expose the original value.

The same distinction applies to scanned invoices. Covering pixels on the visible page does not necessarily remove a hidden OCR text layer. An invoice package may also carry embedded files, comments, form values, XMP metadata, bookmarks, or alternate representations. Redaction has to follow the data, not the rectangle.

This creates an awkward failure around signatures. A PDF digital signature uses a byte range to identify the bytes covered by the signature. Changing signed bytes after approval can invalidate that signature; adding an incremental update still leaves earlier revisions in the file and must be evaluated against the signature policy. The safe default is to redact first and apply the distribution signature last.

Order matters.

Define the artifact and its evidence trail

Start with two identities instead of one mutable filename. The source record has a stable order ID and a cryptographic digest. The distributable invoice gets a new artifact ID, a digest of its own, a redaction policy version, and a reference to the authorized source record. Access to the source and access to the distributable PDF should be separate decisions.

An audit event should answer who authorized the transformation, which fields were selected, which policy and renderer versions ran, when the artifact was produced, and which digests entered and left the job. Do not put the removed values in the event. Record selectors or field classes such as customer.tax_id; logging the secret would recreate the disclosure in another system.

A minimal manifest can be represented without binding the pipeline to one PDF library:

package audit

import "time"

type Manifest struct {
    ArtifactID       string    `json:"artifact_id"`
    SourceSHA256     string    `json:"source_sha256"`
    OutputSHA256     string    `json:"output_sha256"`
    PolicyVersion    string    `json:"policy_version"`
    RemovedFields    []string  `json:"removed_fields"`
    AuthorizedBy     string    `json:"authorized_by"`
    GeneratedAt      time.Time `json:"generated_at"`
    SignatureProfile string    `json:"signature_profile"`
}
Enter fullscreen mode Exit fullscreen mode

Use a canonical serialization for any manifest that is itself signed. Ordinary map serialization is a poor signature input because representation details can vary. RFC 8785 specifies a JSON canonicalization scheme when JSON must be hashed or signed; another deterministic format is reasonable if every producer and verifier implements the same rules.

The PDF signature proves integrity and signer intent within the chosen validation policy. The manifest connects that artifact to the transformation record. Neither proves that redaction was semantically complete, so absence testing remains a separate gate.

Build removal into generation

The cleanest path for generated invoices is field-aware rendering. Construct the distributable document from approved order fields and never insert restricted values. This is stronger than finding coordinates after layout because it avoids ambiguous glyph boundaries, repeated values, and text split across drawing operations. It also makes the policy reviewable: the allowlist is code and configuration, not a set of page rectangles.

Some workflows receive an already-rendered legal invoice. In that case, a redaction engine must remove or rewrite the underlying page objects intersecting each approved region, handle images, and eliminate related text representations. Rasterizing a page can collapse its visible content, but only if the output is a new document that excludes the original objects, OCR layer, attachments, and metadata. Rasterization also sacrifices search, accessibility, selectable invoice numbers, and sometimes signature features. That is a trade-off, not a universal fix.

Make jobs idempotent. A retry with the same source digest, policy version, rendering version, and authorization should resolve to the same logical artifact rather than publish a second invoice under a new identity. The signature may include time-dependent material, so byte-for-byte equality is not always the right idempotency test. Deduplicate on the transformation request, store the resulting artifact ID, and let retries return that result.

The queue worker should move through explicit states: authorized, transforming, verified, signed, published. Publication cannot precede verification. A crash after object storage succeeds but before the database update is handled by reconciling the artifact digest and idempotency key, not by blindly generating again. Duplicate deliveries are normal queue behavior; duplicate legal artifacts should not be.

Keep the interface narrow:

package redact

import "context"

type Request struct {
    Source       []byte
    SourceSHA256 string
    Policy       string
    Fields       []string
}

type Result struct {
    PDF          []byte
    OutputSHA256 string
    Removed      []string
}

type Engine interface {
    Transform(context.Context, Request) (Result, error)
}

type Verifier interface {
    Verify(context.Context, Request, Result) error
}
Enter fullscreen mode Exit fullscreen mode

This boundary permits independent verification and makes a renderer replacement testable. It also keeps signing out of the transformer. Sign only the verified bytes, record the signed artifact digest, and prohibit later cosmetic edits. If a footer or page number must change, issue a new version through the same pipeline.

Verification is an adversarial gate

A successful API response is not evidence of safe redaction. Verification should inspect the final unsigned candidate as an attacker would. Extract text through an implementation independent from the transformer. Parse page objects and annotations. Enumerate embedded files, form fields, metadata, and optional content. Render every page and apply image or OCR checks where restricted values may have appeared as pixels. Then search for both exact forbidden values and normalized variants.

For example, a bank account ending in 0042 may be laid out with spaces, punctuation, or separate glyph runs. Tests should normalize those representations, while avoiding a rule so broad that every page containing 42 fails. Seed test fixtures with sentinel secrets designed to be unambiguous. Include one secret in ordinary text, one split across drawing operations, one in an annotation, one in metadata, and one in an OCR layer. Each fixture should fail before transformation and pass afterward.

The verifier also checks what must remain: invoice number, currency, totals, tax treatment, line-item descriptions allowed by policy, page count expectations, and accessibility requirements. An empty PDF contains no leaked account number, but it is not a valid invoice. Compare extracted business fields with the authorized projection of the order and render pages for visual regression checks.

Use multiple layers of evidence:

  1. Structural inspection confirms forbidden objects, attachments, annotations, and metadata are absent.
  2. Semantic inspection confirms forbidden values cannot be extracted while required invoice data remains correct.
  3. Visual inspection confirms no clipped totals, shifted labels, transparent overlays, or blank pages were introduced.
  4. Signature validation runs after signing and against the exact bytes selected for publication.

Count verification failures by policy version and failure class. Alert on any published artifact that lacks a matching verified event, and reconcile object storage against the audit ledger. Latency and queue depth matter operationally, but they must never turn verification into a best-effort step. Backpressure is safer than publishing an unchecked legal document.

Deployment and rollback runbook

Deploy a new redaction policy against a fixed corpus before it sees live orders. The corpus needs ordinary invoices plus hostile cases: rotated pages, unusual fonts, scanned pages, transparent content, incremental updates, form fields, attachments, and previously signed inputs. Review diffs in extracted structure and rendered pages. Promote the policy by version, canary a bounded set of jobs, and watch transform errors, verification rejections, signature validation failures, and publish lag.

Rollback means stopping publication and returning to the last approved policy for new jobs. It does not mean editing already signed PDFs or restoring a hidden layer from the distributable file. Quarantine candidates produced by the rejected version, preserve their audit records, and regenerate from the separately protected source only after authorization. If a distributed invoice is found unsafe, revoke access where the delivery channel permits it, record the incident, issue a corrected artifact with a new identity, and follow the applicable legal notification process.

There is one hard stop: if independent verification cannot account for every representation of a restricted field, do not sign or publish the invoice. A visible black box is easy to demonstrate. Verified absence is the control that matters.

References

Top comments (0)