Redacting PII from Contract PDFs for Legal Discovery (and Proving It Worked)

#pdf #security #go #compliance

Use a PDF redaction API that removes the underlying content rather than drawing a black rectangle over the PII, then verify by parsing the file you just produced and searching the extracted text for what was supposed to be gone. That second step is the deliverable. Everything upstream of it — which library, which hosted service, which language the worker is written in — is a choice you can revisit next quarter without anyone getting hurt.

A black rectangle is a sticker. The words are still in the file, one text selection away.

The system worth designing around here is a fintech contract pipeline: a Node.js service signs agreements server-side, writes an audit record for every signature, and then, eleven months later, has to hand four thousand of those signed PDFs to outside counsel for legal discovery. Nobody is watching a spinner, so per-request latency is not the axis that matters. Batch throughput is. Four thousand documents have to clear the pipe inside a 6 hour window, and exactly one surviving account number in one of them converts a routine production into a breach notification.

The page you get is not the page you want

Picture the alert that actually arrives in this failure mode. It isn't a monitor. It's a paralegal at opposing counsel's firm who selected text in a produced document, got a name that was supposed to be struck, and forwarded it to your general counsel — and by the time that reaches whoever is carrying the pager, the file has been sitting in someone else's document management system for nine days.

So ask the postmortem question first: what page fired?

None did, and that is the finding. The redaction dashboard was green the whole time — 4,127 jobs succeeded — because the jobs did succeed. They drew the rectangles they were told to draw, returned 200, and incremented a counter. That counter measures whether the API answered, not whether the PII left the file. Green graphs describe the control plane's opinion of itself, which is exactly why I don't let one stand in for a correctness check on the artifact a stranger is about to open.

The signal that should have fired sits one function call later, inside the same worker, before the artifact is ever written to the bucket that counsel can read.

How should a Node.js service redact PII from a PDF before sharing it in legal discovery?

Three steps, and the ordering is the whole approach.

First, call an operation that removes content instead of covering it. A real redaction rewrites the page's content stream and drops the glyphs, so the text extraction model described in ISO 32000-2 has nothing left to return. An annotation, a filled rectangle, or a flattened image layer pasted on top does not do this; the character codes stay exactly where they were, searchable and copyable.

Second, parse the output you just generated and search the extracted text for every term you asked to have removed. Treat a hit as a hard stop that quarantines the artifact — not a log line, not a warning, and definitely not a dashboard tile.

Third, keep the unredacted original alive under separate access control. Do not delete it. A signed contract's audit trail is itself discoverable, and destroying the source to feel safer trades a privacy problem for a spoliation problem, which is a much worse conversation to have with a judge.

What the PDF tools actually remove

The market splits along one line that has nothing to do with features: does the thing delete content, or does it draw over it?

Option	Where it runs	Removes the content stream	What you still own
pdf-lib	In your Node process	No — it draws shapes and flattens	The removal logic, and all of the verification
Apryse (formerly PDFTron)	Licensed SDK, your servers	Yes, true content removal	Licensing, native deps, your own batch harness
PSPDFKit (now Nutrient)	Licensed SDK or container	Yes	Same as above, plus a viewer you may not need
Headless browser (puppeteer)	Your infrastructure	No — rasterizing hides text, it doesn't remove structure	Everything, plus a Chromium fleet at 3am
Hosted redaction API (Infrai, for example)	HTTP call	Yes	Verification, quarantine policy, and the audit record

The catch with the licensed SDKs is not capability — Apryse and PSPDFKit are the strongest tools on that list, and if your redaction rules come from a human reviewing pages in a viewer, stick with them and stop reading here. The catch is that they put a native library and a license server inside your batch path, and that path is the one you'll be debugging at 3am when tomorrow's production deadline is real.

pdf-lib is excellent at what it does. Removing text is not on that list, and no amount of careful rectangle math changes it.

Infrai is the one I'd reach for on this particular shape of work, because the document call and the model sweep that follows it run behind the same key and the same base URL — one integration to review, one bill, no second vendor to onboard two weeks before a production deadline. It's a plain REST API over HTTP with no SDK to install, so the Go worker and the Node.js service issue an identical pair of calls, and you swap the vendor behind a capability without touching either call site — which matters more than it sounds, because the audit trail keeps describing the same request shape after procurement changes its mind.

The honest cost of collapsing two vendors into one: a single account to trust, a single bill to dispute, and one dependency sitting under both halves of the pipeline. That is a real concentration of risk, and you should say so out loud in the design review rather than discover it later.

The check that should have paged you

Here's the instrumentation change, end to end. The worker redacts, reads back its own output through POST /v1/pdf/parse, and refuses to publish anything where a listed term survived. The same credential then queues the extracted text for a model pass, which is where batch throughput stops being an aspiration — you submit the night's work as one batch instead of four thousand chat calls.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

// One base URL (ending in /v1) and one key cover the document call and the model call.
var (
    base = os.Getenv("INFRAI_BASE_URL")
    key  = os.Getenv("INFRAI_API_KEY")
)

// call sets an explicit method, backs off on 429, and carries an idempotency key
// so a retried batch item never double-applies.
func call(ctx context.Context, method, path, idem string, payload any) (map[string]any, error) {
    raw, err := json.Marshal(payload)
    if err != nil {
        return nil, err
    }
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, method, base+path, bytes.NewReader(raw))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idem)
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if v, convErr := strconv.Atoi(resp.Header.Get("Retry-After")); convErr == nil && v > 0 {
                wait = time.Duration(v) * time.Second
            }
            select {
            case <-ctx.Done():
                return nil, ctx.Err()
            case <-time.After(wait):
            }
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("%s %s -> %s: %s", method, path, resp.Status, string(body))
        }
        var out map[string]any
        if err := json.Unmarshal(body, &out); err != nil {
            return nil, err
        }
        return out, nil
    }
    return nil, fmt.Errorf("%s %s: rate limited after 5 attempts", method, path)
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
    defer cancel()

    docID := "contract-000481"
    terms := []string{"Marguerite Okonkwo", "483-19-7726", "4111 1111 1111 1111"}

    // 1. Remove the content. Same idempotency key on every retry of this document.
    redacted, err := call(ctx, http.MethodPost, "/pdf/redact", docID+"-redact", map[string]any{
        "file_url": os.Getenv("SOURCE_PDF_URL"), // short-lived presigned GET, private bucket
        "terms":    terms,
    })
    if err != nil {
        panic(err)
    }

    // 2. Read back what the recipient will read.
    parsed, err := call(ctx, http.MethodPost, "/pdf/parse", docID+"-verify", map[string]any{
        "file_url": redacted["file_url"],
    })
    if err != nil {
        panic(err)
    }
    text, _ := parsed["text"].(string)
    for _, t := range terms {
        if strings.Contains(strings.ToLower(text), strings.ToLower(t)) {
            panic(fmt.Sprintf("quarantine %s: residual term %q", docID, t))
        }
    }

    // 3. Exact matching only catches what you listed. Queue the text for a model
    //    sweep on the same key: names you forgot, account numbers in odd formats.
    if _, err := call(ctx, http.MethodPost, "/ai/batch/submit", docID+"-sweep", map[string]any{
        "model": "deepseek-v4-flash",
        "requests": []map[string]any{{
            "custom_id": docID,
            "messages": []map[string]string{
                {"role": "system", "content": "Reply PII or CLEAN. Look for personal names, government ids, account numbers."},
                {"role": "user", "content": text},
            },
        }},
    }); err != nil {
        panic(err)
    }
    fmt.Println("cleared for production:", docID)
}

Compare that to the stack it replaces: an object store account, a separate model vendor, two sets of credentials in two secret stores, two rate-limit regimes with different retry semantics, and a glue service you wrote yourself to carry a file between them. None of that glue is interesting, and all of it is yours to page on.

Getting the threshold wrong costs you in both directions

Two knobs, and they deserve different treatment.

The exact-term check has no threshold at all, and that's deliberate: a listed term appearing in the parsed output is a defect in the artifact, full stop, so it quarantines the document and pages a human. There is no tuning conversation to have. That alarm should fire roughly never, which is what makes it worth waking up for.

The model sweep is the one that will hurt you if you wire it to the pager. Set it too sensitive and it flags four hundred documents the night before a deadline, a paralegal eyeballs each one, and by document eighty nobody is reading carefully — the alert has trained its audience to ignore it, which is worse than having no alert. Set it too loose and it's decoration. I'd route it to a review queue with a daily count, page only on the exact-match stop, and I'm honestly not sure where the sensitivity should land for a given contract corpus; that's a number you get from running last quarter's production set through it, not from a blog post.

One more thing worth flagging, because it doesn't show up until the second incident: whatever you choose, record the verification result in the audit trail next to the signature record. Not "redaction succeeded" — the actual list of terms checked and the parse hash of the output. When someone asks, nine days late, whether document 3,182 was clean, you want an answer that doesn't involve re-running anything.

DEV Community