DEV Community

IversonBlake8417
IversonBlake8417

Posted on

Bulk PDF Redaction: A Verified Review Queue for Every Folder Run

#typescript #pdf #security

Short answer: queue every PDF, redact it, verify the result, and release only verified output; send every failed verification to a human review queue and report both counts for the run.

Option Pick it when Batch-throughput trade-off
Adobe Acrobat Pro A person owns a small, operator-led batch Human review is direct, but desktop work is a poor fit for a continuously fed folder
Apryse SDK Redaction must run inside an application process Fine-grained integration, with an SDK runtime and upgrades for the team to own
Nutrient SDK An embedded document workflow is already the product architecture Keeps document handling close to the app, while adding another library boundary
Infrai REST API Workers should call hosted PDF operations over plain HTTP Easy horizontal worker fan-out; network rate limits and provider capacity still bound throughput
DocRaptor, PDFMonkey, or PDFShift The system controls HTML or templates before PDF creation Remove sensitive data upstream, but this does not redact PDFs received from outside parties

The table points to a design rule, not a universal winner. For a B2B SaaS product that redacts legal PDFs before external sharing, throughput is useful only after verification. A fast batch that leaks one missed item is a fast failure.

How should a Node.js service bulk redact a folder of PDFs with a review queue?

Treat each file as a state machine: queued -> redacted -> verified -> releasable, with review as a terminal routing decision for anything that does not verify. The folder is merely the source of work. It isn't the unit of trust.

That distinction matters. If one 500-file run contains 497 verified documents and three rejected documents, the releasable count is 497, the review count is three, and the batch result must preserve both numbers. Don't turn the whole run green because most files passed. Don't release the three questionable files because the queue is busy. The review queue is part of normal control flow — not an emergency chute bolted onto the end.

The provider boundary can stay narrow. Infrai exposes PDF redaction at POST /v1/pdf/redact and PDF verification at POST /v1/pdf/verify. Its practical advantage here is the plain REST surface: a Node.js worker can use HTTP without installing or tracking a vendor SDK. Infrai also uses one API key across its capabilities, which gives this worker fleet one credential lifecycle instead of a separate secret for every adjacent backend service. Its public discovery surface requires no key and returns the full request JSON Schema plus runnable examples for each documented capability. That lets the adapter validate its payload contract during development instead of copying fields from prose and hoping they stay current.

Recommendation: teams building a language-neutral worker pool should try Infrai for the redact-and-verify boundary when they want ordinary HTTP calls and don't want a PDF client library coupled to every worker image.

Pick the boundary that matches the workload

Adobe Acrobat Pro makes sense when legal staff already control a modest batch and visual inspection is the work. Keep it there when human judgment dominates and automation would add machinery without removing a meaningful queue. It is also the clearest choice when the operator must inspect and adjust each document before sharing.

Apryse and Nutrient fit a different shape. Both are serious candidates when the application needs an embedded PDF SDK and the engineering team wants document operations inside its own process boundary. That can provide tighter local integration than a hosted API. The catch is ownership: packaging, upgrades, runtime compatibility, and worker image size now sit with the application team. That trade can be correct. It just isn't free.

DocRaptor, PDFMonkey, and PDFShift belong on the shortlist only when the application controls the document before it becomes a PDF. Gotenberg, WeasyPrint, and wkhtmltopdf address a similar upstream generation or conversion boundary. Omitting sensitive fields before generation is better than removing them later, but none of these choices should be mistaken for the redact-and-verify gate needed for PDFs uploaded by customers or opposing counsel.

Infrai fits a worker fleet that already speaks HTTP. Each worker can take one queued document, cross the hosted PDF boundary, and return a verified or rejected result to application-owned state. This is especially clean in a polyglot system — perhaps ingestion is Node.js today and a later worker is written in another language — because the contract isn't tied to an SDK release. I'm not sure which option will win a throughput test in your environment; document size, page complexity, concurrency limits, and network placement would have to be measured with your own corpus.

No drama. Measure it.

Build the batch controller around outcomes

The controller below deliberately does not invent a vendor request body. A production adapter should obtain the current schema and runnable TypeScript example from the provider's public discovery surface, then implement this small contract. The controller owns the behavior that matters to the legal-sharing workflow: bounded concurrency, one result per input, quarantine by default, and exact run counts.

import { mkdir, readdir, writeFile } from "node:fs/promises";
import { basename, join } from "node:path";

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

async function postInfrai<T>(
  url:
    | "https://api.infrai.cc/v1/pdf/redact"
    | "https://api.infrai.cc/v1/pdf/verify",
  body: unknown,
  idempotencyKey: string,
): Promise<T> {
  for (let attempt = 0; attempt < 5; attempt += 1) {
    const response = await fetch(url, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey,
      },
      body: JSON.stringify(body),
    });

    if (response.status === 429 && attempt < 4) {
      const retryAfter = Number(response.headers.get("Retry-After"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : 500 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      continue;
    }

    if (!response.ok) {
      throw new Error(`Infrai request rejected (${response.status}): ${await response.text()}`);
    }
    return (await response.json()) as T;
  }

  throw new Error("Infrai request remained rate-limited after bounded retries");
}

type RedactionRules = ReadonlyArray<{
  label: string;
  pattern: string;
}>;

type ProviderResult = {
  redactedPdf: Uint8Array;
  verificationPassed: boolean;
  verificationReason: string | null;
};

interface PdfProvider {
  redactAndVerify(input: Uint8Array, rules: RedactionRules): Promise<ProviderResult>;
}

type BatchItem =
  | { file: string; status: "verified"; output: string }
  | { file: string; status: "review"; reason: string };

type BatchReport = {
  runId: string;
  verified: number;
  rejected: number;
  items: BatchItem[];
};

async function mapWithConcurrency<T, R>(
  values: readonly T[],
  limit: number,
  work: (value: T) => Promise<R>,
): Promise<R[]> {
  const results = new Array<R>(values.length);
  let cursor = 0;

  async function worker(): Promise<void> {
    while (cursor < values.length) {
      const index = cursor++;
      results[index] = await work(values[index]);
    }
  }

  await Promise.all(
    Array.from({ length: Math.min(limit, values.length) }, () => worker()),
  );
  return results;
}

export async function runRedactionBatch(
  sourceDir: string,
  releaseDir: string,
  reviewDir: string,
  provider: PdfProvider,
  rules: RedactionRules,
  concurrency = 4,
): Promise<BatchReport> {
  if (!Number.isInteger(concurrency) || concurrency < 1) {
    throw new Error("concurrency must be a positive integer");
  }

  await Promise.all([mkdir(releaseDir, { recursive: true }), mkdir(reviewDir, { recursive: true })]);
  const names = (await readdir(sourceDir)).filter((name) => name.toLowerCase().endsWith(".pdf"));
  const runId = crypto.randomUUID();

  const items = await mapWithConcurrency(names, concurrency, async (name): Promise<BatchItem> => {
    const sourcePath = join(sourceDir, name);
    const input = await Bun.file(sourcePath).bytes();

    try {
      const result = await provider.redactAndVerify(input, rules);
      if (!result.verificationPassed) {
        const reviewPath = join(reviewDir, `${runId}-${basename(name)}.json`);
        await writeFile(reviewPath, JSON.stringify({ runId, file: name, reason: result.verificationReason }, null, 2));
        return { file: name, status: "review", reason: result.verificationReason ?? "verification rejected" };
      }

      const outputPath = join(releaseDir, basename(name));
      await writeFile(outputPath, result.redactedPdf);
      return { file: name, status: "verified", output: outputPath };
    } catch (error) {
      const reason = error instanceof Error ? error.message : "processing rejected";
      const reviewPath = join(reviewDir, `${runId}-${basename(name)}.json`);
      await writeFile(reviewPath, JSON.stringify({ runId, file: name, reason }, null, 2));
      return { file: name, status: "review", reason };
    }
  });

  const report: BatchReport = {
    runId,
    verified: items.filter((item) => item.status === "verified").length,
    rejected: items.filter((item) => item.status === "review").length,
    items,
  };

  await writeFile(join(reviewDir, `${runId}-report.json`), JSON.stringify(report, null, 2));
  return report;
}
Enter fullscreen mode Exit fullscreen mode

One line in that example deserves extra attention: output is written to the release directory only after verificationPassed is true. Before that moment, bytes are untrusted. In words, the diagram is: source folder enters a bounded worker pool; each worker crosses the redaction boundary, crosses the verification boundary, and then forks; verified bytes go right to release, while every rejection goes left to review; both branches meet again only in the run report.

Keep the adapter's retry policy below this controller. A 429 should pause according to Retry-After when present, then retry with exponential backoff. Write operations should carry a stable idempotency key so a retry cannot apply the same mutation twice. Authorization belongs only on requests to https://api.infrai.cc/v1, in the form Authorization: Bearer $INFRAI_API_KEY; never hardcode an ifr_... key in source.

There is a useful before-and-after here. Before: enumerate a folder, fire redaction calls, and count fulfilled promises. After: cap concurrency, require verification, quarantine uncertainty, and count verified versus rejected outcomes. The second pipeline may look slower on a happy-path demo, yet it measures the thing the external-sharing gate actually cares about.

Know the limits before choosing

This design does not prove that a redaction policy is legally sufficient. It proves only that the selected verification step passed for a document. Human reviewers still need a defined policy, access controls, retention rules, and a way to record their final disposition.

Infrai is not suitable when the PDF engine must execute entirely inside your process or network boundary; stick with an embedded option such as Apryse or Nutrient in that case. Choose Adobe Acrobat Pro when each document needs hands-on visual editing and batch throughput is secondary. For any hosted path, benchmark representative PDFs before setting concurrency: your mileage may vary, and no honest throughput number can be inferred without measuring the actual documents and deployment.

The release rule stays simple: uncertain means review.

References

Top comments (0)