A redacted document that is not actually redacted is worse than an unredacted one. At least with the second kind, everybody knows what they are holding.
Law firms are now feeding thousands of pages into AI tools for review, summarization and privilege checks. That is useful. It also means old redaction habits, the ones that were already shaky with paper and PDFs, now run at machine speed. A mistake that once exposed one page can expose a whole production.
If you work in legal tech, or you are the person at a firm who gets asked "is this safe to upload?", here are the seven mistakes I see described most often, and what to do about each. Secure AI document processing for law firms is mostly a matter of avoiding these, not of buying anything fancy.
1. Drawing a black box and calling it redaction
This is the oldest mistake and it still happens. Someone places a black rectangle over a name in a PDF. It looks perfect on screen. But the text underneath is still in the file, so anyone can select it, copy it and paste it somewhere else. Court filings have been embarrassed this way more than once.
The same thing happens with scanned documents. A redacted image can sit on top of an OCR layer that still holds the original words, so the page looks clean while the searchable text tells the whole story.
The fix is to use a tool that actually removes the underlying content, then check the result by trying to copy text from the redacted area. If anything comes out, it did not work.
2. Forgetting the metadata
The visible page is only part of a document. Tracked changes, comments, embedded spreadsheets, document properties and even file names can reveal what the page hides. A memo can be perfectly redacted on screen and still carry the client's name in a comment bubble from three drafts ago.
Strip metadata as a separate, deliberate step, and do it on the final file rather than a working copy. AI tools can also pick up on metadata, so a leaked comment can end up inside a summary you never expected to contain it.
3. Redacting inconsistently
A witness is blacked out on page four, then named in a footnote on page thirty-seven. A client's account number is masked in the body but sits in full inside a table. Nobody was careless on purpose. The document was just too long, and humans do not catch every repetition.
This is one place where software genuinely helps, because a detector can find every occurrence of an identifier. But it only helps if the detection is tested properly, which brings us to the next mistake.
4. Trusting OCR on bad scans
Optical character recognition is what turns a scanned page into text a computer can read. It is also a privacy step, because it creates a searchable layer that can be copied, indexed and logged. And it makes mistakes.
A smudged scan can turn a Social Security number into a string your detection rules do not recognize. A name can break across two lines. Handwritten notes, faxes and photographed exhibits are the worst offenders. If your redaction only checks the OCR text, it can miss an identifier a person would spot on the page in half a second.
Test detection on your ugliest documents, not the clean samples. And review handwritten pages by hand, since OCR handles them far less reliably than typed text.
5. Confusing redaction with anonymization
These sound similar and they are not. Redaction removes information permanently. That is what you want for a court filing or a production, where the other side should never see the original.
Anonymization replaces identifiers with consistent placeholders, such as "Party A" or "Account 1", so that an AI model can still follow who did what. That is what you want when the goal is analysis. If you redact everything with black boxes before asking a model to review a contract, it loses the thread. If you anonymize a filing instead of redacting it, you may leave a path back to the original.
Pick the right one for the job. Many firms use both, in different places.
6. Using free converters and plug-ins without asking where the file goes
This one is not really about redaction at all, but it causes more trouble than almost anything above. A free online PDF converter, a cloud OCR site or a browser plug-in that uploads attachments can send privileged material outside the firm before anyone types a prompt. None of it feels like using AI, so nobody puts it in a policy.
ABA Model Rule 1.6(c) asks lawyers to make reasonable efforts to prevent inadvertent or unauthorized disclosure of client information. Uploading a file to a random converter, without knowing its retention or training terms, is hard to describe as a reasonable effort.
The fix is dull and effective: keep an approved list of tools, tell everyone including paralegals and staff, and make the approved route faster than the unapproved one.
7. Redacting too late, or only at filing
Many firms redact when a document is about to be filed, which is the point where Federal Rule of Civil Procedure 5.2 applies. That rule limits what can appear in court filings: the last four digits of Social Security and financial account numbers, the year of birth and a minor's initials.
But the exposure often happens much earlier, when documents are sent to an AI tool for review. By the time you reach the filing, the data may already have travelled. A better habit is to apply the same discipline before AI processing. Detect names, addresses, account numbers, case numbers and medical details, replace them with consistent tokens, and only then let a model see the text.
This is the idea behind anonymizing before inference. Because the model never receives the real identifiers, a leak in the model, the logs or the vendor is far less harmful. Re-identification then happens only for authorized reviewers, ideally inside the firm's own environment.
What good looks like
If you want a quick mental model, a safe workflow has a few steps in a fixed order.
- Classify. Tag documents by matter, client and sensitivity when they arrive.
- OCR somewhere you control. Your servers, or a contract that forbids retention and reuse.
- Anonymize before the model. Consistent tokens, so cross-references survive.
- Limit access by matter. An assistant on one case should not reach another client's files.
- Review the output. A lawyer checks every summary against the source.
- Set retention. Decide how long prompts, outputs and OCR text live, and allow for legal holds. Privilege is the reason this matters. Federal Rule of Evidence 502(b) protects against waiver for an inadvertent disclosure only if the holder took reasonable steps to prevent it and acted promptly to fix it. A documented process is much easier to defend than an associate's personal chatbot account. The ABA's Formal Opinion 512, issued in July 2024, adds a related point. Where a tool may use inputs to train itself, informed consent from the client is needed before client information goes in, and boilerplate in an engagement letter is not enough. Removing identifying details before the AI step reduces how often that question comes up, though each tool and matter still deserves its own judgment.
A five-minute test you can run today
Take one document that was supposedly redacted and run three checks. First, select the redacted area and try to copy the text. Second, open the file properties and look for comments, tracked changes and author names. Third, search the whole document for the redacted name, in case it appears somewhere else. If all three come back clean, you have a reasonable sign the process works. If any fails, you have found a problem cheaply, before opposing counsel or a client found it for you.
Where tools can help
You can build much of this yourself, and plenty of teams do. If you would rather use something purpose-built, Questa AI provides an anonymization layer aimed at legal and M&A teams. According to the company, it masks privileged and deal-sensitive information before any AI model receives it, and it works in front of the review or data room platform a firm already uses, so nothing has to be replaced.
For a longer walkthrough that covers deployment options, access controls, privilege and a set of FAQs, the guide on secure AI document processing for law firms
How Law Firms Can Protect Client Data When Using AI is worth a read.
The short version
Most redaction failures are mechanical, not clever. Boxes over live text, forgotten metadata, missed OCR errors and inconsistent masking cause more trouble than any sophisticated attack. Fix those first, move your protection ahead of the AI step, and you have done most of what matters.
Which of these have you seen in the wild? I would be curious to hear about the strangest one in the comments.
Top comments (0)