Six commands that beat my own PDF fraud detector, ranked by how well

Every vendor in this space publishes what it catches. I have not found one that publishes what makes it stop working. So here is that second list for my own engine, measured rather than reasoned about.

The engine reads PDF structure and returns fraud signals. Nineteen families run on every document; a twentieth, issuer-mismatch, only fires when the caller supplies a baseline, so a bench that inspects a file on its own can never reach it. The six evasions below are one command each, they are all on the first page of results for "how to remove pdf metadata", and anybody selling forged statements already owns them.

The run: 98 PDF fixtures, engine 1.37.0, qpdf 12.4.0, exiftool 13.55, Ghostscript 10.07.1, and a Quartz CGPDFContext print path. One script, checked in:

npx tsx scripts/evasion-bench.mjs --in test/fixtures

The ranking, with the column that decides whether anyone does it

A family counts as silenced when every firing the bench could measure went quiet.

Evasion	families silenced, of 19	what it costs the sender
exiftool -all=	2	nothing, and it appends a revision
qpdf --linearize	1	any signature
print-to-PDF	9	the whole revision history
gs -sDEVICE=pdfwrite	11	any signature
qpdf --encrypt	13	nothing
gs -sDEVICE=pdfimage24	18	the text layer

The right-hand column is the one that matters and it is the one nobody prints. An evasion that costs the sender something is an evasion most senders will not use. Flattening every page to an image is the strongest move on the board and I have nothing that beats it, but it also breaks every downstream pipeline that reads text: no selection, no copy-paste, no extraction, no screen reader. Income parsers and accounting imports fail on arrival, before anyone reaches the question of whether the document is real.

Encryption is the interesting row, because it silences thirteen and costs nothing. An owner password does not stop anyone opening the file.

Four results I did not expect

1. A metadata strip makes four families louder

exiftool -all= on a PDF is not a delete. It is an append: a new revision in which the metadata is blank, with the original bytes left exactly where they were. So:

• incremental-updates goes from one revision to two, which is the thing it looks for.
• On a signed document the appended region sits past the end of the signed /ByteRange, which is the highest-severity finding the engine has.
• id-inconsistency fires on a file that had nothing to say before. The appended trailer keeps the original /ID[0] and writes a fresh /ID[1], so a file whose two halves matched comes out with two that do not. The strip manufactures the finding it was run to remove.
• font-anomalies reads info on a one-save file and worse on an edited one, so it went info to high on five of the six fixtures it fires on. Nothing about the fonts changed. The file simply stopped being a one-save document.

Tidying a PDF before sending it makes the report worse, not better. That is not clever detection. That is the file format.

2. "Whole-file rewrite" is not one column

qpdf and Ghostscript are both described as whole-file rewriters, and they disagree on fifteen of the nineteen rows. qpdf silences one family. Ghostscript silences eleven.

The mechanism is the difference between copying and re-rendering. A structural rewriter copies through what it does not need to understand: the Info dictionary, the XMP packet, the font objects. A re-renderer regenerates all of it from what it drew. So Ghostscript rewrites /Producer and copies /Creator through verbatim, meaning an editor that named itself in both survives a full re-render. It re-subsets fonts without merging them, so two subsets of one typeface stay two subsets. And it does not remove active content so much as move it: an /OpenAction came out the far side as an /A action on the catalog, which satisfies anyone grepping for the old key.

There is no answer to "what does a whole-file rewrite do to my signals" that does not name the tool.

3. Destroying a signature is quieter than invalidating it

Read the three signature-* rows across instead of down and the usual direction reverses.

Of nine signed fixtures that read above low as built:

• the four evasions that keep the signature object (strip, qpdf rewrite, encrypt, and the two-step below) launder 0 of 9. A qpdf rewrite takes signature-coverage to high on all thirteen signed fixtures, because invalidating a signature is the loudest thing this engine reports.
• the three that re-render the page (Ghostscript, print-to-PDF, raster) launder 9 of 9. All three signature families go quiet, because the signature does not survive rendering at all.

Which exposes the largest hole in my own engine, and it is worth stating plainly: there is no signal for "this document should carry a signature and carries none." An invalidated signature is a finding. An absent one is an ordinary PDF, and most PDFs are unsigned. So against a signed document the forger's answer is not to compose evasions harder, it is to print.

4. Printing from Chrome trips nothing, and that is deliberate

The consumer-editor list the engine matches producer strings against has around fifty entries. Quartz PDFContext is on it. Microsoft Print to PDF is on it. Skia/PDF, which is what Chrome writes when you print a page, is not.

I know how to close that and I am not going to. When we parsed 213 PDFs published by US banks, two Bank of America online-banking agreements came back as Skia/PDF m130 and Skia/PDF m135: printed straight out of a browser by whoever maintained the page. Adding Skia to that list turns a real bank's real document into a finding.

So printing from Chrome is the cheapest evasion in the table. It costs one keystroke and my engine says nothing about the tool that did it. That is a false-positive rate I chose to pay for in recall, and I would rather write the sentence than have someone discover it.

The column a forger actually runs

Every column above is one command, which answers the wrong question. Forgers compose. One pair composes into something neither half is:

exiftool -all= in.pdf
qpdf --linearize in.pdf out.pdf

The strip appends a revision and leaves the freed metadata objects in the byte stream, which is why its column reads "louder". The rewrite re-serialises from the object graph, dropping unreferenced objects and the revision chain, but on its own it carries the metadata disagreements through. Composed, the rewrite erases the revision the strip manufactured and the strip empties the stores the rewrite would have preserved.

This is not hypothetical. guillaumemeyer/watermarks-remover, an MIT-licensed tool for removing AI provenance marks, ships exactly this pair behind one command, in a Docker image, with a README, and warns in its own source that exiftool alone leaves "the original metadata bytes recoverable".

What it does to one document, in four states. This is a fictional proof of address whose page content was replaced after generation, and its clean twin:

File	Score	What the report says
clean, untouched	0 low	no signals at all
forged, untouched	70 high	incremental-updates: page content replaced after generation
forged, strip only	100 high	the above, plus the two the strip manufactured
forged, strip then rewrite	10 low	producer-fingerprint low, id-inconsistency info
clean, strip then rewrite	10 low	the same two, identically

On an unsigned document the two-step takes a real forgery from high to low, and the laundered forgery is indistinguishable from the laundered clean file.

It is also not the best evasion available, which I got wrong for about an hour on the day I first measured it. Over the 35 unsigned fixtures that read above low as built: a raster flatten launders 34, qpdf --encrypt 32, a Ghostscript rewrite 24, print-to-PDF 18, the two-step 13, qpdf --linearize alone 11, and exiftool -all= alone 0. Fifth of seven. What stands is the mechanism, and the fact that it beats either half alone.

Three things it cannot do, and they are why this is not a rout:

1. It does not produce an untouched-looking file. Both laundered files land on the same two residual signals. A PDF that declares no producing tool and has mismatched /ID halves has been through a rewriter. For document classes where the issuer always names its software, and statement and payroll pipelines do, "declares no producing tool" is the anomaly rather than the absence of one.
2. It is catastrophic on anything signed. All 13 signed fixtures came out high, including one that reads 0 low untouched.
3. It does not touch the page. The text under the black box is still there, the balance column still fails to add up, the check digit is still wrong.

The bench was counting its own failures as results

The part of this work I would most like other people to check their own harness for. It came out of the first time the checked-in bench was re-run, on 24 August, over the 92 fixtures the repository had that day.

Nine of those 92 are things a tool can legitimately refuse: one is not a PDF at all, one is already encrypted and cannot be re-encrypted, and Quartz will not open six files whose whole purpose is to be malformed. When a derivation failed there was no report to read, the summary read families?.[family] as undefined, and verdict(before, undefined) returned gone.

So a failed tool run was published as a silenced signal family. Nine cells were inflated by it. One cell, document-injection-markers under print-to-PDF, was published as gone on the strength of a single fixture Quartz refuses to open, which means that cell never had a measurement in it at all. It reads unmeasured now.

The irony is that the script already carried the rule it was breaking. Its own header says a variant whose tool is missing is UNMEASURED, "absent, not zero", because "a silently missing column is how a matrix starts lying". It enforced that per column and not per file.

Two general shapes fall out of that, and neither is about PDFs:

• An absent measurement and a negative measurement must not share a representation. undefined flowing into a comparison that returns a meaningful verdict is the whole bug, in one line.
• A rule enforced at the wrong granularity reads as a rule enforced. Nobody re-reads a guard that is present.

The engine, for what it is worth, is not responsible for any of it: running the whole bench on every build from 1.29.0 to 1.32.0 over the same fixtures produces byte-identical tables.

What I take from the table

Nothing in the unconditional set survives all six. Every one of the nineteen has at least one column in which every firing the bench could measure went quiet. The one family that never goes quiet is issuer-mismatch, and it is the one the bench cannot draw, because it compares the file to documents from the institution it claims to come from rather than reading the file's own account of itself.

That is the asymmetry in one sentence: self-description is defeated by regeneration, and comparison to an outside reference is not. The price is that you have to hold the reference.

And the operational reading, if you are the one receiving documents: a clean structural report means the bytes are clean. It does not mean the document is genuine. Those are different claims and the entire risk lives in the gap between them. Which is also why I think the only defensible output is signals rather than verdicts. "This file contains two revisions, a later revision replaced a page content stream, and the Info dictionary names an online PDF editor that the XMP packet does not" is a statement about bytes. "Our system detected fraud" is a statement about a person, and a parser does not get to make it.

---

The full matrix, cell by cell, with the three dated passes of corrections to the cells I got wrong, is at what defeats every signal. There is a free checker at tamperlens.com if you want to drop a document in and read its own report. Nothing is stored, and no account is involved.

If you run one of these evasions against a document you personally know the history of and get a result that contradicts mine, that is the comment I most want.