DEV Community: Marcus Hale

Diffing Two PDFs Is Three Different Problems Wearing One Name

Marcus Hale — Thu, 15 Oct 2026 15:04:35 +0000

A vendor sent contract v7 and swore only one date had changed since v6. Diffing the two files took me an afternoon the first time I tried it naively, because "compare two PDFs" sounds like one operation and is actually three, stacked — and each layer lies to you in its own way if you ask it the wrong question.

Layer one: the text diff (fast, and noisier than you expect)

Extract both texts and diff them:

pdftotext -layout v6.pdf - | head
pdftotext v7.pdf v7.txt && diff v6.txt v7.txt

You will find the changed date. You will also find forty other "changes": a ligature that extracted as one character in v6 and two in v7, a table whose reading order flipped between exports, hyphenation that moved when a font was substituted, page headers that appear mid-stream. PDF stores text as positioned glyphs, not as a reading order; two exports of the same visual document can legitimately extract differently. A text diff answers "did the extracted character stream change," which is a proxy for the question you actually have, and the proxy is noisy in proportion to how layout-heavy the document is.

Taming it: normalize before diffing (collapse whitespace, strip running headers/footers, sort nothing — order matters), and diff with -w -B as a first pass to see whether the signal survives the noise floor. If the diff is still all weather and no climate, the change you care about is not at this layer.

Layer two: the render diff (what a human would notice)

Render both PDFs to images at the same DPI and compare page by page:

pdftoppm -png -r 100 v7.pdf page

Pixel-diffing catches what text extraction cannot in principle see: the moved logo, the table that reflowed, the signature that appears in one and not the other. Its blind spot is the mirror image of layer one's — it will flag a document whose text changed invisibly (a corrected metadata date, a re-encoded but identical-looking page) as "no change," and it drowns in false positives if the two files were rendered by different engines at slightly different sizes. Align page counts first; if v7 has one more page than v6, every subsequent page "changed," and your diff needs page matching, not page pairing.

For one-off checks I use a PDF compare tool, which does the side-by-side render comparison without the shell pipeline; in CI, the pdftoppm route plus an image-diff threshold is perfectly serviceable.

Layer three: the structural diff (what actually changed in the file)

Sometimes neither text nor pixels changed and the file still did: fonts re-embedded, metadata rewritten, a form field's value set, an annotation added. qpdf --json or a pikepdf walk will show object-level differences. This layer matters when the stakes are evidentiary (did the signed file change?) rather than editorial.

Which layer is your question?

"Did the wording change?" → text diff, normalized, noise tolerated.
"Would a reader notice a difference?" → render diff.
"Is this bit-for-bit the same document in disguise?" → structural diff.

The vendor's date had indeed changed — along with a payment term on page nine that the text diff surfaced only after normalization stripped the headers out of the way. Three layers, one afternoon the first time, ten minutes ever since. The trick was never the tools. It was knowing which question I was asking.

Your HTML-to-PDF Invoices Look Different in Every Tool. Here's Why.

Marcus Hale — Thu, 15 Oct 2026 14:32:11 +0000

The first time I generated invoice PDFs from HTML, the staging version looked perfect and the production version quietly moved the totals block to page two, alone, like it had been exiled. Same HTML, same CSS, different converter. That day I learned the fact that governs all HTML-to-PDF work: there is no single "HTML to PDF." There are several rendering engines with different subsets of CSS, different default margins, and different ideas about when a page should break. The output differences are not bugs in your markup. They are the engines, being themselves.

The three families you will actually meet

Browser engines (headless Chromium via Puppeteer/Playwright, or hosted tools built on it) render your page the way Chrome does — modern CSS, flexbox and grid, web fonts — then print it. If your invoice looks right in Chrome's print preview, this family will treat you best.
Dedicated layout engines (WeasyPrint, Prince) implement the CSS paged-media spec more faithfully — running headers, page counters, cross-references — but lag on newer layout CSS.
Legacy converters (older wkhtmltopdf builds and friends) use ancient WebKit. Flexbox is partial, grid is a rumor, and every project eventually has the "why is this column the whole page" incident.

My current one-off jobs go through an HTML-to-PDF converter built on a browser engine; in pipelines I pin one engine in the dependency file and treat changing it as a migration, with visual diffs, not an upgrade.

The CSS that actually controls a printed page

Most invoice-print bugs are five properties wearing trench coats:

@page { size: A4; margin: ... } — set the paper and margins in CSS, or accept each engine's defaults, which differ. US Letter vs A4 alone explains a class of "it fits here but not there."
break-inside: avoid on blocks that must not split — totals, signature areas, table rows groups. Without it, every engine breaks content where the page happens to end.
thead repetition — browsers repeat a table's header group on each printed page if you use real thead markup; divs pretending to be headers do not repeat, which is how invoices end up with a page of orphan numbers.
Print color — backgrounds vanish by default in print CSS (print-color-adjust: exact restores the ones you actually want, like a shaded header row).
Fonts the engine can see. A webfont your page loads from a CDN may or may not be fetched by the converter; self-hosting the font or embedding it removes the coin flip, and invoices are not the place for coin flips.

The discipline that ends the surprises

Keep one golden sample — the nastiest invoice you produce: multi-page, long item descriptions, a discount, a tax line, a note in small print — and render it with your pinned engine on every change. Compare page count and eyeball page two; totals drift is visible in seconds. And generate the PDF at the moment of issue with the same engine that renders your preview, so the customer never sees a layout you have not.

HTML-to-PDF is a good architecture — one template, styled with skills your team already has. It just punishes the assumption that print is a screenshot of the screen. Treat the page as a separate medium with its own five rules, pin your engine, and the exiled totals block stays home.

Get the Original Images Back Out of the PDF (Don't Screenshot Them)

Marcus Hale — Thu, 15 Oct 2026 13:59:53 +0000

A designer sent me a finished brochure as a PDF and, a week later, asked for "the product photos from it" for the website. The originals were on a laptop she no longer had. The photos existed in exactly one place: embedded in that PDF. What she did next is what most people do — opened the PDF, zoomed in, and took screenshots. What she got was screen-resolution crops with viewer borders, at whatever size her window happened to be.

The images inside a PDF are files in their own right, stored in the document's structure. Extraction pulls those stored files out untouched. Rendering (which is what a screenshot is) re-creates a picture of the page at your screen's resolution. For anything you plan to reuse, the difference is the whole game.

Extraction vs. the other things people do

Screenshot / snipping tool: captures pixels at display resolution, includes whatever zoom and cropping you happened to have, and bakes in any downscaling the viewer did. Fine for showing someone where a button is; wrong for source material.
Export page as image: renders the whole page at a resolution you choose. Good for slides and previews, but you get the page — layout, margins, text and all — not the photo.
Extract images: walks the PDF's object structure and writes out each embedded image stream as its own file, at its stored resolution and in (or near) its stored format. This is the one that gives you back the closest thing to the originals.

I use an extract-images tool for the one-off jobs; in a pipeline, pdfimages -all document.pdf out/ from the poppler suite does the same thing and lists what it found with pdfimages -list before you commit.

What "original" honestly means here

Extraction gives you the image as stored in the PDF, which is not always the image as photographed. Three realities to set expectations:

The PDF may already hold a downscaled copy. Many generators shrink images to a target DPI when the PDF is made. If the brochure was produced for print, you are in luck — print pipelines store 300 DPI. If it was produced for email, the embedded photo might be 800 pixels wide and no tool can invent the rest.
Color spaces travel with the file. Print PDFs often store images as CMYK. Extracted as-is, they can look wrong in browsers until converted to sRGB. Check one image before batch-converting two hundred.
Masks and soft transparency extract as separate pieces. A photo with a soft-edged cutout may come out as an image plus a grayscale mask. That is not corruption; it is how the PDF stored it, and the pair recombines in an image editor.

The thirty-second check before you promise anything

Run the list first (pdfimages -list, or the count and dimensions any extraction tool shows you) and read the resolutions. If the stored images are at or above the size you need, extraction will make you look like a hero. If they are below it, say so now — and go ask for the originals while someone may still have them, instead of delivering upscaled screenshots with confidence.

The PDF is not a dead end for assets. It is a container that happens to be carrying files — and on the day the originals are gone, it is the only backup nobody remembered making.

Archiving Invoices as PDF/A: What the Format Forbids, and Why Your Pipeline Should Care

Marcus Hale — Thu, 15 Oct 2026 13:16:04 +0000

A client in Germany asked me last year whether the invoices I send will "still open in ten years." It was not small talk. German bookkeeping rules expect business documents to stay readable and unaltered for a decade, and "it's a PDF" is not the same as "it's an archival PDF." The format designed for that promise is PDF/A — and the first time you convert real-world invoices to it, several of them fail, for reasons that teach you a lot about what an ordinary PDF quietly depends on.

What PDF/A actually is

PDF/A is a restricted profile of PDF (ISO 19005). The restriction is the point: an archival file must be completely self-contained, because anything it references externally — a font on someone's machine, a color profile on a server, content fetched at view time — is a thing that can be gone in ten years. So PDF/A requires all fonts embedded, device-independent color, no encryption, no JavaScript, no audio/video, no external references. If a file validates as PDF/A, any conforming reader, decades from now, should render it the way it looks today.

The levels matter when you choose a target. PDF/A-1 is the oldest and strictest (and has no support for transparency or layers). A-2 allows those and is the common default for business documents. A-3 goes one step further and permits embedded files inside the PDF — which is how the ZUGFeRD/Factur-X e-invoicing formats carry their machine-readable XML inside a human-readable invoice. If a client says "we need PDF/A," the follow-up question is always "which level," and for e-invoicing in the EU the answer is usually A-3 for a reason that has nothing to do with archiving.

Why real invoices fail conversion

In my experience, in order of frequency:

Unembedded fonts. The invoice template uses a font the generator references but does not embed. It renders fine on the machine that made it and substitutes silently everywhere else. This alone fails most first attempts.
Encryption or permissions flags left on by an earlier tool in the chain — PDF/A forbids encryption outright, including the "you may view but not copy" kind.
Device-dependent color (plain RGB with no output intent) — the converter usually fixes this by embedding a standard profile, but only if you let it.
Transparency and blend modes in logos and stamps, which fail A-1 and pass A-2. Choosing the level is choosing how much of your existing artwork survives.

The pipeline habit

Conversion belongs in the pipeline, not in someone's Friday afternoon. When an invoice is issued, the same job that produces the PDF should produce the archival copy and validate it — conversion without validation is a hope, not a control. I run invoices through a PDF-to-PDF/A converter for one-offs; in an automated pipeline the equivalent step is whatever converter you host, followed by a validator (veraPDF is the reference open-source one) whose report gets stored with the file. The validation report is the artifact your future self, or an auditor, actually wants.

Two operational notes. Keep the original alongside the archival copy: conversion can alter appearance subtly — color conversion is the usual suspect — and the pair costs little storage. And convert at issue time, not in a bulk job years later, because the fonts and templates that make conversion easy are the ones you still have today.

"Will it open in ten years" turns out to be an engineering question with a testable answer. PDF/A is how you get to say yes and show the report.

Your PDFs Are Leaking Metadata (and Nobody Checks Before Sending)

Marcus Hale — Thu, 15 Oct 2026 12:45:03 +0000

A vendor once sent me a polished proposal PDF. The document info told a different story than the cover page: the author field carried an employee's full name, the title was an internal codename with a client's name in it, and the creator string named a tool chain they would probably not have chosen to advertise. None of it was secret, exactly. All of it was information they never decided to share — it just rode along in the file.

PDFs accumulate metadata at every step: the authoring app writes a title, author, and creator; generators add producer strings and timestamps; some pipelines embed custom fields with internal IDs. When a file leaves your machine, all of that goes with it. Cleaning it is a two-minute job, but only if you look first.

What is actually in there

The standard fields are Title, Author, Subject, Keywords, Creator, and Producer, plus creation and modification timestamps. Any PDF reader shows them under document properties — no special tooling required. The leaks that matter in practice:

Names. Author fields with personal names on documents meant to be anonymous (blind reviews, RFP responses, anything submitted for neutral evaluation).
Internal vocabulary. Draft titles, codenames, ticket IDs, and client names from an earlier use of the file as a template.
Timestamps that contradict the story. A "final" report whose modification date is twenty minutes after the version you sent last week is fine. One whose creation date predates the data it claims to present gets questions.
Toolchain fingerprints. Creator and Producer strings announce exactly which software made the file — occasionally useful information for someone probing what to exploit, always free information you did not mean to give.

The cleaning pass

Before a document goes outside, I run it through a metadata removal tool — PDFLean's is the one in my routine; the feature exists in most serious PDF toolkits, and exiftool -all= file.pdf does it locally if you prefer the command line. The pass is mechanical:

Open the properties of the outgoing file and actually read the fields. Cleaning without looking means you will never learn what your pipeline writes by default.
Strip everything, or set the fields deliberately: a proper title and your company name as author is often better than a suspicious blank, because a blank says "scrubbed" while a title says "published."
Re-open the cleaned file and confirm the fields are what you intended — and that the document itself is untouched: same pages, same text, same size class. Metadata removal should never alter content; if the page count changed, you used the wrong tool or the wrong mode.

Fix it at the source, too

If your generator writes bad metadata by default — and most do, usually the developer's name or a framework string — the durable fix is one configuration change in the generation step, not a lifetime of manual scrubbing. Set the title from the document's real title, the author to the organization, and let the pipeline produce clean files from birth. The manual pass then becomes a spot-check for the files that arrive from elsewhere: vendor documents you forward, scans from other people's tools, anything whose provenance you did not control.

Metadata is the least glamorous part of a PDF. That is exactly why it leaks: nobody's checklist has a line for it. Add the line. Two minutes of looking beats explaining to a client why their codename was in the file you sent them.

Stop Sending the Whole 90-Page PDF: Split Out the Pages They Actually Need

Marcus Hale — Thu, 15 Oct 2026 12:09:22 +0000

A client asked me for "the warranty section" of a product manual last month. The manual is 90 pages. What I almost did — what I have done for years — is attach the whole 90 pages and write "see page 61." What I did instead took about forty seconds: split out pages 58–64 and sent a 7-page file with exactly the section in it.

Splitting PDFs by page range is one of those operations that sounds trivial and still eats time, because the friction is never the split itself. It is figuring out the right range, verifying you got the right pages, and doing it again for the next document. Here is the workflow that made it a habit instead of a chore.

The range is the hard part

"Pages 58–64" assumes the PDF's page numbers match the numbers printed on the pages. They often do not: covers, unnumbered front matter, and a table of contents shift everything by two or three. Before splitting anything, open the document, scroll to the first page you need, and read the viewer's page counter — that number, not the printed folio, is what every split tool wants. Note the start and end from the viewer, then add nothing. Most "my split came out wrong" stories are an off-by-two nobody checked.

The split itself

I use the split tool on PDFLean for this: upload, enter the range, download the extracted pages as their own file. The same operation exists in most PDF toolkits — qpdf input.pdf --pages . 58-64 -- out.pdf does it from the command line, and if you do this daily, scripting it with fixed ranges per document type is worth the ten minutes. Two modes to know the difference between:

Extract one range gives you a single new PDF with just those pages. This is the 90% case: one section, one recipient.
Split into parts cuts the file at intervals — every 10 pages, or at bookmarks. That is for distribution workflows (one chapter per reviewer), not for answering a question.

Verify like you mean it

A split that "ran successfully" can still be wrong, and you will not hear about it — your client will just read the wrong section politely. Thirty seconds of checking:

The output's page count equals end − start + 1. If it does not, the range was interpreted differently than you expected.
Open the output and look at the first and last page. They should be the pages you saw in the viewer, not the pages whose printed numbers match your range.
Check the file size is sane. A 7-page extract that is 80% of the original's size means the source carries heavy shared resources (fonts, images) — harmless, but worth knowing before you promise someone a "small" attachment.

When splitting is the wrong tool

If the recipient needs the pages in a different order rather than a subset, that is a rearrange job. If the same extract goes out repeatedly with fresh data each time — a monthly statement section, say — the bottleneck is regenerating the source, and the split should be the last step of that pipeline, scripted, not a manual errand. And if what they actually need is one table as data, no page range fixes that; it is an extraction problem wearing a splitting costume.

The whole habit reduces to one sentence: send people the pages that answer their question, numbered the way their viewer counts them, and check the two ends before the file leaves. The 90-page attachment with "see page 61" is how documents get read by nobody.

Your Scanned Archive Is a Pile of Pictures — OCR It Before You File It

Marcus Hale — Thu, 15 Oct 2026 11:28:38 +0000

A client emailed me last month asking for the termination clause from a contract we'd scanned in 2019. I opened the PDF, hit Ctrl+F, typed "termination" — nothing. Tried "notice period" — nothing. The document was 40 pages long, and search insisted it contained no words at all.

It wasn't wrong. The scan was a picture of a contract, not a contract. Every page was a flat image with no text layer underneath, which means no search, no copy-paste, no quoting a clause into an email without retyping it. Multiply that by a few years of scanned invoices, signed agreements, and receipts, and my "archive" was really just a very well-organized pile of photographs.

The two-minute check

Before I fixed anything, I checked which files were actually broken. Plenty of PDFs that come out of modern office software already have a text layer — scanning is only one way a PDF gets made. The quick test: open the file, try to select a sentence with the mouse. If you can highlight words, it has text and you're done. If your cursor just drags a blue rectangle over the whole page like it's selecting an image, the page is image-only.

I ran that test across my archive folders and found the pattern I expected: everything that had ever passed through a scanner or a phone camera was image-only, and everything born digital was fine. The fix only needed to touch the first group.

The workflow I settled on

1. Gather the scans in batches. I process by month — one folder of invoices, one folder of contracts at a time. Batching matters because it keeps the output naming sane and because OCR is the slow step; I'd rather queue a folder than babysit files one by one. When a batch starts as loose phone photos rather than a PDF, I combine them first with a batch image step (image-batch is the one I use on PDFLean) so each document becomes a single file before recognition.

2. Run OCR to add a text layer. The tool I use (PDFLean's local OCR) builds what people call a dual-layer or "sandwich" PDF: the original scan stays exactly as it looks, and an invisible layer of recognized text sits underneath. Nothing about the document's appearance changes — signatures, stamps, and layout are untouched — but Ctrl+F, text selection, and copy-paste all start working. Because the OCR runs locally in the browser, the files never leave my machine, which matters when the folder is full of client contracts.

3. Verify before archiving. I spot-check three pages per document: search for a word I can see on the page, then copy a paragraph out and read it. Then the file goes into the archive with a name a human can guess — date, counterparty, document type — because now that the content is searchable, the filename matters less than it used to, but future-me still appreciates the help.

4. Archive the searchable copy, keep the original. I never overwrite the raw scan. Disk is cheap; re-scanning a paper document someone shredded in 2021 is not possible.

Where OCR honestly struggles

Recognition is good on clean, straight, printed pages. It degrades in predictable ways, and it's worth knowing the failure modes before you trust search results:

Skewed pages. A scan fed through a document feeder at a slight angle will still OCR, but error rates climb. Straightening first helps more than any setting in the OCR step itself.
Stamps and seals over text. A big red "PAID" stamp across an invoice total is charming on paper and destructive in OCR — characters underneath it are gone, and no tool invents them back. Search will miss those spots.
Handwriting. Marginal notes, handwritten dates, initials in boxes: treat recognition of these as a pleasant surprise, not a feature. In my batches, printed text is reliable and handwriting mostly isn't.
Low-resolution phone photos. Shadowed, 100-DPI captures of receipts produce text layers with typos. Fine for finding the file again, risky for quoting numbers out of it. For anything financial, I still eyeball the figure against the image.

None of this makes OCR not worth doing — a text layer that's 95% right turns "open and skim 40 pages" into a ten-second search. It just means search is a finding aid, not a proofreading guarantee.

A note on page counts

One practical thing I learned the hard way: OCR is usually the metered step in most toolkits. Free allowances for recognition tend to be counted in pages and sized for occasional use, while the paid tiers raise the page allowance rather than changing what the tool does. A year of my invoices runs to several hundred pages, so instead of feeding the whole archive through at once, I OCR in monthly batches as part of filing, and big backlogs get their own sitting. If you're evaluating any tool for this, the page allowance per tier is the number to compare — not the feature list, which is identical.

The archive test now: that 2019 contract answers Ctrl+F in under a second. It took an afternoon to fix six years of scans, and it's the highest-leverage afternoon I've spent on my own filing system.

The Weekly Packet Problem: Merging Three PDFs in the Right Order, Every Single Time

Marcus Hale — Thu, 15 Oct 2026 09:06:10 +0000

Every Friday afternoon I send one client the same thing: a delivery packet. One file, three sources — a cover letter from my documents folder, the main report out of a reporting tool, and an appendix of signed pages the client sends back to me. Three producers, three PDF "dialects," and one merge standing between me and the weekend.

It ran fine for months. Then one Friday the appendix landed before the report. The client replied, politely confused, asking why the packet opened with signature pages and why every page reference in my cover letter was wrong. Nothing was corrupt. The merge had simply sorted the files alphabetically — appendix.pdf beats report.pdf every time — and I had clicked through without checking. That one cost me an apologetic re-send and a Monday-morning call to walk through which pages were actually final.

The parts don't help each other. The report comes out of the generator as clean A4 with a proper outline. The signed appendix is phone scans: image-only pages, mixed sizes, one of them reliably sideways. The cover letter is a single page with no bookmarks at all. None of that is a problem for a merger — but it means the merged file is the first moment all three are forced to behave like one document, and it's the last moment I can cheaply catch it if they don't.

Order is not cosmetic

When the parts come from one generator, merging is boring. When they come from three, order is load-bearing:

Printed page numbers. The report's footer says "Page 12 of 34." That math only works if the cover letter is exactly one page in front of it and the appendix starts exactly where the report ends. Slip one part and every printed number in the packet is a lie.
Bookmarks. Each source carries its own outline. Merge tools concatenate those outlines in file order, so a wrong order doesn't just move pages — it promotes the appendix's bookmarks above the report's chapters in the sidebar, and the packet looks misassembled even where the pages are fine.
Cross-references. My cover letter says "sign-off sheet on the final page." Final means final. If the appendix isn't last, that sentence points at the wrong page and I look sloppy in the one document whose entire job is looking careful.

The checklist I now run on every merged packet

The fix wasn't a better merger. It was refusing to send anything I hadn't verified. Sixty seconds, three checks:

1. Page count equals the sum of the parts. Not "looks about right" — equals. I keep a tiny script next to the packet folder:

pdfinfo cover.pdf report.pdf appendix.pdf | grep Pages   # 1, 26, 7
pdfinfo packet.pdf | grep Pages                          # must say 34

2. Click the bookmarks. First, middle, last. Each one should land on the page its label promises. This catches outline scrambling, off-by-one shifts, and the classic "appendix chapter points at report page 3" error that a page count can't see. While I'm in there I also glance at the first appendix page — a scan that merged in sideways or at postcard size announces itself immediately, and it's much cheaper to spot now than in a client's reply.

3. File size makes sense. A merged file should weigh roughly the sum of its parts, minus a little overhead. Five times bigger means something re-encoded the images. Half the size means something got dropped or downsampled. Either direction is a stop-and-look, not a send. I keep the three source files until the client confirms receipt, so a bad merge is a two-minute redo rather than a reconstruction project.

Why the merge itself is not in my pipeline

The obvious developer move is a merge step in code — pikepdf does it in six lines, and I already use it elsewhere. But this packet is assembled once a week, by a human, after eyeballing the parts. Automating the merge means my pipeline now owns the outline quirks of three different PDF producers, forever, across every library upgrade — for a step that was never the bottleneck.

So the split is deliberate: the merge happens in the browser, where I can drag the three files into order and see the page thumbnails before committing — I use the merge tool on PDFLean for exactly that drag-order-and-preview step — and the verification is what stays scripted, because verification is the part humans skip when they're in a hurry.

That's the general shape I've landed on for document chores: automate the checks, keep a human on the one irreversible click. The packet goes out in the right order now — cover, report, appendix — and Friday ends when it should.

The Filled Form You Sent Is Still Editable — Flatten It Before It Leaves Your Pipeline

Marcus Hale — Thu, 15 Oct 2026 06:48:26 +0000

A client operations lead forwarded me a "completed" onboarding form last week with a note: "Signed and finalized, please archive." I clicked into the tax ID field and typed a character. It accepted it.

The form looked finished. Every field was filled, the signature was in place, it rendered perfectly. But nothing about it was locked — every AcroForm field was still live, which means anyone downstream could change a bank detail, a date, or a name, and the archived copy would show no trace of it. If your system generates, collects, or archives filled PDF forms, this is probably sitting in your pipeline right now.

Why a filled form is not a finished form

A PDF form field is not text on a page. It is an interactive annotation floating above the page, with its own value, appearance stream, and editing affordances. Filling a field changes the value; it does not merge anything into the page content. That separation is what makes forms reusable — and it is exactly what makes a "final" filled form fragile:

Any PDF editor can still change the values. No password, no special tooling.
Viewers render fields inconsistently. Some mobile and in-browser viewers show stale appearance streams, so the archived copy can display different values than the ones submitted — the classic "it printed blank / it printed the old number" bug.
The data layer and the visual layer can disagree. Extract the field values programmatically and you may get one answer; read what is printed on the page and you get another.

The 30-second check

Before you trust an archived form, count its live fields. With pikepdf (Python) it is three lines:

import pikepdf

pdf = pikepdf.open("onboarding-filled.pdf")
fields = pdf.Root.get("/AcroForm", {}).get("/Fields", [])
print(f"live form fields: {len(list(fields))}")

Or with qpdf, dump the structure and look for field objects:

qpdf --qdf --object-streams=disable onboarding-filled.pdf - \
  | grep -c "/FT "

If the count is above zero, the document is still a form, whatever the filename says. In the case that landed on my desk, the count was 23 — including the bank account field.

What flattening actually does

Flattening takes each field's rendered appearance and bakes it into the page's content stream as ordinary graphics and text, then removes the interactive field. Same for annotation-type content like stamps and form widgets. After flattening:

The document renders identically everywhere, because there is only one layer left — the page itself.
There is nothing to edit, tab into, or extract. The values are ink, not data.
File behavior stops depending on which viewer the recipient happens to use.

The trade is real and worth stating out loud: flattening is one-way, and it destroys machine-readable field data. If anything downstream still needs to pull values out of the form (an import job, an audit extraction, your own CRM sync), run that extraction first, keep the unflattened original in your internal store, and flatten only the copy that leaves the building or goes to the archive.

Where this belongs in a pipeline

The pattern I settled on for generated and collected forms:

receive / generate filled form
  -> extract field values to the system of record
  -> store the original (still a form) internally
  -> flatten a copy
  -> verify: field count on the copy == 0
  -> archive / send the flattened copy

Step 5 is the one people skip, and it is the only one that proves anything. Flatteners differ in how they handle annotations, optional content, and signature appearances; a zero field count plus a visual diff against the original is the minimum bar before the copy replaces anything.

For one-off files — a form a client sends you, a contract that came back "final" — running a pipeline is overkill. I used the flatten tool in PDFLean (pdflean.com/tools/pdf-flatten) on that onboarding form, then ran the field-count check above on the result: 23 fields in, 0 out, and the page rendered pixel-for-pixel the same in the two viewers I test with. Process, verify, archive — in that order.

Checklist before any filled form leaves your hands

Count the live fields (commands above). Non-zero means not final.
Extract any field data you still need — after flattening it is gone for good.
Keep the unflattened original in internal storage; flatten the outgoing copy.
Verify the flattened copy: field count zero, and open it in a second viewer to confirm it renders the same.
Treat signature fields with extra care — flatten the signed copy only after the signature itself has been applied and validated, never before.

The form that started this post is archived now — as ink, not as a form. Nothing about how it looks changed. Everything about whether it can quietly change later did.

Your Generated PDFs Open Slowly in the Browser? They Are Probably Not Linearized

Marcus Hale — Wed, 14 Oct 2026 16:13:45 +0000

Your Generated PDFs Open Slowly in the Browser? They Are Probably Not Linearized

Last month a user emailed me about a report my little side project generates: "The PDF takes forever to show the first page. I thought it was broken."

The file was 4 MB. Not huge. But the in-browser preview sat on a spinner for several seconds before page one appeared, and on hotel Wi-Fi it was worse. The PDF itself was fine — it opened instantly in a desktop reader once fully downloaded. That was the clue: the browser could not render page one until it had downloaded essentially the whole file.

This post is the diagnosis and the fix, because if your app generates PDFs (invoices, reports, statements, tickets), your users are probably hitting the same thing.

The actual problem: byte order, not file size

A regular PDF is laid out for a writer, not a reader. The cross-reference table and page resources can sit anywhere in the file, objects for page one may be scattered from byte 0 to byte 4,000,000, and a viewer that wants to draw the first page has to go hunting — which, over HTTP, means range requests all over the file or just downloading everything.

Linearization (Adobe calls it "Fast Web View") restructures the file specifically for this:

Page one's objects are moved to the front of the file, in order.
A hint table up front tells the viewer exactly which byte ranges it needs for each page.
The viewer can then fetch just the first page's bytes with an HTTP range request and render immediately, pulling later pages on demand as the user scrolls.

Same content, same visual result, completely different first-paint behavior. The spec detail that matters: the file's first line carries a linearization marker in the header comment, and you can check for it:

# quick check: is this PDF linearized?
head -c 1024 report.pdf | grep -a "Linearized"
# or with qpdf, if you have it installed:
qpdf --check report.pdf | grep -i linearized

If those come back empty, your generated PDFs are not linearized — no matter what library produced them. Most generators (including the very common ones) do not linearize by default, because it is a second full pass over the file.

What I measured on my own file

Personal test, one file, throttled connection (Chrome DevTools "Slow 4G" profile), before and after linearizing the exact same generated report:

	Before	After linearization
File size	4.1 MB	4.2 MB (slightly larger — hint tables cost bytes)
Bytes needed to render page 1	~4.1 MB (effectively all of it)	~180 KB
Time to first page on screen	~3.5 s	under 0.5 s

Two things worth noting from that table. First, the linearized file got marginally bigger, and that is normal — you are trading a few percent of size for page-at-a-time delivery. Second, the win is not "the file loads faster"; it is "the first page renders after ~4% of the bytes." On a fast office connection nobody noticed the problem. On the connections your users actually complain from, it is the whole experience.

Your numbers will differ — this was my file, my hosting, my throttle profile — but the mechanism is deterministic: if page one needs the whole file, slow networks will feel it linearly.

The fix, at three levels

Level 1 — your generator. Some toolchains can emit linearized output directly or via a post-processing flag. If yours can, turn it on in the generation pipeline and be done. The post-processing step in pseudo-code is trivial; the cost is one extra pass per file:

generate report.pdf
linearize report.pdf -> report-web.pdf
serve report-web.pdf (with a server that honors Range requests)

Level 2 — your server. Linearization only pays off if your hosting actually answers HTTP Range requests with 206 Partial Content. Most static hosts and CDNs do. Check yours:

curl -sI -H "Range: bytes=0-1023" https://your-host/report.pdf \
  | grep -iE "HTTP/|accept-ranges|content-range"

If you see 200 instead of 206, the browser will fall back to downloading everything and the linearization buys you almost nothing. Fix the serving layer first.

Level 3 — one-off files. For PDFs that already exist (a backlog of generated reports, files clients send you, anything not flowing through your pipeline), you just need a linearizer. I used the one in PDFLean — the dedicated tool page is pdflean.com/tools/pdf-linearize — dropped the original report in, downloaded the processed copy, and verified it with the head/grep check above before swapping it into my test. For a one-off fix or a sanity check before you automate anything, that is the fastest path: process, verify the marker is there, re-measure.

The checklist I now run for generated PDFs

Generate a sample with production-like data (empty test files hide this — you need real length).
Check the linearization marker (command above).
Confirm the host returns 206 for a range request.
Throttle the connection and time first-page render, not full download.
Only then argue about compression — size and structure are separate problems, and structure is the one that blocks first paint.

That user who emailed me never wrote back, which in support terms is the best possible outcome. The report did not get smaller or prettier. It just stopped making people wait for page one.

The PDF Review Loop That Finally Stopped Eating My Afternoons

Marcus Hale — Wed, 14 Oct 2026 08:00:40 +0000

Most of the contract work I do for clients ends the same way: a PDF goes out, and a PDF comes back covered in comments. Highlights with no note attached. A sticky note that just says “fix this?” on page 9. Three people commenting on three different versions, because the file in the email thread is never the file in the shared drive.

The tools were never the problem. The problem was that “review” had no rules, so every round produced a new pile of comments I had to archaeology my way through. Here’s the loop I run now. It’s half conventions, half a small script, and it cut my review rounds from four-ish to usually two.

Rule 1: One marked copy, named like it means it

There is exactly one file that accepts comments: contract_CLIENT_review.pdf. Not v7, not final-FINAL, not the one attached to the third reply in the thread. When a new round starts, I archive the previous marked copy as contract_CLIENT_review_r1.pdf and the working name stays the same.

This sounds trivial and it eliminated the single biggest time sink: merging comments from parallel versions. If someone sends me comments on a different file, those comments don’t exist until they’re redone on the one marked copy. People adapt to this rule in about one round, because redoing comments is more annoying than following it.

Rule 2: Three annotation types, three meanings

I ask reviewers to use only three things, each with a fixed meaning:

Highlight + note = a question. (“Is this the rate we agreed in March?”) Questions get answered in a reply note, not by silently editing the text.
Strikethrough with replacement text = a proposed edit. If you want a clause changed, strike it and type the replacement. No “maybe soften this paragraph somehow” — that’s a question, use a highlight.
Sticky note starting with DECISION: = something already settled that I should just apply. (“DECISION: payment terms are net 30, confirmed with their finance lead.”)

Every other annotation type — the arrows, the drawing tools, the lone carets with nothing attached — gets ignored, and reviewers know it gets ignored. The constraint feels strict for about ten minutes. Then people realize their comments actually get read, in order, and addressed.

Most of my clients annotate in whatever’s open on their machine. The ones without Acrobat use the annotate tool in PDFLean in a browser tab, which covers exactly these three types and nothing else — honestly the limited palette is part of why it works. A reviewer with fourteen annotation tools will use all fourteen.

Rule 3: The script turns comments into a checklist

Before I touch the document, I extract every annotation into a plain list with page numbers:

import fitz  # PyMuPDF

def dump_annots(path):
    doc = fitz.open(path)
    for page in doc:
        for a in page.annots():
            info = a.info
            yield {
                "page": page.number + 1,
                "type": a.type[1],           # Highlight, StrikeOut, FreeText, ...
                "author": info.get("title", ""),
                "note": (info.get("content") or "").strip(),
            }

for row in dump_annots("contract_CLIENT_review.pdf"):
    print(f'p{row["page"]:>2} [{row["type"]:<9}] {row["author"]}: {row["note"]}')

That printout becomes the round’s checklist. Every line gets one of three marks as I work: applied, answered, or pushed back with a reason. At the end of the round, the marked-up checklist goes back to the client with the new draft. Half the value is the checklist itself — reviewers can see their DECISION notes were applied and their questions got answers instead of vanishing into a fresh PDF. The other half is selfish: two rounds later, when someone asks “why does clause 6 say net 30?”, the answer is in a text file, not buried in a dead PDF’s comment layer.

One gotcha: some tools store the visible note text in the annotation’s popup rather than its content field, so occasionally the script prints an empty note for an annotation that visibly has text. When the checklist shows a blank, I open that page and read the popup directly. It happens rarely enough that I haven’t bothered automating around it.

Rule 4: What ships is flat and clean

When a round closes, two files leave my machine:

The new working draft (still commentable, still the same filename pattern).
When the document is final: a flattened copy. Flattening bakes the annotations into the page content — or, for the truly final version, I produce a clean copy with no annotations at all and archive the last marked copy with its checklist.

The distinction matters. A flattened file is for records and signatures: nobody’s comment layer should be editable in the archived version of a signed agreement. The marked copy plus checklist is the audit trail; the flat file is the artifact. Mixing those two jobs into one file is how you end up, months later, unable to prove which comments were ever addressed.

What this doesn’t fix

It doesn’t make reviewers respond faster, and it doesn’t rescue a document that was badly structured before review started — comments cluster on the confusing parts either way, which is its own kind of useful signal. What it does is make each round terminal: every comment enters as a question, an edit, or a decision, gets extracted into a list, and leaves a written trace. Review stops being a recurring meeting with a PDF and becomes a queue that empties.

If your current process is “send PDF, receive vibes,” steal the parts in order: one marked copy first, the three-type rule second, the extraction script third. The first one alone is worth more than the other two combined.

Batch-Converting 60 Receipt Photos into One PDF (and Back Again)

Marcus Hale — Wed, 14 Oct 2026 02:47:22 +0000

A client of mine documents site work with a phone. Every Friday I get a folder -- sometimes a literal zip over email -- with 40 to 80 photos: receipts, signed delivery notes, whiteboard shots, the occasional crooked picture of a screen. My job is to turn that into one PDF the bookkeeper can file, in capture order, without spending my afternoon in a file dialog.

This is the batch image workflow I landed on. It has two directions, because the reverse job shows up just as often: a PDF arrives and somebody needs the images out of it, one file per image, right now.

Direction 1: Many images, one PDF

The scripting route takes about fifteen lines with Pillow:

from PIL import Image
from pathlib import Path

photos = sorted(Path("site-photos").glob("*.jpg"))
pages = []
for p in photos:
    img = Image.open(p).convert("RGB")
    img.thumbnail((1654, 2339))          # ~A4 at 200 DPI, plenty for receipts
    pages.append(img)

pages[0].save(
    "site-week-41.pdf",
    save_all=True,
    append_images=pages[1:],
    resolution=200.0,
)

Three details that matter more than the code:

Sort order is the document. Phone filenames (IMG_2041.jpg) sort fine until someone AirDrops from a second phone and the names interleave wrong. If order matters -- and for receipts it does -- rename by timestamp first (exiftool "-FileName<DateTimeOriginal" does it in one pass) or sort by modification time and eyeball the first and last page before sending.
Downscale before you bind. A 48MP phone photo at full size makes a 200MB PDF nobody can email. 200 DPI at A4-ish dimensions is more than legible for a receipt and cuts the file by an order of magnitude.
.convert("RGB") is not optional. One PNG with an alpha channel or one HEIC-converted CMYK JPEG will kill the whole save at page 37. Ask me how I know.

Direction 2: One PDF, many images

Extracting with PyMuPDF gets you the embedded images at their stored resolution, which is what you want for re-use:

import fitz  # PyMuPDF

doc = fitz.open("catalog.pdf")
for page_no, page in enumerate(doc):
    for i, img in enumerate(page.get_images(full=True)):
        xref = img[0]
        pix = fitz.Pixmap(doc, xref)
        if pix.n > 4:                    # CMYK etc.
            pix = fitz.Pixmap(fitz.csRGB, pix)
        pix.save(f"p{page_no + 1:02d}_{i + 1}.png")

Know what you're getting: this pulls the raw image objects. If the "image" on a page is actually a rendered chart or a scanned page assembled from tiles, you'll get tiles and charts, not the screenshot you were imagining. For a full-page render instead, page.get_pixmap(dpi=200) is the honest tool -- different job, different output.

Where I stop scripting

I keep the scripts for the Friday folder, because it arrives in the same shape every week. For the one-off jobs -- a client sends three PNGs and wants one PDF back now, or I need a quick batch conversion on a machine where I haven't set anything up -- I use the image batch tool on PDFLean in the browser instead of touching a terminal. Same operations, drag, drop, done. Files I process there without an account are deleted immediately after processing, which is the property I check first whenever a client's documents are involved; a free account raises the per-file limit to 50MB for the weeks the photos arrive huge.

The rule of thumb after a year of this: if the job repeats weekly in the same shape, script it once and never think again. If it happens twice a quarter, a batch tool in the browser is faster than finding the script, remembering its arguments, and updating it because Pillow changed something. Both directions live in my runbook now -- the expensive version of this task was always doing it one file at a time.