How to compare two versions of a PDF contract word by word in Python

A PDF has no track changes. When a contract comes back as a PDF "with a few small edits", the only record of what changed is the difference between two files. There are three obvious ways to compute that difference: compare the rendered pages pixel by pixel, compare the extracted text line by line, or compare it word by word. They give very different answers on the same pair of files, and only one of them survives the other side's word processor re-flowing the text.

Below are measurements of all three on a real pair of PDFs, then a word-level comparison in about 120 lines of Python (PyMuPDF plus the standard library's difflib) that gets the boxes of every changed word and writes the changes back into the PDF as comments. The PDF Editor in the osFoundry desktop app works the same way: its Compare Files dialog says "Text is compared word by word; the differences are shown on the new file", and on this pair it reports the same five changes as the script below.

The test pair

The files are a made-up one-page design services agreement (Example), generated with PyMuPDF so the text layer is real. The version that comes back has five edits:

Clause	Sent	Came back
3. Payment	within 30 days. Late payments add a 2% fee each month.	within 60 days.
4. Revisions	two rounds of revisions	unlimited rounds of revisions
5. Ownership	the final logo once the fee is paid in full.	the final logo and all drafts once work begins.

To simulate the other side's software re-setting the text, a third file has the returned version's exact words typeset again at 12 pt in a 300 pt column instead of 11.5 pt in 468 pt. Same text, different line breaks.

Pixels, lines or words

The pixel diff renders both first pages at 72 dpi in greyscale and counts rows where any pixel differs by more than 40 levels; a "band" is a run of such rows. The line diff runs difflib.SequenceMatcher over page.get_text() split on newlines. The word diff is the script further down.

New file	Pixel diff	Line diff	Word diff
As sent back	30 of 792 rows, 3 bands	3 hunks, 3 lines	5 changes
Same words, re-typeset	284 of 792 rows, 20 bands	6 hunks, 9 lines	5 changes

On the file as sent back, the pixel diff finds three areas and the line diff three lines. Both are right but neither says what changed: the line diff reports clause 3 as one replaced line, and you still have to spot "30" against "60" yourself. On the re-typeset copy, the pixel diff lights up 20 separate bands across most of the text, and the line diff doubles to 9 lines touched because the wrapping moved. The word diff gives the same five changes on both files, because whitespace and line breaks never reach it.

Getting characters out with their boxes

A plain get_text() string is enough to find changes, but writing them back onto the page needs coordinates. get_text("rawdict") gives every character with its bounding box. The function below flattens that into one list in reading order:

import difflib, re, sys, unicodedata
import pymupdf

def char_stream(path):
    """All characters of a PDF in reading order, NFKC-normalised, each with its page and box."""
    chars = []
    for pno, page in enumerate(pymupdf.open(path)):
        for block in page.get_text('rawdict', sort=True)['blocks']:
            for line in block.get('lines', []):
                for span in line['spans']:
                    for ch in span['chars']:
                        for c in unicodedata.normalize('NFKC', ch['c']):
                            chars.append((c, pno, pymupdf.Rect(ch['bbox'])))
                chars.append(('\n', pno, None))  # a line break is only whitespace
    return chars

Two details matter. sort=True orders blocks top to bottom, left to right, which is what you want for a single-column contract (two-column layouts need their own ordering). NFKC folds look-alike characters together, so a full-width "６０" typed on one side and "60" on the other don't show up as a change. Each normalised character keeps the box of the character it came from, so the list stays aligned with the page.

Tokens: words, and one character at a time where there are no spaces

The diff runs on tokens, not characters. A token here is a run of Latin letters and digits, or any single other visible character:

TOKEN = re.compile(r"[0-9A-Za-z][0-9A-Za-z.,%$'’-]*(?<![.,])|\S")

def tokens(chars):
    text = ''.join(c for c, _, _ in chars)
    return text, [(m.start(), m.end(), m.group()) for m in TOKEN.finditer(text)]

For English this gives words, with a trailing full stop split off so "days." and "days" still match. Numbers stay whole, so a change shows as "30" → "60" rather than "3" → "6".

The "any single other character" part is for languages without spaces between words. Splitting a Japanese version of the same contract on whitespace, which is what PyMuPDF's get_text("words") does, gives one "word" per sentence, and the diff reports three whole sentences as replaced. With one token per character it finds the edits inside them, but as 8 fragments, because shared characters such as の split one edit into pieces ("2" deleted, "まで" → "数", "制限なく" inserted). Merging changes that are separated by at most one unchanged character brings that to 5, the same five edits as the English file. In English the merge never fires, since the shortest gap between two changes ("once") is four characters.

The diff

PUNCT = set('.。．!?')

def hunks(old_toks, new_toks, gap_chars=1):
    a = [t for _, _, t in old_toks]
    b = [t for _, _, t in new_toks]
    out = []
    for tag, i1, i2, j1, j2 in difflib.SequenceMatcher(None, a, b, autojunk=False).get_opcodes():
        if tag == 'equal':
            continue
        if tag in ('delete', 'insert'):
            seq, lo, hi = (a, i1, i2) if tag == 'delete' else (b, j1, j2)
            k = 0
            while hi + k < len(seq) and seq[lo + k] == seq[hi + k]:
                k += 1
                if seq[hi + k - 1] in PUNCT and seq[lo + k] not in PUNCT:
                    i1, i2, j1, j2 = (i1 + k, i2 + k, j1 + k, j2 + k)
                    break
        if out:
            pi1, pi2, pj1, pj2 = out[-1]
            if sum(len(x) for x in a[pi2:i1]) <= gap_chars and sum(len(x) for x in b[pj2:j1]) <= gap_chars:
                out[-1] = [pi1, i2, pj1, j2]
                continue
        out.append([i1, i2, j1, j2])
    return out

The sliding loop handles a quirk of every diff algorithm. The deleted sentence sits between "days" and "4. Revisions", and the text around it is ". Late payments add a 2% fee each month." on one side and "." on the other. Deleting ". Late … month" and deleting "Late … month." are equally minimal, and SequenceMatcher picked the first. Sliding a pure deletion or insertion along the repeated tokens until it ends on punctuation gives the version a person would mark.

autojunk=False is deliberate; more on that below. Run against the two files, the full script (contract_diff.py) prints:

5 changes
1. replaced  '30' -> '60'
2. deleted   'Late payments add a 2% fee each month.' -> ''
3. replaced  'two' -> 'unlimited'
4. inserted  '' -> 'and all drafts'
5. replaced  'the fee is paid in full' -> 'work begins'

and exactly the same five lines for the re-typeset copy.

Writing the changes back as comments

Every token carries the boxes of its characters, so marking a change on the new file is a highlight annotation over those boxes, merged per line, with the old and new text as its comment. A deletion has nothing left to highlight, so it gets a note icon just after the text that preceded it:

def mark(new_pdf, changes, out_pdf):
    doc = pymupdf.open(new_pdf)
    for ch in changes:
        by_page = {}
        for pno, r in ch['boxes']:
            rects = by_page.setdefault(pno, [])
            if rects and abs(rects[-1].y0 - r.y0) < 2:  # same line: one box, not one per character
                rects[-1] |= r
            else:
                rects.append(pymupdf.Rect(r))
        note = {'replaced': f'Replaced: "{ch["old"]}" with "{ch["new"]}"',
                'deleted': f'Deleted: "{ch["old"]}"',
                'inserted': f'Inserted: "{ch["new"]}"'}[ch['kind']]
        for pno, rects in by_page.items():
            page = doc[pno]
            if ch['kind'] == 'deleted':
                annot = page.add_text_annot(rects[-1].tr + (2, -4), note, icon='Comment')
            else:
                annot = page.add_highlight_annot(rects)
                annot.set_info(content=note)
            annot.set_info(title='compare')
            annot.update()
    doc.save(out_pdf)

These are standard PDF annotations, so ordinary PDF viewers show them, and the person who sent the edits sees exactly what you are asking about.

Long contracts: autojunk and clause by clause

SequenceMatcher has a heuristic called autojunk: when the second sequence has 200 elements or more, any element that makes up more than 1% of it is ignored as a matching anchor. That is meant for things like blank lines in source code. In contract text the frequent tokens are words like "the" or, in Japanese, の and 甲 (the contract's name for one party). On the one-page Japanese file (238 tokens in the new version) it set aside 18 different characters, 甲, 乙, の and 。 among them, and the result changed from 8 hunks to 6. Not worse in that case, but different, and that is not something to leave to a heuristic in a document you are about to sign.

Turning it off has a cost. The diff is roughly quadratic in the worst case, and a 40-page contract is far longer than one page. To measure it, the Japanese contract was repeated 10 and 100 times with only the last copy edited, on an M1 Mac with Python 3.12:

Tokens	autojunk=False	autojunk=True	Clause by clause
2,550	12 ms	3 ms	1 ms
25,500	1,142–1,564 ms	163–209 ms	7–10 ms

The 25,500-token figures are the range over three runs; other jobs were using the CPU at the same time, and two earlier runs under heavier load took up to 4.2 seconds without autojunk.

The last column is the fix. Contracts already come in numbered clauses, so split both texts at the clause headings, line the clauses up with one SequenceMatcher over the list of clauses (identical clauses match as whole items, cheaply), and run the token diff only inside pairs that differ. That keeps autojunk=False and is still the fastest of the three. In measure.py the split is a regex on 第N条 for the Japanese text; for English, a pattern on lines like "3. Payment" would do the same job.

Where this stops working

Everything above reads the text layer. A scanned contract has none, and needs OCR first, after which OCR errors show up as changes. Tables extracted in the wrong reading order produce noisy diffs. And a clause that was moved rather than edited comes out as one deletion and one insertion; detecting moves needs a second pass that matches deleted and inserted blocks against each other.

The scripts (contract_diff.py, measure.py and the generators for the test files) run with uvx --with pymupdf python. The PDF Editor with the same compare, mark-as-comments and report steps is part of the free desktop app on the osFoundry download page, and it runs on your own machine like the rest of osFoundry's local-first platform.