pdfium and pypdf return different text from the same PDF — four mismatches to know before you feed PDFs to an LLM

I assumed that "extract the text from this PDF" gives you the same characters no matter which library you use. It does not. Feed the same PDF to pdfium and to pypdf, and you get characters that only one of them returns, and characters that one of them silently replaces with a different sentence.

Worse, neither output necessarily matches what a human sees on screen. The page says Hello world; the extracted text says something else. The page shows nothing; the extracted text has an extra line. All of it is within the PDF specification, and neither library is wrong.

Most pipelines that hand PDFs to an LLM take the output of one extraction library and treat it as "the text of the document". This article shows four cases where that assumption breaks, each with a minimal PDF you can build yourself and the actual output of both libraries. The second half is about how I turned the mismatch into a detection signal for hidden text.

TL;DR

• Text with 0 Tz (zero horizontal scaling) does not exist as far as pdfium's text extraction is concerned. pypdf reads it.
• /ActualText makes pdfium return the replacement string instead of what is drawn. pypdf returns the drawn glyphs.
• A rewritten /ToUnicode table makes "A" on screen come out as "B" in the text. Mappings to invisible Unicode tag characters pass through both libraries untouched.
• Text inside a Form XObject that is never invoked with Do is read by neither. pypdf's extract_xform_text gets it.
• So don't trust a single extractor in front of an LLM. Run two, normalize, and treat the difference as the signal.

My previous post (I built a tool that finds "invisible text" hidden in PDFs) was about text that is extracted but cannot be seen, found by rendering the page with and without its text. This one is about a step earlier: the extracted text itself being different. You do not need to have read the first one.

Code in this article was run with Python 3.12 / pypdfium2 5.12.1 (pdfium build 7947) / pypdf 6.15.0. The full reproduction script is at the end.

---

The setup

If you generate test PDFs with a library, you lose control over how the text is hidden. So I write raw PDF syntax, byte by byte. The builder is about 60 lines: stack objects, write the xref table by hand.

class PDF:
    def __init__(self):
        self.objs = [None]                     # object numbers start at 1

    def add(self, body):
        self.objs.append(body.encode("latin-1") if isinstance(body, str) else body)
        return len(self.objs) - 1

    def stream(self, dic, data):
        data = data.encode("latin-1") if isinstance(data, str) else data
        return self.add(b"<< " + dic.encode() + b" /Length " + str(len(data)).encode()
                        + b" >>\nstream\n" + data + b"\nendstream")

    def build(self, root):
        out = bytearray(b"%PDF-1.7\n")
        offsets = [0] * len(self.objs)
        for i in range(1, len(self.objs)):
            offsets[i] = len(out)
            out += f"{i} 0 obj\n".encode() + self.objs[i] + b"\nendobj\n"
        xref = len(out)
        out += f"xref\n0 {len(self.objs)}\n".encode() + b"0000000000 65535 f \n"
        for i in range(1, len(self.objs)):
            out += f"{offsets[i]:010d} 00000 n \n".encode()
        out += (f"trailer\n<< /Size {len(self.objs)} /Root {root} 0 R >>\n"
                f"startxref\n{xref}\n%%EOF\n").encode()
        return bytes(out)

Every page has one font (Helvetica, WinAnsiEncoding) and one visible line. Each experiment adds one more line.

VISIBLE = "BT /F1 14 Tf 0 0 0 rg 60 780 Td (This is the visible body text.) Tj ET\n"

Two readers, both using the most ordinary "give me all the text on this page" API:

import pypdfium2 as pdfium
import pypdf

def pdfium_text(path):
    return pdfium.PdfDocument(path)[0].get_textpage().get_text_range()

def pypdf_text(path):
    return pypdf.PdfReader(path).pages[0].extract_text()

---

Mismatch 1: 0 Tz — characters that do not exist for pdfium

Tz is horizontal scaling. At 0 Tz every glyph has zero width and nothing appears on screen. The height is normal, so a "tiny font" check does not fire either.

BT /F1 12 Tf 0 Tz 0 0 0 rg 60 700 Td (SECRET-1 zero width) Tj ET

pdfium : 'This is the visible body text.'
pypdf  : 'This is the visible body text.\nSECRET-1 zero width'

Not one character of that line makes it into pdfium's text page (FPDFText). It is not even counted by count_chars(). pdfium builds its text page from layout, and a character with zero width apparently does not exist. pypdf just walks the Tj operators in the content stream and returns what it finds.

For what it's worth, at 1 Tz (1%) pdfium does return the text. The character box is then 0.08 pt wide, and a 12 pt letter is drawn as a hairline. Unreadable to a person, present in both extractors.

---

Mismatch 2: /ActualText — pdfium returns the replacement

/ActualText is a marked-content property that says "for text extraction, use this string instead of the glyphs in this span". It has legitimate uses: turning the ligature fi into "fi", rejoining a hyphenated word. Word uses it when exporting PDFs.

/Span << /ActualText (SECRET-2 replaced text) >> BDC
  BT /F1 12 Tf 0 0 0 rg 60 700 Td (Hello world) Tj ET
EMC

The page draws Hello world.

pdfium : 'This is the visible body text.\r\nSECRET-2 replaced text'
pypdf  : 'This is the visible body text.\nHello world'

pdfium follows the spec and returns the replacement. Hello world is nowhere in its output. pypdf ignores /ActualText and returns the glyphs as drawn.

This is not a question of which one is right. The spec intends the replacement, and pdfium honours it. The point is that the string a person verified on screen and the string a pdfium-based consumer receives (copy from Chrome's viewer, a pypdfium2 pipeline) can be two different things. And pdfium assigns the replacement characters the coordinates of the original glyphs. My "render with and without text" check from the previous post removes text at those coordinates, sees Hello world disappear, and concludes the text is visible. The replacement string sailed straight through as "visible text".

---

Mismatch 3: /ToUnicode — both libraries are fooled the same way

A font's /ToUnicode entry is a table from character codes to Unicode. Extractors trust it. Rewrite the table and the glyphs stay the same while the extracted characters change.

I put three tricks into one table:

3 beginbfchar
<41> <DB40DC41>                    % 'A' -> U+E0041 (TAG LATIN CAPITAL LETTER A)
<42> <00490067006E006F00720065>    % 'B' -> "Ignore" (one glyph, six characters)
<43> <200B>                        % 'C' -> U+200B (ZERO WIDTH SPACE)
endbfchar

BT /F1 12 Tf 0 0 0 rg 60 700 Td (ABC abc) Tj ET

The page draws ABC abc.

pdfium : 'This is the visible body text.\r\n\U000e0041Ignore\u200b abc'
pypdf  : 'This is the visible body text.\n\U000e0041Ignore\u200b abc'

This time both agree. A became a tag character, B became "Ignore", C became a zero-width space. If you print() the result it looks like Ignore abc, because tag characters and zero-width spaces are invisible in terminals and editors alike.

Unicode tag characters (U+E0000 to U+E007F) are an invisible copy of ASCII. Humans and editors do not see them, but they survive extraction and can reach a model's input unchanged. Some models have been reported to read them as the ASCII they mirror, which is why they are used to smuggle instructions (the technique is usually called ASCII smuggling). This article only shows that they survive extraction; it does not test how any particular model interprets them. Being able to plant them through a font table means a PDF whose page shows nothing but ordinary English can hand the extractor a different English sentence.

The one-glyph-to-six-characters expansion deserves a look too. Ligature expansion (fi to fi, ffi to ffi) is legitimate, so libraries accept multi-character mappings. The PDF spec sets no length limit; pypdf caps a single mapping at 512 bytes (256 UTF-16 characters) as a safety valve, which is still enough for a sentence or two per glyph.

One detail: pdfium's FPDFText_GetUnicode returns UTF-16 code units, so U+E0041 comes back as two values, 0xDB40, 0xDC41. If you classify characters one at a time, you have to combine surrogate pairs yourself. pypdf returns a Python str, already combined.

pdfium chars (code points >0x7e): ['0xdb40', '0xdc41', '0x200b']
pypdf  chars (code points >0x7e): ['0xe0041', '0x200b']

---

Mismatch 4: an unused Form XObject — read by neither

A Form XObject is a bundle of drawing operators, drawn when the page content invokes it with /X0 Do. If nothing invokes it, nothing is drawn. You can register it in the page's resource dictionary and simply never call it.

%% inside the XObject
BT /F1 12 Tf 0 0 0 rg 60 500 Td (SECRET-4 inside unused xobject) Tj ET

%% registered in the page resources, but the content never says /X0 Do
/Resources << /Font << /F1 5 0 R >> /XObject << /X0 6 0 R >> >>

pdfium : 'This is the visible body text.'
pypdf  : 'This is the visible body text.'

Neither reads it. Both follow "the text that appears when the page is drawn", and an object that is never drawn is never visited. That is correct behaviour, and since nothing appears on screen it is not even hidden text in the naive sense.

But it is still in the file, and it can be pulled out with a different tool. pypdf has an API that reads a stream directly:

r = pypdf.PdfReader(path)
page = r.pages[0]
xo = page["/Resources"]["/XObject"]["/X0"].get_object()
page.extract_xform_text(xo)   # -> 'SECRET-4 inside unused xobject'

"Invisible to the extractor but still in the file" is the classic shape of leftover data. Editing tools can leave old objects in the resource dictionary and remove only the reference.

---

Summary table

Trick	Human eye	pdfium	pypdf	Note
0 Tz	nothing	no characters	reads it	zero-width chars never enter pdfium's text page
/ActualText	the glyphs	replacement string	the glyphs	the spec intends the replacement; coordinates are those of the original glyphs
rewritten /ToUnicode	the glyphs	per the table	per the table	tag characters, zero-width, one-to-many mappings all pass through
unused XObject	nothing	not read	not read	extract_xform_text reads it individually

The four rows are not the same kind of gap. 0 Tz and /ActualText are disagreements between the two extractors. /ToUnicode is a case where both extractors agree with each other but not with the glyphs on screen. The unused XObject is a case where the human and both extractors agree (nothing is shown, nothing is extracted), yet text is still sitting in the file. So the cases expose three different gaps: disagreement between extractors, disagreement between extracted text and visible glyphs, and residual text outside the rendered page.

I did not test PyMuPDF, pdfminer.six or pdf.js. This article covers exactly two readers. Swapping the two reader functions in the script at the end is all it takes to check another library; if you do, please leave a comment.

The one-sentence explanation: each extractor decides for itself which information in the PDF counts as text. pdfium extracts from the layout it builds for rendering, so it drops zero-width characters and honours replacement strings. pypdf does not render at all; it reads the operator stream, so it keeps zero-width characters and ignores replacement strings. Both are valid choices under the spec, and nothing anywhere guarantees "this is the text you should give the model".

---

Turning the mismatch into a signal

The rest is about what I did with this. I build a tool that finds invisible text in PDFs (the previous post), and the four cases above were found as things that slipped past it.

The fix was not to replace pdfium with pypdf. pypdf ignores /ActualText and cannot do the render comparison. The right use is to run both and look at the difference.

Computing the difference

The basic rule is "report words that are in pypdf's output and not in pdfium's". Compared naively, that produces nothing but false positives. Here is what I had to do after running it over 29 real Japanese business PDFs (quotes, delivery notes, vehicle registrations).

Normalize. Strip whitespace, format characters, punctuation and symbols, and apply compatibility normalization (NFKC) so full-width/half-width forms and ligatures line up. pypdf and pdfium break lines in different places and treat punctuation differently; none of that should count.

import unicodedata

def norm(s):
    s = unicodedata.normalize("NFKC", s or "")
    return "".join(c for c in s
                   if not unicodedata.category(c).startswith(("Z", "C", "P", "S")))

Note that norm also removes zero-width and tag characters (category C). So the invisible-character check has to run first, on the raw extracted text. The order is: keep the raw output, check it for invisible characters, normalize for comparison, then diff. Do it the other way round and you delete the evidence you were looking for.

Split words into CJK runs and non-CJK runs, and match them loosely. pypdf tends to concatenate adjacent table cells. Two Japanese headers, "vehicle type" and "registration number", come back as one run with the first character missing; "13th" and "issue date" become one word. Judging those as "missing from pdfium" is wrong every time. For Japanese I accept a run if at least half of its 2-character fragments appear in the pdfium text; for everything else, half of its 4-character fragments.

def segment_present(seg, target):
    cjk = is_cjk(seg[0])           # kana / kanji / half-width katakana range check
    k = 2 if cjk else 4
    if len(seg) < (3 if cjk else 4) or seg in target:
        return True
    if not cjk and seg.isdigit():
        return True            # digit runs get glued to the neighbouring cell's number
    if len(seg) < k + 1:
        return False
    grams = [seg[i:i + k] for i in range(len(seg) - k + 1)]
    return sum(g in target for g in grams) >= len(grams) * 0.5

This leniency has a price: a short Japanese hidden word of 3 or 4 characters (say "confidential") can be missed if its two-character pieces occur in the visible text. I documented that as a known limit. Length is no guarantee either. A run counts as present when half of its fragments appear on the visible side, so a hidden sentence assembled from phrases that already occur in the visible text can pass, however long it is. Length only helps when the hidden text uses words the page does not.

Skip /ActualText whose text also appears on the glyph side. Word attaches /ActualText to ligatures and soft hyphens. If the replacement string is contained anywhere in the glyph-side text (pypdf's output), I do not report it; only the ones that are not contained get reported. This is a rule to reduce noise, not a proof that the replacement is harmless. The comparison is against the glyph text of the whole page, not against the span the replacement covers, so a replacement string built from words that appear elsewhere on the page passes.

Inspect /ToUnicode tables directly. The difference cannot see this case (both libraries return the same thing), so I read the font dictionary. Three checks: a code that expands to 4 or more characters (Latin ligatures stop at 3, as in ffi and ffl; some Arabic ligatures expand to more, so this is a heuristic for choosing what a human should look at, not proof of tampering); any mapping to an invisible character (tags, zero-width, bidi controls); and, for simple fonts with a standard encoding such as WinAnsi, an alphanumeric mismatch between what the encoding says the glyph is and what the table says the character is. Embedded subset fonts with private glyph names (g123 and the like) have no ground truth for "what glyph is drawn", so the third check is skipped for them.

Decode tag characters and show the message. U+E0020 to U+E007E map back to ASCII with chr(cp - 0xE0000). "9 invisible characters" is a fact; "decoded, they say SECRET-44" is something the reader can act on.

Results on real files

With all of that in place I ran the check over 29 real PDFs (mostly Japanese business forms) plus 17 synthetic samples, and I report the two sets separately.

Real files. The difference rules were tuned on these same 29 files, so they are the tuning set, not a held-out evaluation set. No false positives on them is expected, and it does not show there would be none on other documents. On these 29 files, the new checks produced exactly one finding, and I inspected it by hand. How much hidden text those files really contain is unknown, so misses cannot be counted.

Synthetic samples. On the 17 synthetic samples, the new checks produced no false positives. Those 17 were not built as an expected-versus-detected test, so please do not read them as a detection score; the 67 published test PDFs from the previous post are where each technique is checked against an expected result.

The one finding. The one real finding was a delivery note produced by Excel. It had, in its structure tree (the /Alt of a tagged figure), the string C:\Documents and Settings\<username>\My Documents\My Pictures\untitled.gif. The alt text of an image contained a path from the author's PC. That is not a false positive; that is exactly the kind of thing the tool exists to find.

Bonus: for render comparison in pdfium, don't delete text, set render mode 3

The previous post's render comparison removed text objects from the page and re-rendered. pdfium's FPDFPage_RemoveObject only works on top-level page objects, so text inside a Form XObject could not be removed and its area was excluded from the comparison. Text inside an XObject, covered with a white rectangle, walked straight through that gap.

The new version switches the text rendering mode to 3 (invisible) instead of deleting:

for o in page.get_objects(max_depth=16):          # descends into XObjects
    if o.type == R.FPDF_PAGEOBJ_TEXT:
        mode = R.FPDFTextObj_GetTextRenderMode(o.raw)
        want = (R.FPDF_TEXTRENDERMODE_CLIP            # modes 4-7: keep only the clip
                if mode >= R.FPDF_TEXTRENDERMODE_FILL_CLIP
                else R.FPDF_TEXTRENDERMODE_INVISIBLE)
        R.FPDFTextObj_SetTextRenderMode(o.raw, want)
img_without_text = page.render(scale=2).to_pil()  # no gen_content() needed

pdfium renders from the in-memory object state, so the change takes effect without regenerating the content stream (no gen_content() call). It also covers text inside nested Form XObjects, within the configured traversal depth (max_depth=16), so hidden text inside XObjects is now part of the comparison.

---

What this does not catch

For the record:

• It never verifies that the characters on screen really are the extracted characters. That would take OCR, and I chose not to bundle a Japanese OCR engine in an offline desktop app. The mismatch between two readers plus the direct inspection of the mechanisms (/ActualText, /ToUnicode) is the substitute.
• /ToUnicode remapping in embedded fonts. With no ground truth for glyph-to-code, a table that says "A becomes B" cannot be judged. Only expansions and invisible characters are checked.
• Short Japanese hidden words of 3 to 4 characters, and hidden sentences assembled from phrases in the visible text, can be missed by the difference check (see above).
• Digit-only runs are not compared.
• Text drawn as an image never appears in any extractor and is out of scope.

---

What this means if you feed PDFs to an LLM

Whether or not you build tools, if you own a pipeline that hands PDFs to a model:

1. One extractor is not enough. The pdfium family (pypdfium2, Chrome) and the operator-stream family (pypdf, pdfminer) can return different bodies for the same file, and there is no way to pick the "true" one.
2. Don't discard the difference; flag it. Normalize both outputs and compare. If a meaningful amount of text exists in only one of them, that PDF needs a human. Surfacing the fact is safer than trying to adjudicate.
3. Run one pass of invisible-character classification. Counting unicodedata.category(c) == "Cf" and the U+E0000 to U+E007F range catches tag characters and zero-width injection. It is a few lines. Run it on the raw extracted text, before normalizing (normalization removes these characters).
4. Passing images (vision) is a real mitigation. The model then receives only what is drawn, and all four cases above disappear. In exchange you get recognition errors and lose text that is visible but not recognizable. Depending on the use case, a combination is the practical answer.

---

Reproduction script

Builds the four PDFs, reads them with both libraries and prints the outputs side by side. About 150 lines including the builder; the only dependencies are pypdfium2 and pypdf.

article7_experiments.py (full)

# -*- coding: utf-8 -*-
import os
import tempfile

import pypdf
import pypdfium2 as pdfium
import pypdfium2.raw as R

OUT = tempfile.mkdtemp(prefix="article7_")


def _b(x):
    return x.encode("latin-1") if isinstance(x, str) else x


class PDF:
    def __init__(self):
        self.objs = [None]

    def add(self, body):
        self.objs.append(_b(body))
        return len(self.objs) - 1

    def reserve(self):
        self.objs.append(b"")
        return len(self.objs) - 1

    def set(self, num, body):
        self.objs[num] = _b(body)

    def stream(self, dic, data):
        data = _b(data)
        return self.add(b"<< " + _b(dic).strip() + b" /Length " +
                        str(len(data)).encode() + b" >>\nstream\n" + data +
                        b"\nendstream")

    def build(self, root):
        out = bytearray(b"%PDF-1.7\n%\xe2\xe3\xcf\xd3\n")
        offsets = [0] * len(self.objs)
        for i in range(1, len(self.objs)):
            offsets[i] = len(out)
            out += f"{i} 0 obj\n".encode() + self.objs[i] + b"\nendobj\n"
        xref = len(out)
        n = len(self.objs)
        out += f"xref\n0 {n}\n".encode() + b"0000000000 65535 f \n"
        for i in range(1, n):
            out += f"{offsets[i]:010d} 00000 n \n".encode()
        out += (f"trailer\n<< /Size {n} /Root {root} 0 R >>\n"
                f"startxref\n{xref}\n%%EOF\n").encode()
        return bytes(out)


def make_pdf(name, content, font_extra="", resources_extra="", hook=None):
    p = PDF()
    extra = hook(p) if hook else {}
    font = p.add("<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica "
                 "/Encoding /WinAnsiEncoding " + font_extra.format(**extra) + " >>")
    cs = p.stream("", content)
    pages = p.reserve()
    page = p.add(f"<< /Type /Page /Parent {pages} 0 R /MediaBox [0 0 595 842] "
                 f"/Resources << /Font << /F1 {font} 0 R >> "
                 f"{resources_extra.format(**extra)} >> /Contents {cs} 0 R >>")
    p.set(pages, f"<< /Type /Pages /Kids [{page} 0 R] /Count 1 >>")
    cat = p.add(f"<< /Type /Catalog /Pages {pages} 0 R >>")
    path = os.path.join(OUT, name)
    with open(path, "wb") as f:
        f.write(p.build(cat))
    return path


VISIBLE = "BT /F1 14 Tf 0 0 0 rg 60 780 Td (This is the visible body text.) Tj ET\n"


def pdfium_text(path):
    doc = pdfium.PdfDocument(path)
    return doc[0].get_textpage().get_text_range()


def pdfium_chars(path):
    doc = pdfium.PdfDocument(path)
    tp = doc[0].get_textpage()
    return "".join(chr(R.FPDFText_GetUnicode(tp.raw, i) or 0)
                   for i in range(tp.count_chars()))


def pypdf_text(path):
    return pypdf.PdfReader(path).pages[0].extract_text()


def show(title, path):
    print(f"\n=== {title} ===")
    print("pdfium :", repr(pdfium_text(path)))
    print("pypdf  :", repr(pypdf_text(path)))


# 1. zero-width text (0 Tz)
p1 = make_pdf("1_tz_zero.pdf", VISIBLE +
              "BT /F1 12 Tf 0 Tz 0 0 0 rg 60 700 Td (SECRET-1 zero width) Tj ET\n")
show("1. 0 Tz (zero-width text)", p1)

# 2. ActualText
p2 = make_pdf("2_actualtext.pdf", VISIBLE +
              "/Span << /ActualText (SECRET-2 replaced text) >> BDC "
              "BT /F1 12 Tf 0 0 0 rg 60 700 Td (Hello world) Tj ET EMC\n")
show("2. /ActualText", p2)

# 3. rewritten ToUnicode
cmap = ("/CIDInit /ProcSet findresource begin 12 dict begin begincmap "
        "/CMapName /Custom def 1 begincodespacerange <00> <FF> endcodespacerange\n"
        "3 beginbfchar\n"
        "<41> <DB40DC41>\n"
        "<42> <00490067006E006F00720065>\n"
        "<43> <200B>\n"
        "endbfchar\nendcmap CMapName currentdict /CMap defineresource pop end end")
p3 = make_pdf("3_tounicode.pdf", VISIBLE +
              "BT /F1 12 Tf 0 0 0 rg 60 700 Td (ABC abc) Tj ET\n",
              font_extra="/ToUnicode {tu} 0 R",
              hook=lambda p: {"tu": p.stream("", cmap)})
show("3. /ToUnicode remap", p3)
print("pdfium chars (code points >0x7e):",
      [hex(ord(c)) for c in pdfium_chars(p3) if ord(c) > 0x7e])
print("pypdf  chars (code points >0x7e):",
      [hex(ord(c)) for c in pypdf_text(p3) if ord(c) > 0x7e])


# 4. unused Form XObject
def _xobj(p):
    font = p.add("<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica "
                 "/Encoding /WinAnsiEncoding >>")
    xo = p.stream("/Type /XObject /Subtype /Form /BBox [0 0 595 842] "
                  f"/Resources << /Font << /F1 {font} 0 R >> >>",
                  "BT /F1 12 Tf 0 0 0 rg 60 500 Td (SECRET-4 inside unused xobject) Tj ET\n")
    return {"xo": xo}


p4 = make_pdf("4_unused_xobject.pdf", VISIBLE,
              resources_extra="/XObject << /X0 {xo} 0 R >>", hook=_xobj)
show("4. unused Form XObject (never drawn with Do)", p4)
r = pypdf.PdfReader(p4)
pg = r.pages[0]
xo = pg["/Resources"]["/XObject"]["/X0"].get_object()
print("pypdf extract_xform_text:", repr(pg.extract_xform_text(xo)))

print("\nPDFs written to:", OUT)

Output, verbatim:

=== 1. 0 Tz (zero-width text) ===
pdfium : 'This is the visible body text.'
pypdf  : 'This is the visible body text.\nSECRET-1 zero width'

=== 2. /ActualText ===
pdfium : 'This is the visible body text.\r\nSECRET-2 replaced text'
pypdf  : 'This is the visible body text.\nHello world'

=== 3. /ToUnicode remap ===
pdfium : 'This is the visible body text.\r\n\U000e0041Ignore\u200b abc'
pypdf  : 'This is the visible body text.\n\U000e0041Ignore\u200b abc'
pdfium chars (code points >0x7e): ['0xdb40', '0xdc41', '0x200b']
pypdf  chars (code points >0x7e): ['0xe0041', '0x200b']

=== 4. unused Form XObject (never drawn with Do) ===
pdfium : 'This is the visible body text.'
pypdf  : 'This is the visible body text.'
pypdf extract_xform_text: 'SECRET-4 inside unused xobject'

---

Takeaways

• Each extractor decides for itself which information in the PDF counts as text. 0 Tz, /ActualText, /ToUnicode and unused XObjects expose three different gaps: disagreement between extractors, disagreement between extracted text and visible glyphs, and residual text outside the rendered page.
• Every library is correct under the spec, and nothing guarantees "the text to give the model".
• So run two extractors, normalize, diff, and use the diff as the signal. On Japanese business forms that needs a loose match that absorbs glued table cells and glued numbers.
• A rewritten /ToUnicode is read the same way by both extractors in these tests, so the diff alone cannot catch it. Read the table itself. Decode tag characters and show the message.
• For render comparison, switching text to render mode 3 beats deleting it: it also reaches text inside nested XObjects, within the configured traversal depth.

These checks shipped in the next version of the tool from the previous post, PDF Privacy Checker, on the Microsoft Store (detection is free; it runs fully offline, and not a single byte of your file leaves your PC).

https://apps.microsoft.com/detail/9PLRJHFTPS53?hl=en-us&gl=US

If you have a trick you want to see run through this, leave a comment. Adding a case to the script above takes one function call.

---

About the author

Okinawa Software Lab. I lead in-house digital transformation at a small company in Okinawa, Japan. I build the tools we need ourselves, and I publish PDF apps on the Microsoft Store that follow the same principle: everything happens on your own PC.

• Website: https://okinawasoftwarelab.com/en/