DEV Community

Matvey
Matvey

Posted on

One IPCC Page Gave Me Two Headings, or Eight. The Difference Was Which Pages I Asked For

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them reads PDFs: a link in, one row out, with the plain text, per-page text, Markdown, tables, metadata and — if you ask for it — retrieval chunks that remember which heading they sit under.

Markdown is the field most people actually want, because it is the field you hand to a model. It is also the field with a secret: a PDF has no headings. It has glyphs at coordinates, in a font, at a size. "This line is an H2" is not read out of the file, it is inferred — and the inference uses the rest of the document as its yardstick. Change which pages you ask for and the yardstick moves.

How the guess works

The rule inside my extractor is four lines long and deliberately dumb:

  1. Group words into visual lines, keep each line's largest font size.
  2. Take the median line size across the pages being read — that is "body text".
  3. A line is a heading if it is at least 1.12× body size, is 120 characters or shorter, has no token longer than 30 characters, does not end in ., , or ;, and does not trail off on a function word (of, the, and …).
  4. Level by ratio: ≥1.6× is #, ≥1.3× is ##, anything else ###.

Step 2 is the problem. The median is computed over the pages you selected, not over the document. The Actor has a pageRange field — 1-5, 8, 12- — and buyers use it, because pages are billed. So the same physical page can be parsed in two different contexts, and the context is what decides what a heading is.

The measurement

On 11 November 2026 I took six public PDFs with six different producers, and read each page twice through the Actor's own parser: once as part of the whole document, once alone, as if I had passed pageRange: "13". Same file, same code, same page. Only the window changed.

file          pages  producer                 tested  headings   headings   pages
                                              pages   whole doc  read alone changed
attention.pdf    15  pdfTeX-1.40.25              15        16         16       0
gpt3.pdf         75  pdfTeX-1.40.17              20         7          7       0
gpt4.pdf        100  pdfTeX-1.40.25              20        11         51       6
ipcc.pdf         42  Adobe PDF Library 17.0      20        23         35       4
gdpr.pdf         88  PDFlib+PDI 8.0.5p2          20         2          2       0
f1040.pdf         2  Designer 6.5                 2        22         22       1
                                              ------                        ----
                                                  97                          11
Enter fullscreen mode Exit fullscreen mode

Eleven of ninety-seven, and all eleven failed in one of three distinct ways.

1. The title page loses its title. Page 1 of the IPCC AR6 Synthesis Report is a cover: everything on it is set large. Inside the whole document the median body size is 9.5pt, so four lines clear the bar — # CLIMATE CHANGE 2023, # Synthesis Report, # Summary for Policymakers, ## A Report of the Intergovernmental Panel on Climate Change. Read that page alone and the median of the page is the cover type: 32.0pt. Nothing on the page is 1.12× its own neighbours any more. Three headings vanish and the one survivor is demoted to ###.

2. The caption page invents headings. The opposite case, and the more expensive one. Page 17 of the GPT-4 technical report is the tail of the contributor and acknowledgements section, set small. In the full document it produces zero headings — correct, it is a wall of names. Alone, its own median drops, and seven lines clear the bar, including the running page number. Across the six changed pages of that one file, reading alone added 40 headings that the whole-document read did not produce, and removed none. They are lines like:

### Deployment
## We thank Microsoft for their partnership, especially Microsoft Azure for supporting model
## 17
Enter fullscreen mode Exit fullscreen mode

The same thing happens on IPCC figure pages: page 13 goes from two headings to eight, and the new ones include ### 2020 and ## 7 — an axis label and a page number.

3. The levels slide. The quiet one. On the IRS Form 1040 both reads find the same eight headings on page 1 with the same text, but alone six of the eight move up a level: ### becomes ##, ## becomes #, and the two already at # have nowhere to go. Nothing looks wrong in the output. Your document tree is just one storey taller than the last time you ingested it.

Why I care more than the output looks like it deserves

Because the chunker reads those headings. Chunks carry a heading path so a retrieval hit can be traced to its section, and the path is built from exactly these # lines. Page 13 of the IPCC report, chunked at 1,000 characters both ways:

whole document:        5 chunks, path ["climate change will continue to intensify"]
that page alone:       7 chunks, paths include
                         ["climate change will continue to intensify", "2020"]
                         ["climate change will continue to intensify", "7"]
Enter fullscreen mode Exit fullscreen mode

Same text, different chunk boundaries, and two chunks now claim to live in a section called "2020" and one in a section called "7". Embed that and your index has sections that do not exist. The GPT-4 contributions page does the same: 5 chunks whole, 8 chunks alone, three of them filed under a heading called Deployment that is a column label in a list of names.

Nothing throws, every row comes back "status": "ok". This is the failure mode I keep meeting in document pipelines: output that is plausible, well-formed, and wrong in the metadata rather than in the text.

"But tagged PDFs have real headings"

They do, and I checked, because it is the obvious objection. Of my six files, three carry a structure tree: the IPCC report, the GDPR text and Form 1040. Walking those trees:

f1040.pdf   1236 nodes   /H1 ×3, /H2 ×11, /H3 ×1
ipcc.pdf     972 nodes   no heading roles at all (/Span, /SPM_statements, /Caption, /footnote)
gdpr.pdf      40 nodes   /Document and /Span, nothing else
Enter fullscreen mode Exit fullscreen mode

One file out of six tells you where its headings are. The IPCC report is tagged by InDesign with a custom role set and never says "heading" once; the GDPR consolidated text is tagged to the depth of "this is a document containing spans". A reader that trusted tags would do better on Form 1040 and worse on everything else. Font size is the ugly heuristic that works on all six — it just needs the whole document in front of it.

What I actually do about it

For now: read the whole document, and use pageRange to save money, not to get a cleaner outline. The fix in the parser is to sample line sizes from every page of the file and emit only the pages that were asked for — one extra pass over the layout, no change at all for whole-document reads, and a one-page request that returns the same Markdown as the full read. That is the next change to this Actor. I am publishing the measurement first because the measurement is the part that generalises: if you promote headings by font size on top of pdfplumber or PyMuPDF, you have this bug too, and your tests all pass.

Running it

Input, with chunks on so you can see the heading paths:

{
  "urls": ["https://www.ipcc.ch/report/ar6/syr/downloads/report/IPCC_AR6_SYR_SPM.pdf"],
  "includePages": true,
  "extractTables": true,
  "chunkForRag": true
}
Enter fullscreen mode Exit fullscreen mode

The row that comes back for that file, trimmed to the fields this article is about:

{
  "format": "pdf",
  "pageCount": 42,
  "title": null,
  "author": null,
  "creator": "Adobe InDesign 18.3 (Macintosh)",
  "producer": "Adobe PDF Library 17.0",
  "documentCreatedAt": "2023-07-19T22:21:03+09:00",
  "wordCount": 21419,
  "charCount": 147113,
  "tableCount": 26,
  "ocrPageCount": 0,
  "warnings": ["5 page(s) have no text layer and look scanned. Switch on \"OCR pages without a text layer\" to read them."],
  "status": "ok"
}
Enter fullscreen mode Exit fullscreen mode

Note the two nulls: a 42-page flagship report ships with no /Title and no /Author. If your pipeline names documents from PDF metadata, it will name a lot of them None.

From Python, this is the check worth stealing — read a page both ways before you trust an outline:

import requests

API = "https://api.apify.com/v2/acts/lergassy~pdf-text-extractor/run-sync-get-dataset-items"

def pages(**extra):
    body = {"urls": [URL], "includePages": True, **extra}
    return requests.post(API, params={"token": "apify_api_..."}, json=body, timeout=900).json()[0]["pages"]

heads = lambda md: [l for l in md.split("\n") if l.startswith("#")]
whole = {p["number"]: p for p in pages()}
print(heads(whole[13]["markdown"]))                  # inside the document
print(heads(pages(pageRange="13")[0]["markdown"]))   # on its own
Enter fullscreen mode Exit fullscreen mode

The Actor is PDF Text Extractor: $0.003 per PDF plus $0.0001 per page, no fee per run, no login, no key. Scanned pages with no text layer are OCR-ed at $0.008 per page, and only when OCR actually returns text — the IPCC file above has five such pages, which is why that warning is in the row.

When it is the wrong tool. If your PDFs are properly tagged for accessibility and you need the publisher's own structure, read the structure tree instead of my Markdown — on a file like Form 1040 the tags are better than any font-size guess. If the document is handwritten, or a stamp-covered scan, OCR will read it badly and no amount of layout logic saves that. And if what you want is which pages mention a term rather than a whole-document outline, use includePages and search the per-page text: it is immune to everything in this article, because plain text does not depend on the median.

Top comments (1)

Collapse
 
kolmopdf profile image
KolmoPDF •

Sampling line sizes across the full document before filtering down to pageRange fixes the window drift cleanly, especially on cover pages and dense contributor appendices.

One extra edge case shows up even with a full-document baseline: reports packed with multi-page tables or figure captions. If a 40-page technical PDF has 15 pages of 7pt table cells and axis labels, an unweighted per-line median gets pulled below the actual prose size, which pushes 10pt bold callouts over the 1.12x bar. Computing the baseline from the modal font size weighted by character count (or restricting the baseline sample to multi-word lines above ~40 characters) keeps short table fragments and chart ticks from skewing the body yardstick.

Filtering out purely numeric lines unless they match a section prefix like 1.2 also stops stray page numbers like ## 17 or axis years like ### 2020 from entering the chunk metadata.