I built an API that turns messy PDFs into clean, readable text

#api #python #pdf #opensource

PDFs are made for printing, not reading. Copy text out of one and you get page numbers in the middle of sentences, the same header on every page, words split in half by hyphens, and every line broken where the page ended.

So I built Clean PDF Reader: send it a PDF link or upload a file, and it hands back the text the way a person would want to read it. It's the second of 3 reading APIs I'm building, after the Public Domain Book Finder.

What it does

Endpoint	What you get
`GET/POST /pdf/read`	Clean text, served page by page. Headers, footers and page numbers removed, broken lines and hyphenated words joined back up.
`GET/POST /pdf/chapters`	The chapter list, from the PDF's bookmarks or from detected headings, plus the text of any one chapter.
`GET/POST /pdf/info`	Title, author, dates, page count, page size, and tables as rows.

Every endpoint takes either ?url= (GET) or a file upload (POST), up to 25 MB and 400 pages.

Clean text from a link

import os
import requests

HEADERS = {
    "X-RapidAPI-Key": os.environ["RAPIDAPI_KEY"],
    "X-RapidAPI-Host": "clean-pdf-reader.p.rapidapi.com",
}
PDF = "https://arxiv.org/pdf/1706.03762"  # "Attention Is All You Need"

page = requests.get(
    "https://clean-pdf-reader.p.rapidapi.com/pdf/read",
    params={"url": PDF, "words_per_page": 900},
    headers=HEADERS,
).json()

print(page["title"], "-", page["word_count"], "words,", page["total_pages"], "pages")
print(page["cleaned"])

Attention Is All You Need - 5889 words, 7 pages
{'headers_footers': 0, 'page_numbers': 14, 'hyphens_fixed': 14, 'lines_joined': 245}

The cleaned counts show what was fixed. Ask for format=markdown and headings come back as ## Heading.

Uploading a file

with open("report.pdf", "rb") as f:
    page = requests.post(
        "https://clean-pdf-reader.p.rapidapi.com/pdf/read",
        files={"file": f},
        headers=HEADERS,
    ).json()

Chapters, one at a time

chapters = requests.get(
    "https://clean-pdf-reader.p.rapidapi.com/pdf/chapters",
    params={"url": PDF},
    headers=HEADERS,
).json()

for ch in chapters["chapters"]:
    print(ch["number"], ch["title"], f'(pages {ch["start_page"]}-{ch["end_page"]})')

# Then fetch just the one you want:
one = requests.get(
    "https://clean-pdf-reader.p.rapidapi.com/pdf/chapters",
    params={"url": PDF, "chapter": 3},
    headers=HEADERS,
).json()
print(one["chapter"]["text"][:300])

source tells you where the chapters came from: outline (the PDF's own bookmarks), headings (detected from big or "Chapter 1"-style lines), or none.

How it works

Headers and footers are found by repetition. The first and last lines of every page are compared with the numbers swapped out, so "Page 3 of 40" and "Page 4 of 40" count as the same line. Anything that repeats on 40% or more of the pages is removed. Big text is left alone, so a chapter title is never mistaken for a running header.
Paragraphs are rebuilt from the layout. Line gaps, indents, short last lines and font size changes mark where a paragraph really ends. Everything else is joined back together, including paragraphs that carry on to the next page.
Word spacing scales with the font. My first version glued words together in LaTeX papers ("Thedominantsequence...") because their word gaps are tighter than a fixed limit allows. Measuring gaps relative to font size fixed it: the same paper went from 2,005 words to 5,889.
Rotated text is skipped. Labels running sideways along figures were showing up as backwards junk, so only upright text is read.
Safe link fetching. Links that point at private or internal addresses are refused, downloads stop at 25 MB, and every redirect is checked too.

It's written in Python with FastAPI and pdfplumber, and runs on Render alongside the Book Finder.

Try it

Clean PDF Reader on RapidAPI (free Basic plan)

Scanned PDFs (images with no text layer) and password-protected files aren't supported yet. You'll get a clear error instead of empty text.

Next up in this series: a Doc Summarizer / Chapter Splitter. If you build something with this one, I'd love to see it!

DEV Community