DEV Community: Catalin Friptu

Turn PDFs (even scanned ones) into RAG-ready chunks with one API call

Catalin Friptu — Sat, 24 Oct 2026 05:38:01 +0000

Every RAG project hits the same wall on day two: the documents. Half are born-digital PDFs, a quarter are scans someone ran through a photocopier in 2011, and the rest are Word files. Your embedding model wants clean text in sensible chunks, with page numbers so the answer can cite its source.

This post shows the two ways I do it: a small Python script you own, and a hosted API for when you don't want to run OCR yourself.

What "RAG-ready" actually means

Before code, the target. For each document you want:

Text in reading order, not columns glued together mid-sentence.
Structure kept as Markdown: headings, lists and tables survive, because they carry meaning for the LLM and help chunking.
OCR only where needed. Running OCR on a born-digital page is slower and worse than reading its text layer. Running nothing on a scanned page gives you an empty string, and your retriever silently never finds that page.
Chunks with overlap (1,000–2,000 characters with ~200 overlap works for most embedding models) and page numbers on each chunk.

Point 3 is where most pipelines break. A "PDF" is often mixed: 40 typed pages and 2 scanned signature pages.

Option 1: do it yourself in Python

For born-digital PDFs, pdfplumber gets you most of the way:

import pdfplumber

def pdf_pages(path):
    with pdfplumber.open(path) as pdf:
        for i, page in enumerate(pdf.pages, start=1):
            text = page.extract_text() or ""
            yield i, text

def chunk(text, size=1500, overlap=200):
    start = 0
    while start < len(text):
        end = min(start + size, len(text))
        yield text[start:end]
        if end == len(text):
            break
        start = end - overlap

for page_no, text in pdf_pages("paper.pdf"):
    if not text.strip():
        print(f"page {page_no}: no text layer -> needs OCR")
        continue
    for c in chunk(text):
        ...  # embed c, store with page_no

The if not text.strip() line is the important one: it's how you detect a scanned page. From there you render that page to an image (pypdfium2) and run Tesseract on it (pytesseract), with the right language packs installed. Then you'll want headings from font sizes, tables from ruled lines, DOCX support, language detection... It's a weekend, plus the fun of installing Tesseract language data in every environment you deploy to.

Option 2: one API call

Disclosure: I built this one. It's a pay-per-page Actor on the Apify Store, PDF Text Extractor, that does exactly the list above: text layer where there is one, OCR only on pages without one (13 languages), Markdown with headings/lists/ruled tables, per-page rows, chunks with overlap, and language detection. It also reads DOCX and image files.

curl -X POST "https://api.apify.com/v2/acts/malmurk~pdf-text-extractor/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
        "startUrls": [{"url": "https://arxiv.org/pdf/1706.03762"}],
        "outputMode": "page",
        "chunkSize": 1500,
        "chunkOverlap": 200
      }'

You get one JSON row per page (page 2 of that real run, text shortened):

{
  "file_name": "1706.03762v7.pdf",
  "page": 2,
  "page_count": 15,
  "markdown": "### 1 Introduction\n\nRecurrent neural networks, long short-term memory [13] and ...",
  "used_ocr": false,
  "language": "en",
  "chunks": [
    {"index": 0, "start": 0, "end": 1431, "text": "### 1 Introduction\n\nRecurrent neural networks ..."},
    {"index": 1, "start": 1234, "end": 2526, "text": "integral part of compelling sequence modeling ..."}
  ]
}

Feeding a vector store from Python:

from apify_client import ApifyClient

client = ApifyClient(APIFY_TOKEN)
run = client.actor("malmurk/pdf-text-extractor").call(run_input={
    "startUrls": [{"url": u} for u in pdf_urls],
    "outputMode": "page",
    "chunkSize": 1500,
})
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    for c in row["chunks"] or []:
        store.add(text=c["text"], metadata={"file": row["file_name"], "page": row["page"]})

Pricing is $1 per 1,000 pages, plus $7 per 1,000 for pages that actually needed OCR. Files that fail (dead link, password-protected PDF, an HTML page instead of a PDF) cost nothing and come back as a row with an error field, which is handy for spotting the "download" links that are really web pages.

Gotchas either way

"PDF links" that are HTML pages. Many sites link to a viewer page with a Download button. Check the Content-Type before you parse.
Tables without ruling lines (most scientific papers) come out as text in reading order with any text-layer tool. If tables matter, check them by eye on a sample.
OCR language. Tesseract with only eng on a German scan produces confident garbage. Set the languages you actually have.
Rotated margin stamps (like arXiv's vertical ID) are noise; skip rotated text on text-layer pages.

Use it from an agent

If you work in Claude, Cursor or another MCP client, the same Actor is available as a tool through Apify's MCP server: add https://mcp.apify.com?tools=malmurk/pdf-text-extractor and ask it to "read this PDF and summarise section 3".

I'm Cătălin, a Python developer in Chișinău building scraping and document tools. Questions or a PDF that breaks it: leave a comment. Drafted with AI assistance, tested and edited by me.

How to find local businesses with no website (a lead list for web designers)

Catalin Friptu — Sat, 24 Oct 2026 05:37:28 +0000

If you build websites for small businesses, the best prospects are the ones that don't have a website yet: a dentist with only a phone number on the map, a salon that lives on Instagram, a restaurant whose domain expired two years ago.

Here's how to build that list for any city from OpenStreetMap, for free, plus the three traps that make most "no website" lists useless.

Step 1: query OpenStreetMap with Overpass

OpenStreetMap stores businesses as tagged objects: amenity=dentist, shop=hairdresser, amenity=restaurant, shop=car_repair, and so on. Many have phone, email, opening_hours, and sometimes website. The Overpass API lets you ask for "dentists in this city with a name but no website tag":

[out:json][timeout:60];
area["name"="Graz"]["boundary"="administrative"]["admin_level"="6"]->.city;
nwr["amenity"="dentist"]["name"](area.city)[!"website"][!"contact:website"];
out center tags;

Paste it into overpass-turbo.eu, press Run, and export the result. For Graz (run on 2026-10-24) that returned 76 dentist entries with no website tag, 62 of them with a phone number (a few are the same practice mapped twice, as a point and as a building).

Swap amenity=dentist for any niche: shop=beauty, shop=hairdresser, amenity=restaurant, office=lawyer, craft=plumber. The admin_level of a city differs by country: if you get zero results, look up the city's boundary relation on openstreetmap.org and check its admin_level tag.

Step 2: don't trust the empty field (the three traps)

This is where raw lists fall apart.

Trap 1: "no website tag" is not "no website". OSM volunteers often skip the field. A missing tag means unknown. Google the name before you pitch, or you'll be the person offering a website to someone who has a nice one.

Trap 2: chains. Franchise locations are often mapped without a website, because the brand site is elsewhere. Your "leads" fill up with Subway and Dave & Buster's. Filter out anything with a brand or brand:wikidata tag, and any name that appears at several locations in your area.

Trap 3: the website field lies the other way too. When a business does list a website, check it:

the domain doesn't resolve any more (expired: great lead),
it times out or returns 404 (broken: great lead),
it's actually a Facebook or Instagram URL, or the domain redirects to a Facebook page (social only: great lead).

A simple HEAD request, falling back to GET, with a short timeout catches all three. Be generous in the other direction: a 403 from a bot wall or a 500 means a server exists, so don't count that as "no website".

Step 3: make the phone numbers usable

Mappers type phone numbers however they like: 0316 822052, +43 316 822 052, (0316) 82-20-52. Convert them to international E.164 (+43316822052) with the phonenumbers library and the country of the city, so they work in a dialer, WhatsApp or a CRM.

The shortcut

Disclosure: I packaged all of the above as a pay-per-result Actor on the Apify Store, Businesses Without Websites Finder. You type a city and pick categories, and it:

runs the Overpass queries per category,
removes duplicates (shops are often mapped twice, as a point and as a building),
skips chains using OSM brand tags and the name-suggestion-index,
checks every listed website live and labels each business none, social_only or dead, with the reason (DNS lookup failed, timeout, website redirects to www.facebook.com),
converts phones to E.164.

A run for dentists in Chișinău returns 100 leads in about 90 seconds. It costs $3 per 1,000 leads, and businesses that get filtered out are free. It also works from Make, Zapier or n8n if you want new leads to land in a Google Sheet every week.

Whichever route you take: check each lead by hand before you call, and keep the local rules for cold outreach in mind (in the EU, GDPR and ePrivacy apply to calls and emails).

Data © OpenStreetMap contributors, ODbL. I'm Cătălin, a Python developer in Chișinău. If you try the query on your city and get zero results, post the city in the comments and I'll help with the area filter. Drafted with AI assistance, tested and edited by me.