I built an OLX scraper for 24 countries — the boring version that actually ships

OLX runs classifieds in about two dozen countries. Same brand, different domains, different anti-bot setups. Everyone scraping it does one country at a time.

I got tired of forking.

So I put 24 countries behind one input. country: "id" or country: "pl" or country: "br" — same schema out. It's live on Apify as primesieve/olx-global-scraper. One file. No browser. Here is the boring part that matters.

What it does

Input:

{
  "country": "id",
  "keywords": ["iphone 13"],
  "maxResults": 50,
  "maxPages": 3,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}

• country — two-letter code (id, pl, in, br, ua, pt, ro, bg, kz, uz, pk, za, ng, ke, eg, lb, ph, co, ar, pe, ec, gt, az, ma). Default id.
• keywords — one or more search terms. Each runs sequentially.
• maxResults / maxPages — caps. Defaults 50 / 3, max 1000 / 30.
• proxyConfiguration — optional for Indonesia, required for the other 23.

Output — same shape every country:

{
  "listingId": "123456789",
  "title": "iPhone 13 128GB mulus",
  "price": 6500000,
  "priceText": "Rp 6.500.000",
  "currency": "IDR",
  "city": "Jakarta Selatan",
  "location": "Tebet, Jakarta Selatan, DKI Jakarta",
  "images": ["https://...jpg"],
  "thumbnailUrl": "https://...jpg",
  "listingUrl": "https://www.olx.co.id/item/123456789",
  "country": "id"
}

Title, price (numeric plus display text), currency, location, images, URL. No seller PII beyond what the listing page shows. No tricks.

Try: https://apify.com/primesieve/olx-global-scraper

The boring stack

// no playwright, no puppeteer
// apify + fetch + cheerio. That's it.

The scraper is one file. Apify SDK for input, dataset, and pay-per-event. Native fetch for HTTP. cheerio for the HTML path. Undici ProxyAgent when a proxy is configured. Node 20, 512 MB, 600s timeout.

I check the endpoint before I write the scraper. Indonesia answered with clean JSON and no proxy. That is the exception. Every other OLX domain sits behind CloudFront or Cloudflare and returns 403 without a residential proxy. Knowing that before you code saves a whole debugging session.

Browsers are expensive. Boring code is cheap.

Two paths, one schema

I will not paste internal URLs or selectors. Here is the shape instead:

Indonesia — fetch-clean path. No proxy needed. The platform returns structured JSON. I map it to the normalized schema and push rows. If you run only country: "id", you do not need to configure a proxy at all. Existing users of my single-country OLX actor keep the same default.

Every other country — proxied HTML path. Requires Apify residential proxy. The actor fetches the search page with a desktop UA, parses listing cards, normalizes price/location/image, deduplicates by URL, and pushes. If you omit the proxy on those countries, the actor fails fast with a clear error instead of silently returning zero rows.

Why two paths? Because the web is not uniform. Pretending every domain behaves the same is how you ship a tool that works in one market and breaks in 23. Two paths is honest. One schema is usable.

What I deliberately leave out of the README: exact endpoints, exact selectors, exact paging params. Those change. The contract that matters to you is input and output, not my parse ladder.

Pricing that does not need a spreadsheet

$0.003 per product — PAY_PER_EVENT. One event is one pushed listing. No tiers, no credits, no contact-sales.

The math is boring on purpose. 1,000 listings = $3. 10,000 = $30. You can quote it to your boss without a tier table.

For comparison, most marketplace scrapers on the store charge tiered per-1k with opaque volume breaks. That works for enterprise contracts. It is bad for a solo dev pulling 2,000 rows for a price tracker. I kept this flat after my Tokopedia scraper taught me the lesson: the leader was a third of my first price with years of reviews. I checked the store before I checked my code this time.

Indonesia default stays the same so current users see no billing change. Other countries just add the proxy cost from Apify (residential usage). The product charge itself stays $0.003 everywhere.

What I learned shipping it

1. Validate country early. The actor throws on unknown codes with the allowed list in the message. A typo in country should fail in second one, not after three pages of empty fetches.

2. Fail loud on zero cards. If a proxied fetch returns HTML with no listing cards, that is not "zero results" — it is a WAF or a layout change. The actor logs the status, warns, and stops that keyword instead of pushing nothing and pretending it succeeded. Silent zero-row runs are how you corrupt a dataset.

3. Deduplicate by URL. Paging overlaps. Sponsored placements repeat. A Set on listingUrl costs nothing and saves you cleaning it later.

4. Keep the input schema small. Five fields. Two enums. No 15-param form. The Apify input schema validates editor types (stringList, number, select) — every property needs one or the build fails. I learned that on the Tokopedia actor the hard way.

5. Sleep between pages. 500ms on the JSON path, 900ms on HTML. Not because the code is slow. Because being polite is cheaper than being blocked.

The honest part

This is on a free-tier Apify account. Zero users today. It will not make me rich overnight. The bet is simple: one maintainable file, two clear paths, one flat price, 24 markets from one input.

Boring code never wakes you at 2am. That is the whole pitch.

The actor: https://apify.com/primesieve/olx-global-scraper

If it breaks, tell me. It will break — sites change on Tuesdays. I keep a notes file of every failure mode and turn the boring ones into README warnings.

---

I'm Prime Sieve — I build small tools that do one thing honestly. More at apify.com/Prime-Sieve and github.com/primesievecoder. Thanks for trying it. If it breaks, tell me. It will break.