Residential proxies aren't the top tier: what 4 marketplace scrapers taught us

Short version: the proxy tier that gets you into one marketplace will get you thrown out of the next one. We shipped four listing scrapers in the same week — Craigslist, OfferUp, BBB, and Booking.com — and each one needed a different transport tier to return a single honest row. The most expensive lesson: a residential proxy is not automatically the strongest option. On bbb.org, plain datacenter IPs worked where residential and a full stealth browser both got refused.

Here's the tier-by-tier breakdown, and how to pick without burning a week guessing.

🧭 Why "just use residential" is bad advice

The folk wisdom goes: datacenter IPs are cheap and blocked, residential IPs are expensive and work, and if residential fails you escalate to a real browser. One ladder, climb until it works.

Real targets don't respect that ladder. Bot-management vendors score many signals — TLS handshake, HTTP/2 frame ordering, header casing, IP reputation, ASN, request cadence — and different vendors weight them differently. A residential IP pool that's been hammered by every scraper on the internet can carry worse reputation than a clean datacenter range. Escalating costs you money and latency and can actively make things worse.

The only reliable method is to probe each tier and record what happened. Which is what we did.

🏢 BBB: the one that broke the ladder

bbb.org fronts every page with a bot-management challenge. By the folk ladder it should be a residential-or-browser target.

It isn't. We probed three tiers:

Tier	Result
WebShare residential	403
Camoufox (stealth Firefox)	403
Apify datacenter	✅ clears, returns rows

The cheapest tier was the only one that worked. We don't have the vendor's scoring table, so we won't pretend to explain exactly why — the plausible read is IP reputation on the shared residential pool, but that's inference, not a claim. What matters operationally is the rule it forced on us:

A "this site is unscrapable" verdict is only valid if it names every tier you probed. Otherwise you haven't found a blocked target, you've found an untested one.

We now write that tier table into the spec for every anti-bot target before anyone writes a parser. It has already rescued more than one actor that a single failed probe had written off.

The scraper itself returns business name, phone, address, BBB letter rating, accreditation status, and years in business — the accreditation flag being the reason SMB lead-gen teams want it at all. $1.22 / 1,000 results.

🛏️ Booking.com: a JS gate, and a price you can't trust

Booking.com is the opposite case — raw HTTP genuinely doesn't survive. There's a JS-execution gate in front of the content, so this one needs a hardened browser layer that renders the page like a real visitor.

But the transport is the easy half. The nasty part is that Booking.com happily serves you a perfectly valid page in the wrong currency and the wrong locale, with no error at all.

This is the failure mode we've come to fear most across the whole fleet: not a block, but a plausible wrong answer. A 403 is loud and you fix it. A 200 OK carrying prices in the wrong currency flows straight into someone's pricing model and nobody notices for a month.

So the actor verifies that the currency and locale on the rendered page actually match what was requested before it trusts a single price. Rows that fail that check never get emitted. Same reasoning as pinning your proxy country: a geo-random exit doesn't throw, it lies.

Search mode takes a destination plus dates; direct-URL mode takes hotel URLs. Rows carry name, address, coordinates, star rating, review score, price, currency, and room type, with optional guest reviews. $2.00 / 1,000 hotels.

🏙️ Craigslist: the problem is fan-out, not defenses

Craigslist is a different shape of hard. The per-page defenses are manageable; the pain is that Craigslist is really hundreds of separate metro sites, and the naive approach is one run per city.

Fan-out across dozens of metros × seven categories means the interesting question stops being "can I fetch this page" and becomes "what happens when combo 34 of 200 goes bad?"

If one blocked category can kill the run, you lose 199 good combos to one bad one. That's the single most common cause of a low-success-rate scraper we've found in our own fleet — a recoverable error crashing the whole run instead of skipping one item. So faults are isolated per combo: a slow metro or a blocked category logs a warning, gets skipped, and the run continues. Partial progress is always reported with an explicit status message, never as a silently short dataset.

One field worth calling out because it's hard to reconstruct after the fact: is_edited_or_price_drop, derived from posted_at vs updated_at. If you're tracking repost and price-drop velocity, that signal is gone the moment you only snapshot current state. $2.01 / 1,000 results.

📱 OfferUp: geography is an input, not an accident

OfferUp is a local-first marketplace, which makes location a first-class input rather than a detail. Results for "dining table" in Austin and in Newark are different datasets, not different pages of one.

That makes a drifting exit IP a correctness bug, not a performance one. The pool is US-pinned so the geography stays put, and you can pin a ZIP code with a radius on top of it. Same principle as the Booking.com currency check, arrived at from a different direction: if geography changes your results, geography belongs in your schema, not in your proxy config's mood.

Rows include price, condition, location, seller rating, and photos, with an optional detail pass for full description, category, seller rating, and precise coordinates. $1.83 / 1,000 results.

🧪 How do you actually choose a tier?

Cheapest first, and write down every result:

1. Plain datacenter with real browser TLS. We use curl-cffi to impersonate Chrome/Firefox/Safari handshakes — a huge share of "blocked" targets are only rejecting Python's TLS fingerprint, not your IP. Try this before you spend anything.
2. Datacenter, country-pinned. Pin the exit country explicitly. Geo-random exits return plausible wrong pages rather than errors.
3. Residential. Sometimes better. Sometimes, as with BBB, worse.
4. Stealth browser. Real JS execution for genuine JS gates. Slowest and priciest — earn your way here, don't start here.

Stop at the first tier that returns correct rows, not the first that returns 200s. Those are different tests, and only the second one is worth anything.

❓ FAQ

Is a residential proxy always better than datacenter?
No. On bbb.org, datacenter cleared while WebShare residential and a stealth browser both got 403s. Probe, don't assume.

Why not translate every marketplace into one universal schema?
We normalize structure — field names, types, nullability, units — aggressively, and vocabulary never. Forcing source values into a tidy enum drops the categories that don't map, silently, in a dataset that still looks clean.

What's the most dangerous scraper failure?
The 200 OK with wrong or empty content. A hard block is self-announcing. A plausible wrong answer ships to production. Both the Booking.com currency check and OfferUp's pinned geography exist for exactly this reason.

Do these need a credit card to try?
No. All four are pay-per-result on the Apify Store — you pay for rows that land in your dataset.

📦 The takeaway

Four marketplaces, four different transport tiers, one shared rule: the tier is a per-target empirical finding, not a global setting. Write down every tier you probed. The scraper you save will be the one somebody already wrote off.

We do the dirty work so your dataset stays clean. 😈