Short version: three review/listing platforms, three different reasons you stop getting more data. Redfin's 350-listing cap is a countable server limit, and we bisect around it with price bands. Apple's ~50-review ceiling on its customer-reviews feed is architectural and permanent — page 2 is dead, not blocked, and no amount of retrying brings it back. Glassdoor's ~3-full-reviews-per-page-load ceiling for anonymous visitors isn't a count limit at all; it's how much of the page actually renders before you're not logged in. Treat all three as "a rate limit" and you'll either burn a week chasing a wall that was never there, or under-price a scraper that's doing far more work than the row count suggests.
Here's each ceiling's real shape, how we found it, and why the fix is different every time.
🧱 Why "just paginate harder" is the wrong first question
The instinct when a scraper stops returning data past some point is to assume a rate limit and add a delay. Sometimes that's right. It wasn't right for any of the three targets below. Each one hit a hard stop that had nothing to do with request pacing — a server-side count cap, a feed that was never built to page past one screen, and a rendering budget for logged-out visitors. Backing off and retrying does nothing for any of them. The fix has to match the shape of the ceiling, and the three shapes below cover most of what you'll run into scraping review and listing data anywhere.
🏠 Redfin: a countable ceiling, so we count around it
Every call to Redfin's gis search endpoint returns at most 350 homes, full stop, no matter what you ask it for. Ask for a dense metro's for-sale inventory and the 351st home just doesn't come back — no error, no truncation flag, nothing that tells you to keep going.
Because the cap is a count against a filterable result set, it's defeatable: split the price range into buckets, request each bucket separately, and recurse into any bucket that still comes back at 350 (mpt=99 plus a min_price/max_price pair does the splitting). We stop bisecting once a bucket's width drops below a configurable floor ($25,000 by default) so a genuinely thin price band doesn't get sliced forever. Every home gets deduped by property_id across buckets, because the same listing straddles bucket boundaries more often than you'd expect. Each bucket also fails in isolation — a slow or blocked bucket gets logged and skipped rather than taking the whole search down.
That's the Redfin Property Listings Scraper: normalized for-sale/sold/pending listings with MLS id, days-on-market, HOA, exact lat/long, and open-house windows, at $2.52 per 1,000 results. The cap here is annoying but honest — it always tells you the same number, and a countable ceiling is the one shape you can engineer straight through.
🍎 App Store reviews: a ceiling that was never a wall to climb
Apple's customer-reviews RSS/JSON feed looks like it should paginate — the URL has room for a page number, and every review-scraping tutorial on the internet assumes page=2 works. During recon on 2026-08-15 we tried every path-segment ordering we could construct for page 2 and higher against that feed. All of them came back dead. Page 1 — roughly 50 reviews — is the entire dataset Apple will hand you per app, forever, and we scoped the code to never even build a page parameter, because a permanent dead end doesn't deserve a retry loop pretending otherwise.
That's a different kind of ceiling than Redfin's. There's no bucket to bisect and no query shape that unlocks more. The only honest move is to say so — in the input field description, the README, and the pricing — instead of quietly under-delivering against an implied "as many as you ask for."
Where this Actor earns its keep is the half of the ceiling that isn't fixed: Google Play's review RPC does paginate toward whatever cap you set. The App Store Reviews & ASO Scraper fetches both stores for the same app in one run — one unified schema with a store discriminator — and computes a per-app ASO rollup on top: rating distribution, month-by-month rating trend, review velocity, and top praise/complaint keywords via a deterministic stoplist-frequency count (no sentiment ML, no black box). Every incumbent we found on the Store covers exactly one platform; nobody else stitches Play and Apple into one comparable ASO view. Priced at $1.20–1.50 per 1,000 rows blended across a $0.05 warm-up, $0.0012 per review, and $0.005 per rollup row.
💬 Glassdoor: a ceiling that isn't measured in rows at all
The Glassdoor case is the one that breaks the "it's a rate limit, add a delay" instinct hardest, because the ceiling isn't about how many pages you can request — it's about how much of one page renders for a visitor who isn't logged in. Anonymous access surfaces roughly three full review entries per page load; the rest of that page's reviews exist in the DOM in truncated or teaser form. You can absolutely request page 20. You'll get a 200 and a page. You will not get twenty pages' worth of full review text out of it.
Getting onto that page at all is its own problem — Glassdoor's review pages sit behind bot defenses that a plain HTTP request doesn't survive, so the Glassdoor Reviews Scraper runs a full browser session: a proven navigation sequence through the same browser context and page object, not a fresh request per URL, with the proxy session pinned for the life of that context. One company failing to resolve or parse never sinks the run — it's logged, skipped, and every other company still completes.
Because the real throughput is ~3 full reviews per page load, not the page count you asked for, maxReviewsPerCompany and pricing are built around that reality instead of promising cheap bulk collection we can't actually deliver without a login we refuse to use. That's also why this one sits at the top of our usual range: $10.00 per 1,000 review rows, because each row costs real browser-navigation compute, not a cheap HTTP round-trip.
🧪 The three shapes, side by side
| Target | What the ceiling actually is | Can you defeat it? | The right response |
|---|---|---|---|
| Redfin | Server-side count cap (350/call) on a filterable set | Yes — partition the filter (price bisection) and dedup | Bisect, recurse, dedup by ID |
| Apple App Store feed | Architectural — page 2+ doesn't exist, verified dead | No | Scope it out permanently, state it in docs and pricing |
| Glassdoor (anonymous) | Rendering depth per page load, not a count | No, without a login we won't use | Price and cap around the real per-page yield |
Mistaking any row of that table for either of its neighbors is where scrapers go wrong: bisecting against an architectural ceiling wastes requests on a wall that logs nothing new; retrying against a rendering-depth ceiling just re-fetches the same three reviews with extra latency; and quietly capping a countable ceiling "for safety" leaves real, gettable data on the table.
❓ FAQ
Can I get more than ~50 Apple App Store reviews per app?
Not from this feed. Page 2 and beyond returned nothing across every URL shape we tried during recon — it's a permanent, one-page ceiling on Apple's side, not a setting we could raise.
Does Redfin's price-bucket bisection risk getting rate-limited?
We retry with exponential backoff on 408/429/5xx and rotate the session on a block, same as every Actor in the fleet — bisection adds more requests per search, not more requests per second.
Why is Glassdoor priced at $10/1,000 when the others sit closer to $1–2.50?
Because the real unit of work is a full browser navigation per company, not a cheap API call, and anonymous access only renders a handful of full reviews per page load — the price reflects the compute that actually happens, not a theoretical row count.
Do any of these need a login or API key?
No, for all three. Apple and Google Play are public feeds; Glassdoor and Redfin are scraped from their own public pages, no account, no credentials.
Will one bad app or company stop the whole run?
No — all three Actors isolate faults per item. A resolution failure or empty result for one entry gets logged and skipped; the rest of the run completes and the run's status message reports exactly what happened.
📦 The takeaway
"That's all the data you get" means three different things depending on which wall you hit. A countable ceiling gets engineered through. An architectural one gets documented and priced around, not chased. A rendering-depth ceiling gets measured honestly so the price matches the real work, not the row count on the label. We built all three read on the ceiling into the Actor before it ever shipped, so the number on the pricing page is the number you'll actually get — not the number the target's marketing implies.
We do the dirty work so your dataset stays clean. 😈
Top comments (0)