DEV Community

Devil Scrapes
Devil Scrapes

Posted on

Redfin silently drops your price filter unless you set one undocumented parameter

Quick answer

Redfin's search API caps every response at 350 homes, no matter what you ask for. That part is documented behavior and easy to work around with pagination — except the obvious way to page past it doesn't work, because of one query parameter almost nobody sets: mpt. Leave it at its default and your min_price/max_price filters are silently ignored — you get back the same unfiltered top-350-by-relevance every time, dressed up as a filtered result. Set mpt=99 and the same filters start working. Nothing in the response tells you which behavior you're getting.

Does paging past 350 results just work if I add min_price and max_price? 🏠

No, and this is the finding that makes or breaks a Redfin scraper at any real scale. Redfin's stingray/api/gis search endpoint — the JSON call its own search pages fire internally — hard-caps num_homes at 350 server-side, full stop:

GIS_BASE_URL = "https://www.redfin.com/stingray/api/gis"
REDFIN_PAGE_CAP = 350
Enter fullscreen mode Exit fullscreen mode

The intuitive fix for any dense metro is to bisect by price: fetch $0–$500k, then $500k–$1M, and so on, so no single bucket exceeds the cap. That's the right idea. It just doesn't work by adding min_price/max_price alone, because of a third parameter riding along with every request:

MPT_BISECT = 99

# The default mpt=1 silently ignores min_price/max_price and returns the
# unfiltered top-350 by relevance — live-confirmed required behavior.
Enter fullscreen mode Exit fullscreen mode

mpt defaults to 1 on a plain search request, and at that value, Redfin's endpoint answers your price-bounded query with the same 350 homes it would return with no price bounds at all — sorted by relevance, not filtered by price. The request looks correct. The response looks like data. It is the wrong 350 homes, silently, every single bucket, and there is no error, no warning header, nothing that flags the filter as ignored. Only mpt=99 makes the price bounds actually bind.

Bisection without that one parameter isn't broken pagination — it's the illusion of pagination. Every "page" after the first returns duplicates of the same top-350 relevance-sorted set, and a naive scraper reports full coverage of a market it only sampled once.

Can a filter silently doing nothing be worse than a filter that errors? 🔍

Yes, and this is the general shape of the bug worth internalizing. An API returning 400 Bad Request for a malformed filter is annoying but honest — you find out immediately. An API that accepts a filter, ignores it, and returns a plausible dataset anyway is the failure mode that survives code review, survives a manual spot-check of the first few rows, and only shows up when someone notices the "$2M+ homes" bucket and the "under $200k" bucket returned suspiciously identical listings.

The only way to catch this class of bug is empirical: run the same query with and without the suspect parameter and diff the actual home IDs returned, rather than trusting that a 200-status response with populated fields means the request did what it looked like it should do.

What else does Redfin do at HTTP 200 that isn't what it looks like? 🧱

A second, related trap: Redfin has been observed geo-redirecting a plain request to a different city — at HTTP 200, with a normal-looking payload of homes for the wrong place. Nothing distinguishes that response from a correct one except the actual city names inside it not matching what you asked for.

We guard for this with a two-signal check: parse the geo hints out of the requested search URL itself (city, state, region type) and compare them against the geo fields actually present in the returned homes. A mismatch triggers one session rotation and one retry before the entry is logged and skipped — never silently accepted as if it were correct.

class _GeoMismatchError(Exception):
    """A search entry's response failed the geo-splash guard twice."""
Enter fullscreen mode Exit fullscreen mode

Two unrelated defenses, one shared shape: Redfin's search endpoint will hand you a syntactically valid, semantically wrong answer, and a 200 status code is not proof that the content matches the request.

The part that generalizes 🧭

Test a filter by diffing results, not by checking the status code. A parameter that's silently no-op'd will pass every test that only checks "did the request succeed" and fail every test that checks "does bucket A actually differ from bucket B."

When an endpoint returns homes/listings/records tied to a location or a range, verify the response matches the request — not just that it parsed. A geo-mismatch and a silently-ignored filter both produce clean-looking, wrong data, and both are cheaper to catch with one extra comparison than to discover downstream in a customer's deal-flow report.

What the Actor gives you

One deduped row per property, across as many price-bucket pages as it takes to cover a market:

  • full MLS/geo field coverage — MLS ID, days-on-market, HOA presence, exact lat/long, listing remarks and tags, open-house windows, photo counts
  • reliable pagination past the 350-home cap via confirmed price-bucket bisection (mpt=99), deduped by property_id across buckets
  • geo-splash guard that rotates and retries once on a location mismatch, then skips rather than returning wrong-city data
  • two-level fault isolation — per search entry and per price bucket — so one bad entry never kills the run
  • for-sale, sold, and pending listing status, from any /city/, /zipcode/, /neighborhood/, or /county/ search-page URL

The honest limitations 🚧

US listings only in v1 — no Canada. Price history, tax/assessment records, and agent-contact fields aren't available: they don't ship in the search-page payload this Actor reads, and getting them means Redfin's property-detail pages, which sit behind a real anti-bot challenge a plain HTTP client can't clear. Every field confirmed on the search endpoint ships reliably; nothing here is a guess.

FAQ

Why does my bucketed search sometimes return the exact same homes across different price ranges?
The bisection request is missing mpt=99. Without it, Redfin ignores your price bounds and returns the same unfiltered top-350 every time.

Does Redfin ever redirect a search to the wrong city?
Yes, at HTTP 200 with a normal-looking payload — no error, no distinguishing status. We check returned geo fields against the requested URL's city/state and retry on a mismatch.

Can I target a ZIP code or neighborhood instead of a city?
Yes — paste any /city/, /zipcode/, /neighborhood/, or /county/ search-page URL copied straight from redfin.com.

Does one bad search URL stop the whole run?
No. Each entry, and each price bucket within it, is fault-isolated — a failure on one is logged and skipped while the rest of the run continues.

Pricing

$0.20 per run plus $0.0025 per unique result row — about $2.70 per 1,000 results. No subscription, no minimum, no card to start.

Redfin Property Listings Scraper on Apify


Built by Devil Scrapes. We handle the price-bucket bisection, the geo-splash guard, and the parameters that quietly do nothing, so you get a flat table instead of a weekend.

Top comments (0)