DEV Community

Devil Scrapes
Devil Scrapes

Posted on

PropertyFinder.ae's category parameter wasn't what the spec assumed. We tested it before shipping.

Quick answer

We tested PropertyFinder.ae's c= search parameter before trusting it, and it wasn't what the spec assumed. The written spec read c= as a plain residential-vs-commercial switch. The wire disagreed: c=1 is residential-sale, c=2 is residential-rent, c=3 is commercial-sale, c=4 is commercial-rent — one parameter carries both category and transaction type. Shipped on the original reading, the PropertyFinder UAE Property Scraper would have quietly handed a customer asking for for-sale listings a page of for-rent ones. We caught it before it shipped, not after a refund request.

Does PropertyFinder.ae's c= parameter just mean residential vs commercial?

No, and assuming so is the mistake worth flagging for anyone building against this target directly. c= is a combined category-and-transaction axis, not two independent filters bolted together — category and purpose in this Actor's input map onto that single parameter internally so you never have to know the encoding. The listings themselves live in the page's embedded __NEXT_DATA__ JSON blob — roughly 584,000 characters of it — at props.pageProps.searchResult.listings, which is a far more reliable source than parsing rendered DOM, but the query encoding sitting in front of that JSON still has to be verified against real requests, not inferred from the parameter's name.

What happens when you ask for a bedroom filter that doesn't exist on the wire?

There is no server-side bedroom parameter on this target — we checked, and it isn't there. That left two options: invent a query param the server silently ignores, which looks like it worked and does nothing, or filter client-side on the parsed rows and say so plainly in the docs. We chose the second. An unverified filter is worse than a missing one, because a filter the server ignores returns a plausible, confidently wrong result set, and nothing in the output tells you it was never applied. We'd rather ship an honest client-side filter than a fake server-side one. We also confirmed the failure mode on the other end: an absurd price range returns a genuine zero-match SUCCEEDED run with a status message — not a crash dressed up as a bug.

An unverified filter is worse than a missing one — a filter the server ignores returns a plausible, confidently wrong result set, and nothing in the output tells the buyer it was never applied.

What the Actor gives you

One row per listing: listing_id, title, price_amount + price_currency, property_type, bedrooms, bathrooms, size_value + size_unit, location and location_hierarchy (city → community → sub-community), agent_name, agency_name, listing_url, image_urls, posted_at / updated_at, and latitude/longitude where PropertyFinder's own JSON carries them. A real sampled row: a 4-bedroom villa in Meadows 7, Meadows, Dubai, AED 23,500,000, location_hierarchy ["Dubai", "Meadows", "Meadows 7"], with agent name and coordinates attached. Proxy is RESIDENTIAL, pinned to AE — forced in code, not trusted from whatever the run's prefill happens to say, because a geo-random exit on a country-scoped portal is a known way to get wrong-currency or wrong-inventory data back.

Honest limitations 🚧

v1 is search-results only — full description text, amenities lists, and floor plans live on the detail page and aren't fetched. location_hierarchy, latitude/longitude, and agency_name are null when PropertyFinder's own JSON doesn't carry them for a given listing — we don't backfill guesses. bedrooms is applied client-side, as covered above, not as a confirmed server-side filter. Non-UAE PropertyFinder country portals are out of scope for this Actor.

FAQ

Does category/purpose map onto one parameter or two on the real site?
One — c= is a combined category-and-transaction axis (1–4), not two independent switches. This Actor encodes it correctly so you just pick category and purpose and never touch the raw parameter.

Is the bedroom filter applied by PropertyFinder's server or by the Actor?
By the Actor, client-side, on the parsed rows — there's no confirmed server-side bedroom parameter on this target, and we'd rather filter honestly after the fact than send a query param the server silently ignores.

Does a zero-result search count as a failed run?
No — an absurd price range verified as a genuine SUCCEEDED run with zero rows and a status message, not a crash.

$0.20 per run plus $0.008 per listing row — $8.20 per 1,000 results.

→ PropertyFinder UAE Property Scraper on Apify


Built by Devil Scrapes 😈. We rotate Chrome/Firefox TLS fingerprints, retry with backoff, pin the proxy exit country where the target needs it, and verify every filter against the real wire before we ship it as working.

Top comments (0)