How to Scrape PropertyFinder.ae Listings in the Gulf Without a Paid API

#apify #webscraping #api #realestate

The problem

Gulf property data is a licensing problem before it is a technical one. Listings live behind portals like PropertyFinder.ae, and the organizations that legitimately hold the data — licensed data providers, brokerages with active mandates, owners, property managers — receive it as exports, not APIs. Before those records can enter a CRM, a warehouse, or a review queue, someone has to normalize the fields, deduplicate them, attach provenance and freshness, and produce an audit trail that answers "who gave us this row, when, and under what rights?"

The paid alternatives are portal-side scraping subscriptions, which put you on the wrong side of both the portal's terms and your own compliance team. If you already hold an authorized export, what you need is a deterministic normalization and evidence layer — not another crawler.

What the actor does

The PropertyFinder.ae UAE Real Estate Listings actor (slug propertyfinder-gulf, kept for existing Tasks) is the Authorized Gulf Property Listing Intelligence layer. Two modes:

Export mode (default, sourceMode: "export"). Fully network-free. You submit one to 100 closed property records you own or are licensed to process, and the actor normalizes them into review-ready evidence rows. Zero external-source requests: no portal page, no recorded URL fetch, no browser, proxy, API, DNS, robots, or sitemap call. Descriptions, photos, contact fields, and free text are deliberately rejected by the closed contract — unknown fields invalidate the record.
Live mode (sourceMode: "live"). Exactly one bounded request to an allowlisted public www.propertyfinder.ae results URL you supply, with a 15-second deadline, redirect/challenge/2 MB rejection, reading at most 100 RealEstateListing schema.org JSON-LD nodes. Live rows are marked property_listing_live with sourceLicense: null — public visibility is not a licence or proof of availability.

Per the README, each delivered row carries:

The legacy-compatible listing fields (url, title, price, currency, deal_type, property_type, rooms, area_sqm, location, lat/lng, posted_date) plus a deterministic stableId (SHA-256 over source name plus listing ID) for joins.
Provenance and rights context: buyer-supplied sourceName, sourceLicense, optional sourceRetrievedAt, and the exact authorization attestation recorded as an operational gate — not legal proof.
freshness (fresh/aging/stale/unknown) computed only from your supplied retrieval time, change always not_measured (a single snapshot proves nothing), evidence-sufficiency confidence with explicit dataGaps, and mechanical pricePerSqm when area is supplied — explicitly separated from valuation or investment advice.
A mandatory human route: recommendedAction, review priority (HIGH/MEDIUM/LOW from freshness and locality coverage), and safeToAutomate always false.
Settlement-neutral billing intent on the row, with the authoritative paid/free/withheld/unknown reconciliation in the current-run KVS OUTPUT.

Deduplication is scoped to lowercase sourceName plus listingId and never merges across sources. Legacy search-style inputs (deal_type, emirate, max_items, max_pages, max_listing_age_days) are migration-only: one free diagnostic, no network access.

Example: input and output

The documented input (from the manifest and README):

{
  "sourceMode": "export",
  "schemaVersion": "2.0",
  "authorization": "I confirm I am authorized to process and deliver these property records",
  "sourceContext": "provider_licensed_export",
  "listings": [
    {
      "listingId": "listing-101",
      "listingUrl": "https://inventory.example.com/listing-101",
      "title": "Two-bedroom apartment in Dubai Marina",
      "price": 2450000,
      "currency": "AED",
      "dealType": "sale",
      "propertyType": "Apartment",
      "bedrooms": 2,
      "areaSqm": 128.4,
      "location": "Dubai Marina",
      "sourceName": "Licensed Gulf property export",
      "sourceLicense": "Buyer confirms downstream processing and delivery rights",
      "sourceRetrievedAt": "2026-08-12T12:00:00Z"
    }
  ]
}

The README's happy run-bound example shows the KVS OUTPUT receipt for that one paid row (trimmed):

{
  "status": "SUCCEEDED",
  "output": {
    "status": "COMPLETE",
    "input": {"requestedCount": 1, "uniqueCount": 1, "duplicateCount": 0, "invalidCount": 0},
    "run": {"deliveredRowCount": 1, "paidRowCount": 1, "freeRowCount": 0, "withheldRowCount": 0, "replaySafe": false, "safeToAutomate": false},
    "delivery": {"eventName": "result-found", "confirmedEventDelta": 1, "confirmedDatasetWrites": 1, "lastAttempt": {"state": "confirmed_paid"}}
  }
}

Pricing and the free limit

Pay-per-event: $0.005 per actor start plus $0.003 per delivered authorized row (result-found), with tiered discounts on paid Apify plans (the Store headline works out to $2.55 per 1,000 rows). The README computes start plus ten paid rows at $0.035; at list prices a 100-record run costs $0.005 + 100 × $0.003 = $0.305. Apify's free plan includes $5 of usage credits per month, which covers about 16 full 100-record runs — roughly 1,600 delivered rows — before you pay anything. Legacy migration and invalid-record diagnostics are free.

Try it

Normalize one authorized record first, reconcile the Dataset against the KVS OUTPUT receipt, then scale the batch: PropertyFinder.ae UAE Real Estate Listings.

For AI agents and MCP

The actor takes JSON in and returns structured JSON via the Apify API, and the README includes an Apify MCP server setup plus an "MCP or agent workflow" recipe: treat the actor as a deterministic normalization tool — an agent may propose a review task but cannot infer source authority, ownership, valuation, availability, or permission to contact. Integrations must read the current-run KVS OUTPUT as a separate run fact, never copy a final paid/free state into the pre-settlement Dataset row, and never blind-retry an unknown delivery.

DEV Community