DEV Community

uzie
uzie

Posted on

Dubai property listings: a schema for price-per-sqm analysis

#datascience #dataset #realestate #api

Dubai property data looks easy until you try to compare two listings. One portal shows
"2BR, Dubai Marina, AED 2,100,000"; the next shows "2 Beds, 1,180 sqft, Marina, 2.1M".
Same building, different fields, no shared key. Multiply that by a few thousand listings
and any price-per-sqm analysis becomes a manual spreadsheet exercise.

This post documents the schema I settled on for Dubai listings, the parts that actually
matter for analysis, and a free 40-row sample you can inspect.

Why listing data is harder than it looks

  1. The listing id is not the property id. The same unit can be re-listed by two brokers with different ids. Dedupe on the portal id, then cluster on coordinates + size + bedrooms if you need property-level counts.
  2. Price is not comparable across transaction types. A sale price and a yearly rent both look like integers. Without transactionType and pricePeriod, averages are meaningless.
  3. Size is published in both units, inconsistently. Some records carry sqft, some sqm, some both rounded differently. Keep both and store the conversion you used.
  4. Community metrics are snapshots. "Average community price" is computed by the portal at collection time. It is useful context, not a time series.
  5. Broker contact data is personal data. If you publish a sample, strip names, phones, emails, and full descriptions. The listing facts are enough for analysis.

The schema

Each record is one listing:

Field Why it matters
listingId Stable dedupe key across runs
url Canonical source for verification
price, currency, pricePeriod Comparable only when read together
transactionType, propertyType Sale vs rent; apartment vs villa
bedrooms, bathrooms Studio = bedrooms: 0, not null
sizeSqft, sizeSqm Both units, no silent conversion
pricePerArea Portal value when present; null otherwise
city, community, subCommunity Location hierarchy for grouping
completionStatus Ready vs off-plan changes the price logic
isVerified, isExclusive, isNewConstruction Source flags, not guarantees
latitude, longitude Approximate; useful for cluster checks
listedDate, scrapedAt Snapshot integrity
communityAvgPrice, communityPriceTrend12mPct, communityActiveListings Context at collection time
rera Permit string when the portal publishes it

Two design choices I would defend:

  • Never coerce a missing price to zero. A zero in a price column silently destroys every average. Use null and filter explicitly.
  • Keep portal-computed fields separate from your own. pricePerArea from the portal and price / sizeSqft from your pipeline will disagree at the edges; label which is which.

The sample

A real 40-row snapshot collected on 6 November 2026 from public PropertyFinder search
results (sale and rent). Broker names, emails, phones, and descriptions are removed.

The sample is small on purpose. Forty rows validate a schema; they cannot estimate the
Dubai market.

Limitations and ethics

  • Public listings only. No login, no private data, no broker contact details.
  • Asking prices, not transaction prices.
  • Listings change daily; a snapshot is not a feed.
  • Respect the portal's terms and applicable data-protection rules. Do not republish personal data.

What I packaged

I turned the collection into an Apify Actor so pagination, normalization, and dedupe are
reusable:
https://apify.com/matakupanlabs/dubai-property-listings-scraper

Disclosure: this is my Actor and it is pay-per-listing. The sample and schema are free.
If you are building something similar, I am happy to compare notes on deduplication or
price-per-sqm normalization.

Top comments (0)