DEV Community

Can Yılmaz
Can Yılmaz

Posted on • Originally published at apify.com

Why Imot.bg Bulgaria Real Estate data is more interesting than you would think

#webscraping #apify #realestate #dataengineering

On the surface, Imot.bg Bulgaria Real Estate sounds like the kind of dataset you would file under "boring infrastructure data" -- the sort of thing that lives in a corner of a warehouse and gets queried twice a quarter. After spending a bit of time actually looking at it, I have changed my mind. Here is why.

What is in it

Imot.bg Scraper Bulgaria Real Estate Listings to JSON, CSV & Excel Scrape property listings from imot.bg, Bulgaria's #1 real estate portal, into a clean, structured dataset. Each record carries a fairly rich set of fields:

  • listingId -- listing id
  • listingUrl -- listing url
  • title -- title
  • titleBg -- title bg
  • listingType -- listing type
  • propertyType -- property type
  • price -- price
  • priceCurrency -- price currency
  • priceFormatted -- price formatted
  • pricePerSqm -- price per sqm
  • area -- area
  • rooms -- rooms
  • floor -- floor
  • totalFloors -- total floors
  • constructionType -- construction type
  • yearBuilt -- year built
  • city -- city
  • cityBg -- city bg
  • neighborhood -- neighborhood
  • neighborhoodBg -- neighborhood bg
  • address -- address
  • description -- description
  • descriptionBg -- description bg
  • agencyName -- agency name
  • agencyPhone -- agency phone
  • agencyUrl -- agency url
  • isPrivateSeller -- is private seller
  • imageUrls -- image urls
  • imageThumbnail -- image thumbnail
  • publishedDate -- published date
  • scrapedAt -- scraped at

The interesting bit is the combination. Individually, none of these fields is exotic. Together, they describe an entity precisely enough that you can do real analytics on it -- segmentation, trend analysis, even simple anomaly detection -- without needing a second data source.

Two records from a sample run 

{
  "listingId": "1b176062698062510",
  "listingUrl": "https://www.imot.bg/obiava-1b176062698062510-prodava-dvustaen-apartament-grad-plovdiv-ostromila",
  "title": "Продава 2-СТАЕН",
  "titleBg": "Продава 2-СТАЕН",
  "listingType": "sale",
  "propertyType": "apartment",
  "price": 110000,
  "priceCurrency": "EUR",
  "priceFormatted": "110,000 EUR",
  "pricePerSqm": 1375
}
Enter fullscreen mode Exit fullscreen mode 
{
  "listingId": "1b177874323496598",
  "listingUrl": "https://www.imot.bg/obiava-1b177874323496598-prodava-dvustaen-apartament-grad-sofiya-belite-brezi-ul-nishava",
  "title": "Продава 2-СТАЕН",
  "titleBg": "Продава 2-СТАЕН",
  "listingType": "sale",
  "propertyType": "apartment",
  "price": 234900,
  "priceCurrency": "EUR",
  "priceFormatted": "234,900 EUR",
  "pricePerSqm": 3051
}
Enter fullscreen mode Exit fullscreen mode

When you look at a couple of records side by side the analytical surface area opens up. The categorical fields invite grouping. The numeric fields invite ranking and distribution analysis. The timestamps invite time-series breakdowns. The text fields invite NLP.

Three things you can actually do with this

  1. Build a leaderboard. Pick a numeric field, group by a categorical field, sort. Trivial in SQL or Pandas, surprisingly useful for rental yield analysis, neighbourhood pricing trends, investor due-diligence and market-timing models.
  2. Detect shifts over time. Snapshot the dataset daily, compute simple deltas between snapshots, alert on anything that moves more than a sensible threshold.
  3. Cluster the long tail. The categorical fields probably have a power-law distribution. The long tail is often where the interesting outliers live -- the new entrants, the niche players, the anomalies.

Why it is not just "another scrape"

The reason this dataset is more interesting than typical scrape output: the source has organic structure. The fields are not invented by the scraper, they reflect how the underlying domain organises itself. That gives the dataset a kind of semantic coherence that synthetic or heavily-derived datasets lack.

Caveats

  • Sample sizes from a one-off run will not let you do anything statistically serious -- you want a longitudinal feed.
  • Some optional fields are sparsely populated; check density before relying on them.
  • The source can change. Treat any production pipeline as something that will need maintenance.

How I would prove the analytical thesis

If I were trying to justify investing engineering time in this dataset for a real project, the path would be: pull a one-week recurring sample to get past the snapshot bias, run the three analytical patterns above on the larger pull, and judge whether the conclusions hold up. If you can get a single non-obvious insight out of that exercise, the dataset is worth keeping. If everything you find is something you already knew, it probably is not -- find a different feed. That bar sounds harsh, but it saves you from a portfolio of datasets that nobody actually queries.

For live, customizable extractions of this data, the actor that produced the dataset shown above is published on the Apify Store: logiover/imot-bg-scraper-bulgaria-real-estate. It supports JSON, CSV and Excel exports and runs on a schedule.

Top comments (0)