DEV Community

Cover image for Building a Universal Property Listing Scraper with Python and ...
Oaida Adrian
Oaida Adrian

Posted on Edited on Originally published at apify.com

Building a Universal Property Listing Scraper with Python and ...

#webscraping #python #data #tutorial

Building a Universal Property Listing Scraper with Python and JSON-LD

Ever wanted to extract structured data from real estate listings across multiple sites without writing custom parsers for each one? I built a Property Listing Scraper that works with any property website — from Zillow to Rightmove to Imobiliare — using a combination of JSON-LD, OpenGraph metadata, and smart HTML pattern matching.

The Problem

Every real estate site structures its data differently. Zillow uses one schema, Rightmove uses another, and smaller sites like Imobiliare.ro have their own format entirely. Building separate scrapers for each is a maintenance nightmare.

The Solution: Multi-Layer Extraction

The scraper uses three extraction layers, falling back gracefully:

1. JSON-LD (Structured Data)

Many modern property sites embed structured data using Schema.org vocabulary. This is the gold standard — clean, machine-readable, and standardised.

for script in soup.find_all("script", type="application/ld+json"):
    data = json.loads(script.string)
    if "Product" in str(data.get("@type", "")):
        # Extract price, address, coordinates, images
Enter fullscreen mode Exit fullscreen mode

2. OpenGraph Meta Tags

When JSON-LD isn't available, we fall back to OpenGraph metadata that most sites provide for social sharing.

og_title = soup.find("meta", property="og:title")
og_image = soup.find("meta", property="og:image")
Enter fullscreen mode Exit fullscreen mode

3. Regex Pattern Matching

As a final fallback, we scan the page text for common price and property patterns:

price_match = re.search(r"([$£€]\s*[\d,]+(?:\.\d+)?)", text)
bed_match = re.search(r"(\d+)\s*(?:bed|bedroom|camera)", text)
Enter fullscreen mode Exit fullscreen mode

Extracted Data Fields

Each property listing yields:

  • Price & Currency — parsed from any format (USD, EUR, GBP, RON)
  • Property Type — apartment, house, studio, land, office
  • Bedrooms & Bathrooms — multi-language support (English, Italian, Romanian)
  • Area — square metres detection
  • Address Components — street, city, region, postal code, country
  • Coordinates — latitude/longitude from geo data
  • Images — up to 20 high-quality images per listing

Search Page Support

The scraper automatically detects whether a URL is:

  1. An individual listing — extracts data directly
  2. A search results page — discovers listing links, then scrapes each one concurrently (5 parallel requests)

Try It

You can use the scraper right now:

The Apify actor uses pay-per-event pricing at $0.01 per property extracted — you only pay for actual results.

Use Cases

  • Market Analysis: Track property prices across multiple markets
  • Lead Generation: Build databases of available properties
  • Price Monitoring: Watch specific neighbourhoods for price changes
  • Investment Research: Compare yields across countries and currencies

Conclusion

By combining JSON-LD extraction with meta tags and regex fallbacks, we achieve universal compatibility without sacrificing data quality. The scraper handles the messy reality of real estate websites so you can focus on analysing the data.

Built with Python, BeautifulSoup4, httpx, and the Apify platform.

More from me

Top comments (0)