Scraping Real Estate Data with Apify Domain.com.au Actor

Original post
In this blog post, I’ll walk you through the Apify actor I built for scraping real estate data from Domain.com.au. This actor is capable of extracting a wide range of property details and formatting the results in CSV. Whether you are a real estate professional or a developer building real estate data applications, this post will provide you with insights into the actor's design and how it functions.

Why Scrape Domain.com.au?

Domain.com.au is one of Australia's leading real estate platforms, offering thousands of property listings across various states. Extracting this data can offer valuable insights into property trends, market prices, and agent details.

Actor Configuration

The actor has been designed with flexibility in mind, allowing you to control several key parameters:

• CSV-Format: If set to true, the results will be formatted as a CSV.
• Data-Center Proxy: If you need to use data-center proxies, set this to true (default is false).
• Residential Proxy: For more anonymous scraping, this can be enabled.
• Residential Proxy: For more anonymous scraping, this can be enabled.
• Max Pages: Limits the number of pages to scrape (in this case, set to 50).
• Start URLs: A list of the URLs to start scraping from.

Here's an example of the input configuration for the actor:

{
  "CSV-Format": true,
  "data-center-proxy": false,
  "residential-proxy": false,
  "fullScrape": true,
  "maxPages": 50,
  "startUrls": [
    {
      "url": "https://www.domain.com.au/sale/melbourne-region-vic/?excludeunderoffer=1&lastsearchdate=2024-09-30t19%3a02%3a18.625z&price=200000-9000000"
    }
  ]
}

Data Extracted

The actor gathers rich details about properties, including images, pricing, and agent information. Below is a sample output showing a single property listing:

{
  "id": 2019540812,
  "promoType": "premiumplus",
  "url": "https://domain.com.au/25-barolo-loop-sunbury-vic-3429-2019540812",
  "images": "https://rimh2.domainstatic.com.au/X0PjH0WBOkpHSSFzEJOTXPOGZeY=/660x440/filters:format(jpeg):quality(80)/2019540812_1_1_241004_051134-w6243-h4161, https://rimh2.domainstatic.com.au/SU26or2Jg3QQkVtR8fK_nBj8h1I=/660x440/filters:format(jpeg):quality(80)/2019540812_2_1_241004_051134-w6242-h4162",
  "brandingAppearance": "light",
  "price": "$800,000",
  "hasVideo": false,
  "agencyId": 5265,
  "agentNames": "Haydn Drummer",
  "brandLogo": "https://rimh2.domainstatic.com.au/62d7jeDqoKkMbJIjcpPeD-0NUfg=/170x60/filters:format(jpeg):quality(80)/https://images.domain.com.au/img/Agencys/5265/logo_5265.png",
  "agentPhoto": "https://rimh2.domainstatic.com.au/0QkOHXPomsvFpMiUI4oVwuZVhQA=/90x90/filters:format(jpeg):quality(80)/https://images.domain.com.au/img/5265/contact_738630.jpeg",
  "street": "25 Barolo Loop",
  "suburb": "SUNBURY",
  "state": "VIC",
  "postcode": "3429",
  "lat": -37.588314,
  "lng": 144.75046,
  "beds": 4,
  "baths": 2,
  "parking": 2,
  "propertyTypeFormatted": "House",
  "landSize": 448,
  "landUnit": "m²",
  "openTime": "2024-10-09T17:00:00",
  "closeTime": "2024-10-09T17:30:00"
}

As you can see, the actor provides essential information such as the property’s price, address, images, and agent details, along with latitude and longitude data that can be used for mapping.

Key Features of the Actor

1. Property Details: The actor extracts core information like the number of bedrooms, bathrooms, parking spaces, and land size.
2. Images: URLs of multiple images for each listing are captured, which can be useful for property galleries.
3. Agent Info: Details about the selling agent, including name, photo, and contact information.
4. Open House Data: The actor scrapes the open house schedule, including start and end times.
5. Geo-Location: Latitude and longitude are included, enabling easy integration with mapping tools.
6. CSV Output: The results can be exported as CSV files for further analysis or integration into other systems. ## Use Cases
7. Market Research: Analyze real estate trends across different areas.
8. Lead Generation: Collect contact information of agents for marketing or collaboration.
9. Property Listings: Integrate the data into your platform to provide users with up-to-date listings.

Conclusion

This Domain.com.au Apify actor is an efficient way to gather real estate data for analysis, market trends, and competitive research. With customizable inputs and a wealth of property data at your disposal, the possibilities for integration into business solutions or data analysis are endless.

For custom solutions

In case you need some custom solution, you can contact me : [e-mail] , Upwork
Or learn more about me on Github