DEV Community

Bugrahan Saka
Bugrahan Saka

Posted on

Domain.com.au Scraper in Python: Sold Prices, Rentals and Suburb Medians from Australia's Property Portal

Domain.com.au is one of Australia's two big property portals. Every suburb has a page of homes for sale, for rent and recently sold, and every listing carries the price text, sale method and date, beds, baths, parking, property type, address with coordinates, agency and agents. Open a listing and you also get Domain's suburb insights: median sale price, median rent, days on market and auction clearance rate. For price research, rental yield models or tracking a market week by week, that is the dataset. Here is how to pull it into Python.

Why plain requests fail

import requests

r = requests.get("https://www.domain.com.au/sale/sydney-nsw-2000/")
print(r.status_code)  # 403
print(r.text[:60])    # <HTML><HEAD><TITLE>Access Denied</TITLE>
Enter fullscreen mode Exit fullscreen mode

Domain sits behind Akamai, which answers a script, or any datacenter IP, with 403 Access Denied before sending a single byte of HTML. Headers alone don't fix it. Sending the request through Scrape.do with super=true routes it through a residential proxy, and the real page comes back: 10 credits per page, about 4 to 12 seconds.

import json
import re
import requests

TOKEN = "<your_token>"


def fetch(url):
    r = requests.get("https://api.scrape.do/",
                     params={"token": TOKEN, "url": url, "super": "true"}, timeout=90)
    r.raise_for_status()
    return r.text
Enter fullscreen mode Exit fullscreen mode

Where the data is

Domain is a Next.js site, so every page carries its full data as JSON in a <script id="__NEXT_DATA__"> tag. No HTML parsing, no CSS selectors that break with the next redesign:

def next_data(html):
    m = re.search(r'<script id="__NEXT_DATA__"[^>]*>(.*?)</script>', html, re.S)
    return json.loads(m.group(1))["props"]["pageProps"]["componentProps"]
Enter fullscreen mode Exit fullscreen mode

On a search page, componentProps holds:

  1. listingSearchResultIds: the listing IDs on this page, in order.
  2. listingsMap: one entry per ID. listingType is listing for a home and project for a new development; skip the projects.
  3. listingModel inside each entry: price, features (beds, baths, parking, property type, land size), address (street, suburb, state, postcode, lat, lng), branding (agency and agents), tags.tagText (for sold homes, the method and date: "Sold at auction 10 Nov 2026"), auction and inspection times.
  4. totalListings and totalPages for pagination.

Search URLs follow one pattern: /sale/<suburb>-<state>-<postcode>/, /rent/... or /sold-listings/..., plus ?page=N.

Recent sales to CSV

Sold listings are the most useful part of Domain: each one says how and when it sold.

import csv

PATHS = {"sale": "sale", "rent": "rent", "sold": "sold-listings"}


def search_page(slug, mode="sold", page=1):
    data = next_data(fetch(f"https://www.domain.com.au/{PATHS[mode]}/{slug}/?page={page}"))
    rows = []
    for lid in data["listingSearchResultIds"]:
        item = data["listingsMap"][str(lid)]
        if item["listingType"] != "listing":
            continue  # new-development projects share the map
        m = item["listingModel"]
        f, a = m["features"], m["address"]
        rows.append({
            "url": "https://www.domain.com.au" + m["url"],
            "price": m["price"],
            "status": (m.get("tags") or {}).get("tagText") or "",
            "type": f.get("propertyTypeFormatted"),
            "beds": f.get("beds"), "baths": f.get("baths"), "parking": f.get("parking"),
            "address": f'{a["street"]}, {a["suburb"]} {a["state"]} {a["postcode"]}',
            "lat": a.get("lat"), "lng": a.get("lng"),
            "agency": m["branding"].get("brandName"),
        })
    return rows, data["totalPages"]



rows, page, pages = [], 1, 1
while page <= min(pages, 3):
    batch, pages = search_page("bondi-beach-nsw-2026", "sold", page)
    rows += batch
    page += 1

with open("bondi_sold.csv", "w", newline="", encoding="utf-8") as fh:
    writer = csv.DictWriter(fh, fieldnames=rows[0].keys())
    writer.writeheader()
    writer.writerows(rows)
print(len(rows), "sold listings")
Enter fullscreen mode Exit fullscreen mode

In a live run on Bondi Beach this returned 60 sold homes from three pages (30 credits): 25 sold by private treaty, 21 prior to auction and 14 at auction. Only 28 of the 60 had a price; the rest say "Price Withheld", which is common in Sydney. So keep the price as text, and parse a number only when there is one.

Listing pages and suburb medians

A listing URL is the address slug plus the listing ID. The same __NEXT_DATA__ object holds the full description, structured features, the price guide with agents, listing dates and suburbInsights:

def listing(url):
    p = next_data(fetch(url))
    s = p["listingSummary"]
    insights = p.get("suburbInsights") or {}
    return {
        "id": p["listingId"],
        "price": s.get("title"),
        "status": s.get("status"),
        "method": s.get("method"),
        "beds": s.get("beds"), "baths": s.get("baths"), "parking": s.get("parking"),
        "headline": p.get("headline"),
        "description": "\n".join(p.get("description") or []),
        "features": p.get("structuredFeatures"),
        "agents": [a.get("name") for a in (p.get("priceGuide") or {}).get("agents", [])],
        "listed": p.get("createdOn"),
        "suburb_median": insights.get("medianPrice"),
        "suburb_median_rent": insights.get("medianRentPrice"),
        "days_on_market": insights.get("avgDaysOnMarket"),
        "auction_clearance": insights.get("auctionClearance"),
    }
Enter fullscreen mode Exit fullscreen mode

suburbInsights is the shortcut most people miss: the median price, median rent, average days on market and auction clearance rate for the suburb and property type, without computing anything yourself.

Limits to know

  • 20 listings per page, 50 pages per search. Bondi Beach has 16,626 sold listings, but Domain stops at page 50. For big areas, run neighbouring suburbs instead of more pages.
  • Projects share the map. On page one of Sydney NSW 2000 for sale, 4 of the 20 entries were new-development projects, not homes.
  • Prices are text. "Price Withheld", "Contact Agent", "Auction Guide $1,200,000": store the text and parse it separately.

Skip the code

The same scraper is an open-source package with a CLI:

pip install domain-com-au-scraper
export SCRAPEDO_TOKEN=<your_token>
domain-scraper "Bondi Beach NSW 2026" --mode sold --pages 3 -f csv -o bondi_sold.csv
domain-scraper "Sydney NSW 2000" --details --limit 20 -o sydney.json
Enter fullscreen mode Exit fullscreen mode

Source and tests: github.com/bugrahansss/domain-com-au-scraper

Use the data responsibly

Listings are public, but agents are real people: use their names and numbers only for legitimate purposes, follow the Australian Privacy Act and Spam Act, and respect Domain's terms. Don't republish listing photos or descriptions.

Run it at scale with Scrape.do

Every page is one Scrape.do call with super=true: no proxy pool, no Akamai handling, no headless browser. 1,000 sold listings from search pages cost about 500 credits. Get 1000 free credits and start scraping with Scrape.do.

Top comments (0)