Stop Scraping Bayut - Build a Dubai Property App in 48 Hours

#webdev #api #automation #data

Your Bayut scraper just broke again.

They updated their HTML. Your regex is wrong. The IP got blocked. You're debugging at midnight instead of shipping.

Sound familiar?

I've been there. I built a scraper for Dubai property data. It worked for two months. Then Bayut updated their site. I spent a week fixing it. Then it broke again.

That's when I found a better way.

The Problem with Scraping Bayut

Web scraping Bayut is painful for a few specific reasons:

Aggressive bot detection - Bayut uses Cloudflare and fingerprinting. Your scraper gets blocked fast.
Frequent HTML changes - The site structure changes regularly. Your selectors break.
Proxy costs - To scrape at scale, you need rotating proxies. That's $200-500/month minimum.
Legal grey area - Scraping violates Bayut's terms of service. It's a real risk.
Maintenance hell - Every site update means debugging time. Time you could spend building.

The Solution: Bayut API

The Bayut API on RapidAPI gives you programmatic access to Bayut's data without any of those problems.

No proxies needed
No scraping
Clean JSON responses
UAE-wide coverage
Fast (sub-400ms)

Here's what you can access:

Property listings (sale + rent)
Off-plan development projects
Agent profiles and listings
Agency data
Transaction history
Location search and IDs

Building the App: Step by Step

Let's build a Dubai property search app. I'll use Node.js and Express for the backend.

Step 1: Get Your API Key

https://rapidapi.com/happyendpoint/api/bayut14/

Step 2: Location Search

First, implement location autocomplete so users can search by area.

const axios = require('axios');

async function searchLocations(query) {
  const response = await axios.get(
    'https://bayut14.p.rapidapi.com/autocomplete',
    {
      params: { query, langs: 'en' },
      headers: {
        'x-rapidapi-host': 'bayut14.p.rapidapi.com',
        'x-rapidapi-key': process.env.RAPIDAPI_KEY
      }
    }
  );
  return response.data.data.locations;
}

// Returns: [{ id, externalID, name, slug, adCount }]
// Use externalID for property search

Step 3: Property Search

Now the core search functionality:

async function searchProperties(params) {
  const response = await axios.get(
    'https://bayut14.p.rapidapi.com/search-property',
    {
      params: {
        purpose: params.purpose || 'for-sale',
        location_ids: params.locationId,
        property_type: params.type || 'apartments',
        rooms: params.bedrooms,
        price_min: params.minPrice,
        price_max: params.maxPrice,
        sort_order: 'popular',
        page: params.page || 1,
        langs: 'en'
      },
      headers: {
        'x-rapidapi-host': 'bayut14.p.rapidapi.com',
        'x-rapidapi-key': process.env.RAPIDAPI_KEY
      }
    }
  );

  const { properties, total, totalPages } = response.data.data;
  return { properties, total, totalPages };
}

Each property in the response includes:

Title, price, bedrooms, bathrooms
Area in sqm
Location details
Cover photo
Agent info
Amenities
Completion status (ready vs off-plan)

Step 4: Property Details

When a user clicks a listing, fetch full details:

async function getPropertyDetails(externalId) {
  const response = await axios.get(
    'https://bayut14.p.rapidapi.com/property-details',
    {
      params: { external_id: externalId, langs: 'en' },
      headers: {
        'x-rapidapi-host': 'bayut14.p.rapidapi.com',
        'x-rapidapi-key': process.env.RAPIDAPI_KEY
      }
    }
  );
  return response.data.data;
}

Step 5: Off-Plan Projects

Add a section for new developments:

async function searchOffPlan(locationId) {
  const response = await axios.get(
    'https://bayut14.p.rapidapi.com/search-new-projects',
    {
      params: {
        location_ids: locationId,
        property_type: 'residential',
        sort_order: 'latest',
        page: 1
      },
      headers: {
        'x-rapidapi-host': 'bayut14.p.rapidapi.com',
        'x-rapidapi-key': process.env.RAPIDAPI_KEY
      }
    }
  );
  return response.data.data.properties;
}