DEV Community

datalume
datalume

Posted on Edited on

I scraped Google Hotels for a competitor rate calendar — and found the trap that breaks most of them

#python #webscraping #datascience #tutorial

I wanted a simple thing: the nightly price of 5 competing hotels for every date of the next few weeks, plus which booking site sells each one cheapest. The kind of table a hotel revenue manager checks every morning.

Google Hotels already shows all of this — it just doesn't let you export it. So I scraped it. Here's what I learned, including one trap that makes most "rate calendars" useless.

The trap: Google re-ranks hotels for every date

My first attempt was the obvious one: search "hotels in Shinjuku, Tokyo" for 7 consecutive check-in dates and keep the results.

Then I checked how stable that list is. With the top 50 hotels per date over 7 dates, I got 148 different hotels — and only 3 of them appeared on all 7 dates. The same hotel can be #2 on one date, #42 on another, and missing on a third.

That's fine for browsing, but for rate shopping it's fatal: your "calendar" ends up full of holes exactly where you need the comparison.

The fix is to stop asking "which hotels are in Shinjuku?" and ask "what does this exact hotel cost on each date?" — lock each competitor to its Google Hotels ID once, then look up that same ID for every check-in date.

The setup (Python, ~15 lines)

I packaged this as an Apify Actor (Google Hotels Scraper) so I don't have to babysit proxies or parsing. You give it your competitive set and a date range:

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run_input = {
    "hotels": [
        "Keio Plaza Hotel Tokyo",
        "Hotel Gracery Shinjuku",
        "Shinjuku Granbell Hotel",
        "Mitsui Garden Hotel Jingugaien Tokyo PREMIER",
        "MIMARU TOKYO SHINJUKU WEST",
    ],
    "location": "Tokyo",        # helps pick the right hotel for each name
    "checkIn": "2026-11-12",
    "dateRangeDays": 7,         # up to 60
    "adults": 2,
    "currency": "USD",
    "includeOffers": True,      # every booking site's price, not just Google's headline price
}

run = client.actor("datalume/google-hotels-scraper").call(run_input=run_input)
df = pd.DataFrame(client.dataset(run.default_dataset_id).iterate_items())
Enter fullscreen mode Exit fullscreen mode

The run took 41 seconds and returned 35 rows (5 hotels × 7 dates) — no gaps. Each row has the hotel, the date, and an offers list with 19–38 booking sites (Booking.com, Expedia, Agoda, Trip.com, the official site…), so about 1,000 site prices in total.

1. The rate calendar 

calendar = df.pivot_table(
    index="name", columns="checkIn", values="lowestOfferPricePerNight"
).round(0)
Enter fullscreen mode Exit fullscreen mode

Cheapest price per night (USD, 2 adults, names shortened):

Hotel Thu 11-12 Fri 11-13 Sat 11-14 Sun 11-15 Mon 11-16 Tue 11-17 Wed 11-18
Hotel Gracery Shinjuku 176 226 228 191 191 153 225
Keio Plaza 269 259 270 246 245 257 251
MIMARU Shinjuku West 421 420 682 453 333 336 333
Mitsui Garden Jingugaien 223 249 354 242 235 232 229
Shinjuku Granbell 147 145 228 117 118 133 125

The Saturday spike jumps out immediately: MIMARU +62% vs. Friday, Granbell +57%, Mitsui Garden +42% — while Keio Plaza barely moves. If you were pricing a hotel in this set, that's exactly the signal you'd want before the weekend fills up.

2. Which booking site is cheapest? 

df["lowestOfferProvider"].value_counts()
Enter fullscreen mode Exit fullscreen mode
Cheapest site Hotel-nights
klook 18
Vio.com 3
KAYAK.com 2
HyperAir.com 2
trivago DEALS 2

Not Booking.com, not Expedia: a single OTA had the lowest price for 18 of 35 hotel-nights. In 4 cases the hotel's own website was the cheapest.

3. Official site vs. the cheapest OTA (rate parity)

Each offer has an isOfficialSite flag, so comparing the direct price with the market takes two lines:

df["officialSite"] = df["offers"].apply(
    lambda offs: next((o["pricePerNight"] for o in offs if o["isOfficialSite"]), None))
df["officialVsCheapestPct"] = ((df["officialSite"] / df["lowestOfferPricePerNight"] - 1) * 100).round(0)
Enter fullscreen mode Exit fullscreen mode

For Hotel Gracery Shinjuku:

Date Cheapest site Cheapest Official site Official vs. cheapest
11-12 klook $176 $225 +27%
11-13 Vio.com $226 $292 +29%
11-14 klook $228 $281 +23%
11-15 klook $191 $215 +13%
11-16 Trip.com $191 $215 +13%
11-17 klook $153 $215 +41%
11-18 official site $225 $225 0%

On 6 of 7 nights someone undercut the direct rate, by up to 41%. (The official price was usually within a few dollars of Booking.com — the undercutting came from smaller OTAs.)

One honest caveat: sites don't always quote the same room type. In my data one hotel's official site showed $1,045 on a Saturday while the cheapest OTA was $228 — not a parity violation, just a much bigger room. For strict like-for-like checks, turn on includeRooms and compare the same room name.

Cost

The Actor charges per row: $0.004 per hotel-date with booking-site prices ($0.0015 without). This run was 35 rows → $0.14. A month for 5 competitors (150 rows) is about $0.60, and you can schedule it to run every morning in Apify. Hotel-dates that Google doesn't show (e.g. sold out) come back as priceStatus: "notListed" and aren't charged.

Takeaways

  1. Don't build a competitor calendar from a city search — Google re-ranks per date and you'll get holes. Pin the hotels.
  2. "Cheapest site" is surprisingly concentrated; it's worth knowing which OTA that is in your market.
  3. Rate parity checks need a room-type sanity check before you send anyone an angry email.

Disclosure: I run datalume, the publisher of the Google Hotels Scraper Actor used here. Both the Actor and this article were built with AI assistance; every number comes from a real run on Oct 29, 2026 (Google Hotels, US) and was checked against the raw data. Questions welcome in the comments.

Top comments (0)