A hostel bed has four prices. Three of them are hidden.

If you scrape one accommodation price, you have a number. It is nearly useless.

€48 tells you nothing on its own. What a revenue manager, an OTA analyst or a travel-market researcher actually acts on is the shape of that number over time:

it sat at €48 for three weeks, then moved twice in one day
it dropped to €41 exactly nine days before arrival
the property stopped being listed for that date entirely

None of those facts exist in any single response. No rate site publishes "here is what I charged yesterday". So if you do not store the previous state, the interesting part is not merely missing — it is unknowable.

That reframing is the whole engineering problem, and it has a pleasant consequence: once your scraper keeps state, it can return changes instead of rows, and a daily run over a whole city costs almost nothing on a quiet day.

The grid, not the row

The unit of state cannot be "a property". It has to be a cell: (property × arrival date × guest configuration). A hostel bed is priced per arrival date and per party size, and those are independent axes.

key = f"{property_id}|{arrival_date}|{guests}|{nights}"
prev = state.get(key)
now  = {"lowest": lowest, "dorm": dorm, "private": private, "promo": promo_stack}

if prev is None:
    change = "new"
elif prev["lowest"] != now["lowest"]:
    change = "price_up" if now["lowest"] > prev["lowest"] else "price_down"
elif prev["promo"] != now["promo"]:
    change = "promo_changed"
else:
    change = None          # unchanged: do not emit, do not bill

One city over 30 forward dates is 30 requests and a full month of the priced competitive set — because a whole city arrives in one request per date. London returns 83 properties at once; Munich 17. That is what makes daily monitoring viable rather than a crawl.

The trap that cost me a whole baseline

I shipped this same pattern on a reviews scraper first, and the second run reported every item as new. The fingerprints were fine. The ordering was not.

The baseline run had paged through results in the site's default relevance order. The next run happened to load them in date order. Two runs, two disjoint slices of the same set — so nothing matched and the diff was garbage.

The fix is one line, and it belongs in every delta scraper you write:

# Delta mode must pin the sort order ALWAYS — not only when a baseline exists.
# Relevance ordering is not stable between runs.
if await sort_deterministically(page):
    stop_after_all_seen_pages = 1

Pin the order and "I hit a full page of already-seen items" becomes a trustworthy stop signal instead of a coin flip.

Four prices, not one

The headline rate is the least interesting field on the page. On this surface each cell carries:

the lowest available rate — what a price-comparison table would show
the dorm / private split — two different products under one property, and they move independently
the pre-discount rate plus the promo stack, with discount percentages. This is the one competitors miss: a property holding rack rate while stacking a 25% promo is running a very different strategy from one that simply cut its rate, and only the stack tells them apart.
the commission split for that date — the economics of the listing, not just its price

Plus the organic ranking position, which answers a question the rate alone cannot: did they get cheaper, or did they get promoted?

Absence is a signal

The awkward case is a property that vanishes from a date it was priced on yesterday. Sold out, pulled, or de-listed — the response simply does not contain it. There is no row to diff.

gone = [k for k in previous_keys if k not in current_keys]   # only knowable by comparison

Two details that matter more than they look:

Rank the lifecycle change first. When a cell both moved price and disappeared, a buyer's alerting cares far more about "it is gone" than "it is €3 cheaper".
A first run has no baseline, so everything looks new. Billing a change rate for what is really a snapshot is not a pricing choice, it is a bug. Bill the baseline at snapshot rate, or not at all — and persist the watermark before charging, or a crash makes you re-bill the same changes next run.

Running it

I packaged this as Hostelworld Rate & Availability Monitor: forward-dated rate calendar, dorm/private split, promo stack with discount percentages, organic ranking position and the per-date commission split — HTTP-only, no login, no API key, no browser. Delta mode returns only the cells that moved.

There is a scheduled config ready to clone here: London hostel rate changes — daily delta.

dateStepDays: 7 is the cheap way to watch a long season — same weekday each week instead of every date.

If you would rather build your own, the five patterns behind the delta engine are written up in Scraping a price is easy. Knowing it changed is the product.

Watching a rate surface I have not covered? Reply with the source and I will tell you whether it is reachable anonymously and what the delta key should be.

DEV Community