Type your uncertainty: the nullable field that saves a dataset

Here's a small thing that quietly ruins restaurant datasets: collapsing a rating into one number.

A restaurant on OpenTable isn't rated 4.3. It's rated 4.3 overall, with separate scores for food, service, and ambience underneath. Those sub-scores are the interesting part — a 4.6 food / 3.4 service split is a completely different business from a flat 4.0, and the two are indistinguishable once you've flattened them.

So the OpenTable Restaurants & Reviews Scraper keeps them apart:

rating_overall:  float | None   # 1-5 aggregate
rating_food:     float | None
rating_service:  float | None
rating_ambience: float | None
review_count:    int | None

Four columns instead of one. Costs nothing to carry, impossible to reconstruct later.

🏷️ The convention we actually want to talk about

Look at those type annotations again. Every one of them is | None.

That's not defensive padding. It's a rule we hold ourselves to: a field we haven't confirmed on live pages ships nullable and documented as unconfirmed — never asserted as always-present, and never quietly dropped from the schema because it was inconvenient.

In the source, those fields are literally annotated that way:

rating_food: float | None = Field(
    default=None, description="Food sub-score (UNCONFIRMED)."
)

The word UNCONFIRMED is doing real work there. It's the difference between two failure modes:

• Field asserted, sometimes missing → your pipeline throws on row 4,000 of a 20,000-row export, or worse, a KeyError swallowed by a try somewhere becomes a silent zero. Zero and "not published" are very different facts about a restaurant.
• Field nullable and labelled → the field is null, your COUNT(rating_food) tells you the real coverage, and you can decide whether it's good enough for what you're doing.

The second one is honest. It says we saw this on some pages and not others, and we're not going to pretend otherwise. Every scraper of a site you don't control is a set of hypotheses about someone else's HTML. Encoding which hypotheses are confirmed and which aren't, right there in the type, is the cheapest documentation you will ever write.

The same discipline applies to opentable_id and neighborhood — both real, both inconsistently exposed, both nullable and marked. Nobody downstream has to guess.

🧱 The other reason coverage varies: getting the page at all

OpenTable runs a site-wide bot-defense wall, and it's noticeably harsher on datacenter IP ranges. That shapes the actor's defaults more than any parsing decision:

• TLS fingerprint impersonation via curl-cffi, so the handshake looks like real Chrome/Firefox/Safari rather than Python's requests. Getting the JA3 fingerprint wrong is enough on its own.
• Residential proxy rotation with a fresh session and exit IP on every block.
• Exponential backoff on 408/429/5xx, up to five attempts, Retry-After honoured. When a target pushes back, slowing down beats getting banned.

None of that is glamorous, and all of it is why coverage on any given field is a measured number rather than a promise.

📊 What you get per row

One row per restaurant: name, canonical URL, city, neighborhood, price tier, the four rating fields above, review count, a booking-popularity signal, phone, address, and — when includeReviews is on — a capped set of recent review snippets, each with its own author, date, and per-review rating.

That shape suits the three things people actually build with it: local-restaurant SaaS keeping a metro dataset current, franchise and competitive-intel work comparing cuisine mix and price tiers across neighborhoods, and site-selection analysis mapping restaurant density before signing a lease.

🧭 The takeaway

Two rules, both cheap, both hard to retrofit:

Don't flatten a composite metric into a scalar. If the source publishes four numbers, store four numbers. Aggregation is a query; disaggregation is a re-scrape.

Type your uncertainty. float | None with an UNCONFIRMED note is more useful to a buyer than a confident float that turns out to be a lie on 30% of rows. Clean-looking output that's wrong is worse than honest output with nulls in it, because nobody thinks to check it.

Live on the Apify Store, pay-per-result, no credit card to try it.

We do the dirty work so your dataset stays clean. 😈