The silent empty success: how a nested JSON path passed 83 green tests

Quick answer

A Houzz professional listing page embeds its data as a <script id="hz-ctx"> JSON blob — but that blob nests each Flux store under data.stores.data.<StoreName>, not at the top level, and our own first build assumed the flat shape. Get the path wrong and the page still fetches, still parses, and the run still exits 0 — it just merges zero professionals. The Houzz Professionals Scraper reads the real nested path, fails loud instead of silently on an empty merge, and returns 13 typed fields per lead — name, city/state, phone, rating, license number, project count, years-in-business — for $0.02 per run plus $0.002 per result, $2.02 for 1,000 leads.

The bug that shipped a green run with zero rows 🕳️

Here's the failure mode that should scare anyone who's shipped a scraper: our first build of this Actor passed 83 tests, ran clean, exited 0, and logged "Done — 0/5 leads scraped" in 11 seconds. No stack trace, no red text — a perfectly healthy-looking run that did nothing.

The root cause was one wrong dictionary path. Houzz's listing and profile pages embed a large Flux-style store dump — UserStore, ProfessionalStore, UserStatsStore — inside <script id="hz-ctx" type="application/json">. Our parser was built and tested against static fixture captures that had that dump sitting at the top level: hz_ctx.UserStore. Reasonable, well-tested, wrong — the live site nests it one level deeper, under hz_ctx.data.stores.data.UserStore. The fixtures were frozen captures, so they were the wrong shape consistently across all 83 tests; no amount of local test-passing could have caught it.

That's worse than a crash. A crash bills nothing and tells you immediately. A silent empty success bills the run-start fee, reports victory, and leaves nothing to debug. We only caught it because a live cloud-run diagnostic capture against a real Houzz IP disagreed with every fixture we had.

Two fixes shipped together. parser._locate_store() now checks the real nested path first, falling back to the flat shape so old fixtures still pass. More importantly, the scraper now tells "nothing matched" apart from "parsing silently broke": an empty-merge listing page logs an ERROR with the exact UserStore/ProfessionalStore record counts and captures the raw hz_ctx, and a run that nets zero rows because of that condition now exits non-zero. A legitimate zero-match run — your location filter was too narrow, say — still succeeds with an honest shortfall message; a parser regression doesn't get to hide behind one.

License number and years-in-business are opt-in, not universal ⚖️

Two of the fields that make this Actor worth running — license_number and years_in_business — come from data Houzz treats as optional self-reporting, and we don't pretend otherwise. license_number lives on the profile page, not the listing page, so it's null on every row unless fetchProfileDetails: true — and even then it's present on roughly half of profiles, because roughly half of Houzz pros haven't added one. We never fabricate a value to make the fill rate look better.

years_in_business is stricter still: parsed only from an explicit "N years in business" merit badge, matched against a fixed regex. We don't infer it from review history, project count, or account age — no badge, no value. Treat both fields as high-signal-when-present, not universally populated columns.

Location filtering matches, it doesn't radius-search 📍

The location input is "City, ST" free text, and it's worth being precise about what it does: it's a post-scrape match against nationwide category results, not a native Houzz geo-radius query. Houzz's own radius search needs an internal region ID we can't resolve from a plain string, so instead the Actor pulls the category's national result set and filters by city/state after the fact. Invisible for a well-populated metro category; for a niche category in a small city, raise maxPages to walk deeper into the national list before enough in-region matches surface.

What we handle for you 🛡️

• We rotate browser fingerprints — curl-cffi impersonation across Chrome, Firefox, and Safari TLS profiles.
• We retry with exponential backoff on 408 / 429 / 5xx, up to 5 attempts per page, honoring Retry-After.
• We rotate residential proxies through Apify Proxy when enabled — a fresh session and exit IP on every block.
• We fail loud, not silent. An empty-merge listing page raises an ERROR with store record counts and a diagnostic dump; a run that nets zero rows because of it exits non-zero, never a quiet green checkmark over nothing.
• We keep the dataset clean — Pydantic-validated rows, ISO-8601 timestamps, a stable houzz_professional_id dedup key.
• You pay only for results that land. No data, no charge, beyond the small run-start fee.

Full output schema 📦

Thirteen fields per professional:

Field	Type	Notes
houzz_professional_id	int	Stable Houzz professional ID — the dedup/identity key
name	string	Business or pro display name
category	string	Houzz category display name, e.g. "General Contractors"
city	string | null	City
region	string | null	State
phone	string | null	Formatted phone number, verbatim
website	string \	null
review_count	int	Review count; never null
average_rating	float \	null
project_or_photo_count	int \	null
years_in_business	int | null	Parsed only from an explicit merit badge; never inferred
license_number	string \	null
houzz_profile_url	string	Canonical Houzz profile URL
scraped_at	string	ISO-8601 timestamp for when the row was recorded

Who this is for

Franchise and vendor prospecting — build a call list of licensed contractors, architects, or designers by category and city.

Selling into the trades — materials, software, and insurance vendors targeting pros by verified review count and license presence.

SDR/sales list-building — filter by rating, review count, and license status before outreach, without hand-copying rows off Houzz.

Market research — track license coverage, years-in-business, and portfolio size across a trade category.

Frequently asked questions

Why did an early run report success with zero results?

An early build read Houzz's embedded hz-ctx JSON at the wrong nesting depth — hz_ctx.UserStore instead of the real hz_ctx.data.stores.data.UserStore — so pages fetched and parsed without error but merged zero professionals: a green run, 11 seconds, no data. Fixed: the parser reads the correct nested path, and a run that nets zero rows because of an empty-merge condition now fails loud instead of exiting 0.

Why is license_number often null?

Houzz only exposes it on profile pages (needs fetchProfileDetails: true), and only about half of pros have added one. We never fabricate a value to raise the fill rate.

Why is years_in_business usually null?

It's a self-reported, opt-in Houzz badge, parsed only when Houzz explicitly shows the "N years in business" text — never estimated from review or project count.

Does the location input do an exact radius search?

No. It's a post-scrape city/state match against nationwide results, not Houzz's native geo-radius search. Raise maxPages for niche categories in smaller cities.

What does 1,000 leads cost?

$2.02 — 1,000 × $0.002 per result, plus the $0.02 run-start charge.

Try it

Live on the Apify Store: Houzz Professionals Scraper.

Point it at a Houzz category — general contractor, architect, landscaper, or any of the platform's other trades — and get back typed, deduplicated leads with the fields other Houzz scrapers leave unparsed in the embedded JSON. Pay-per-event, no subscription, no card required to try.

---

Built by Devil Scrapes — we build scrapers for the targets that fight back.