Craigslist's JSON-LD has no ID field — we join 290 of 325 listings by title alone

Quick answer

Craigslist search pages ship two copies of every listing: a static HTML list, and a JSON-LD <script> block with images, currency, and geo-coordinates. The obvious move is to join them by ID. Don't — Craigslist's JSON-LD carries no shared identifier at all, not a bare post ID, not a URL, not a SKU. The only field both copies reliably share is the listing's title, and titles repeat. We joined by title through a per-title FIFO queue and measured it recovering 290 of 325 listings (89%) end-to-end on a captured 298-item page. That number is the ceiling of what a title-only join can do on this page shape — plan your field completeness around it, don't assume 100%.

Why can't you just match the JSON-LD by ID? 🧩

When we built the Craigslist Multi-City Listings Scraper, the first design assumed what almost every JSON-LD block on almost every e-commerce-shaped site provides: a productID, a sku, a url, an @id — something that lines up a JSON entry with its DOM counterpart deterministically. Live inspection of a captured Craigslist search page found none of those. Each itemListElement entry has exactly name, image, offers, @type, and a position field that looks like it should solve the problem — until you check it against the static list past the first ~18 entries, where some static-list rows have no JSON-LD counterpart at all and the position numbering drifts out of alignment.

So the join key that's actually usable, live, is title — and titles aren't unique. The fix is a FIFO queue per title: walk the static <li> list in document order, and for each title pop the next unconsumed JSON-LD entry with a matching name.

# actors/craigslist-listings-scraper/src/search_parser.py
def _parse_ld_json(tree: HTMLParser) -> dict[str, deque[_LdEntry]]:
    by_title: dict[str, deque[_LdEntry]] = defaultdict(deque)
    for list_item in data.get("itemListElement", []):
        entry = _ld_entry_from_item(list_item)
        title = list_item.get("item", {}).get("name")
        if title:
            by_title[title].append(entry)
    return dict(by_title)

def _pop_ld_match(ld_by_title, title: str) -> _LdEntry | None:
    queue = ld_by_title.get(title)
    return queue.popleft() if queue else None

This handles the ordinary case of duplicate titles reasonably — two identical "iPhone 14 Pro 256GB Unlocked" postings get their JSON-LD entries handed out in the same document order they appear in the static list — but it is not a guarantee. When the queue for a title runs dry (more static-list rows carry that title than JSON-LD has entries for), those listings still land with a title, price, and location from the static HTML — they just come back with currency, latitude, longitude, and images as null/empty, rather than being dropped. Fault isolation happens at the field level here, not just the row level: a partial join beats no row.

What changed under the URL scheme? 🔗

The other live-wire finding: Craigslist has migrated off its old numeric-ID, per-subdomain URL scheme onto a unified /view/d/<slug>/<id> shape, and the <id> segment in that new scheme is not purely numeric anymore — fixture capture found opaque alphanumeric strings like 39wYA6wpdyRS7ynm63QsKf. Any post-ID regex written against the legacy numeric-only assumption silently drops every listing on the new scheme:

POST_ID_RE = re.compile(r"/view/d/[^/]+/([A-Za-z0-9]+)")

A spec or scraper written from documentation or memory rather than a live fixture capture is exactly the kind of place this assumption survives unchallenged — Craigslist's own help pages don't advertise the migration, you only find it by pulling a real page.

What do you deliberately never touch? 🚧

Craigslist's robots.txt disallows the reply flow (/reply/, /fb/, and a handful of related paths) outright, and this Actor checks every detail URL against that list before any network call — not as a courtesy, as a hard refusal baked into the client:

FORBIDDEN_PATH_SEGMENTS: tuple[str, ...] = ("/reply/", "/fb/", "/suggest", "/flag", "/mf", "/mailflag", "/eaf")

The public reply_url still ships as a data field — it's on the page, it's useful context — but it is captured exactly as rendered and never opened, and no email de-obfuscation happens on top of it. That's a scope boundary, not a missing feature.

The part that generalises 🧭

Structured data (JSON-LD, webDigitalData, embedded RSC payloads — pick your target) is usually more complete than the visible DOM, but "more complete" doesn't mean "keyed the way you need." When a target's structured block has no shared identifier with its rendered markup, resist the urge to assume position or array order will hold past the first page of results — verify it against a real, large fixture before you trust it as a join key, and design the fallback (partial row, not dropped row) for when the join comes up short.

What the Actor gives you

• Up to 20 explicit metro areas plus a curated top-25-metro preset, across all seven Craigslist top-level categories, in one run instead of one call per city.
• Title, price, currency, location, coordinates, images, posted/updated timestamps, structured attributes, description, and the public (never-followed) reply link per listing.
• Per-(area, category) combo fault isolation — one blocked or slow combo is skipped and logged, the rest of the run keeps going.
• A best-effort repost_fingerprint (normalized title + area + category) for tracking reposts and price drops.

Honest limitations 🚧

Each (area, category) combo returns up to 300 listings — Craigslist's observed single-page ceiling, no further pagination in v1. Total combos are capped at 150 per run. allTopUsMetros covers a curated 25-metro list, not the full ~700-area directory. One search query per run.

FAQ

Can I get a seller's phone number or email?
No. reply_url is surfaced exactly as Craigslist renders it — never opened, never de-obfuscated. robots.txt disallows that flow and we respect it.

Why one run instead of one run per city?
Every incumbent we benchmarked requires a separate run per metro. This Actor fans out across areas and categories in one call, into one normalized dataset.

What happens if a city or category is unreachable mid-run?
That (area, category) combo is skipped and logged; the rest of the run continues, and a status message reports what finished.

Do I need Residential proxy?
Not by default — standard Apify Proxy with session rotation and fingerprint impersonation clears this target.

Pricing

$0.20 per run plus $0.002 per result — $2.01 per 1,000 listings. A search that matches nothing costs only the start fee.

→ Craigslist Multi-City Listings Scraper on Apify

---

Built by Devil Scrapes. We handle the fingerprint rotation, the multi-city fan-out, and the JSON-to-DOM join Craigslist doesn't make easy, so you get a flat table instead of a weekend. 😈