Mercari sold-comps scraper: "sold" isn't a field, "200 OK" isn't a guarantee, and "403" isn't always a block.
Quick answer
Mercari has five item statuses — on_sale, trading, sold_out, stop, cancel — and none of them is called sold. A "sold" comp is actually status in {trading, sold_out}, and trading means the item is mid-escrow, not that it's still buyable, which is the opposite of what the name suggests. Layer onto that a Cloudflare managed challenge that can arrive with a plain 200 OK status, and a 403 that's sometimes a legitimate "this listing was purged" signal rather than a block — and a naive client either serves stale live listings as "sold" comps, or throws away real data because it treated every non-200 the same way.
Why isn't there a sold: true field on the item?
Mercari's category pages embed a server-rendered GraphQL cache in __NEXT_DATA__ — the same props.pageProps.serverState idiom as other Next.js storefronts — but the item entity's own status field is a five-way enum, not a boolean:
SOLD_STATUSES = frozenset({"trading", "sold_out"})
def status_to_sold(status: str) -> bool:
"""`sold` is derived: status is trading or sold_out."""
return status in SOLD_STATUSES
trading is the interesting one to get wrong. It means a buyer has committed and the sale is in escrow — not "still on sale," but also not a closed, final sale price the way sold_out is. A reseller pricing tool that only counts sold_out will under-count recent comps; one that treats trading as "still buyable" will double-list an item that's actually gone. We fold both into sold=True, matching what Mercari's own UI calls "Sold."
Why does itemsList break a parser copied from a similar site?
The search entry under ROOT_QUERY looks like a typical Apollo GraphQL connection at first glance, but it isn't one:
# Critical difference: the search entry's itemsList is a flat list of
# refs, not an edges/node connection — callers must never add a
# .get("node") hop here.
def resolve_search_entry(server_state, *, keyword):
...
return entry # {"count": int, "itemsList": [ref, ref, ...]}
itemsList is a bare list of {"__ref": ...} pointers — no edges, no node wrapper. That's a real trap because it's exactly the shape a different site's Apollo-cache parser (we maintain more than one) would have — copy that pattern verbatim here and every item resolves to an empty dict.
Why does a 200 OK page sometimes contain zero items?
Cloudflare's bot-management challenge doesn't always answer with 403 or 503. It can return the interstitial page with a plain 200 status, so checking status_code == 200 before inspecting the body is backwards:
# Challenge is checked before the 200 fast-path: Cloudflare's managed
# challenge can itself arrive with status 200 (interstitial HTML), not
# just 403/503 — the body-marker check must run regardless of status.
def _is_challenge(status, headers, body):
return headers.get("cf-mitigated") == "challenge" or "Just a moment" in body
We check the cf-mitigated header and the "Just a moment" body marker on every response, 200 included, before ever trusting the page. A block that hides behind a 200 is worse than one that announces itself with a 403 — it looks like a successful, empty search instead of a failure.
Why does a 403 on an item page sometimes mean nothing is wrong?
Item detail pages can legitimately 403 when Mercari has purged a listing's history — a real, expected outcome, not a block:
ITEM_PURGED_STATUS = 403
ITEM_PURGED_ERROR_CODE = "BundleNotAvailableException"
def _is_item_purged(status, body):
if status != ITEM_PURGED_STATUS:
return False
payload = json.loads(body)
return payload.get("gqlErrorCode") == ITEM_PURGED_ERROR_CODE
That check runs before the challenge check and before any retry logic, because retrying a purged item forever would just burn attempts on something that will never succeed. The base listing row still gets emitted — only the optional per-item enrichment (exact listing/sold date, seller rating) is skipped, with a warning logged, never a failed run.
This is also why the retry-status set here is deliberately narrower than on other Actors we've built against similar Next.js targets: 408 / 429 / 503 retry, but 403 and 500/502/504 don't — a 403 either means "purged" (handled above) or a real block that a plain retry won't fix.
What does sold_date actually measure?
It's the item detail page's updated timestamp — a Unix epoch integer we convert to ISO 8601 — used as a proxy for "when this sold." It's a last-status-change timestamp, not a guaranteed instant of sale, and the code says so explicitly rather than overselling it:
sold_date=patch.sold_date if (patch and sold) else None,
We only attach it when the row's own derived sold is True, even if the enrichment fetch succeeded — an enrichment patch never overrides the base row's own sold state.
FAQ
Does this scrape active listings, sold comps, or both?
Both — set soldOnly for Mercari's own sold/completed facet, optionally narrowed with soldWithinDays, or leave it off for everything currently listed.
Do prices come back in cents or dollars?
Mercari's wire format is USD cents; we convert to dollars (5900 → 59.0) before it reaches your dataset.
Can I filter by condition or price band?
Yes, but as a post-filter applied to already-fetched rows — Mercari's search endpoint has no confirmed server-side parameter for either, so filtering client-side is the honest approach rather than guessing at a param that might silently be ignored.
What happens if one department (category) gets blocked mid-run?
That department is skipped with a logged warning; the other 16 keep scraping. A single Cloudflare wall doesn't zero out a multi-category run.
Packaged and ready to run: Mercari US Sold Listings Scraper — keyword search across all 17 Mercari departments with a first-class sold/completed filter, price, condition, brand, seller and photos per row, plus optional per-item listing/sold-date and seller-rating enrichment.
We do the dirty work so your dataset stays clean. 😈
Top comments (0)