DEV Community

Devil Scrapes
Devil Scrapes

Posted on

We queried Wallapop for zzznonsensequery9999xyz. Zero results is what proved the search actually works.

Quick answer

Before we trust any target's search parameter, we run one cheap test: query it with a real keyword, then query it again with a string that cannot possibly match anything, and check that the second query comes back near-empty. For the Wallapop Scraper, keywords=bicicleta against api.wallapop.com/api/v3/search returned 40 items, first hit "Bicicleta infantil B'Twin Roja." keywords=zzznonsensequery9999xyz against the same endpoint, same coordinates, same session, returned items: []. That's the whole test, and it's the reason we shipped keywords as the Actor's real filter instead of a decorative one.

Why does a filter need proving before you ship it?

Because we'd already been burned by one that looked fine and wasn't. A sibling classifieds Actor built earlier the same week — Avito, Russia's largest classifieds site — shipped on a key=<query> parameter that read like a normal search filter in recon: real 200s, real JSON, real listings back. It wasn't filtering anything. Swap the query string for gibberish and Avito's endpoint soft-degrades to its own unfiltered default feed, handing back a full page of plausible, real, completely irrelevant listings. A scraper built on that assumption doesn't fail loudly — it just quietly returns the wrong 40 rows for every query a customer ever runs, and every one of those rows looks legitimate enough that nobody notices until a customer does the diff themselves.

That failure mode got a name in this codebase — the Avito rule — and a standing checklist item: before any new classifieds or marketplace Actor ships, prove the query parameter actually constrains the result set. Not "it returned data," but "a query with zero possible matches returned zero (or near-zero) results, on the live endpoint, today."

How do you actually run that check?

With the cheapest possible instrument: a direct curl_cffi request against the search endpoint, no Actor invocation involved, so it costs nothing and touches no billing. We hit Wallapop's /api/v3/search twice in the same session — once with a keyword we know has matches, once with a string engineered to have none — and diff the item counts. For Wallapop this passed clean: 40 items for a real term, zero for the nonsense one, meaning the endpoint's own relevance filtering is doing real work server-side rather than window-dressing an unfiltered feed.

That evidence didn't stay in a chat transcript or a spec note where it could quietly go stale — it's committed as an opt-in pytest.mark.smoke test in the Actor's own test suite (test_nonsense_query_returns_far_fewer_items_than_real_query), excluded from the default run (addopts = "-m 'not smoke'" so CI doesn't hit the live target every run) but there, checkable, and re-runnable by anyone who doubts it later. A claim that only lives in a commit message can't be re-verified in thirty seconds; a claim that lives in a test file can.

A 200 status code and a plausible-looking JSON array tell you the endpoint answered — they tell you nothing about whether your query parameter changed the answer. The only way to know is to send a query that should return nothing and check that it does.

What the Actor gives you

One row per Wallapop listing matching your keyword search: title, description, price and currency, category, listing URL, city and region, image URLs, creation and modification timestamps, reservation status, and the seller's user ID. Multiple keyword queries run in the same call, deduplicated by listing ID across pages and across queries, and Wallapop's own cursor-based pagination is followed verbatim so listings never repeat or silently drop mid-walk. We retry 429/503/network hiccups with backoff so a single flaky response doesn't kill a run, and every row lands Pydantic-validated with ISO-8601 timestamps.

Honest limitations 🚧

This scrapes search results only — no per-listing detail fetches beyond what the search response itself carries, no seller messaging, no login-gated data. Coverage is whatever Wallapop's own search endpoint surfaces for a keyword; there's no separate category-browse mode.

FAQ

Does the keywords field actually filter results, or is it decorative?
It's real. We verify this against the live endpoint with a nonsense-query smoke test before trusting any target's search parameter — it's committed in the Actor's own test suite, not just claimed in the README.

Can I run multiple keyword searches in one call?
Yes — pass a list, and results are deduplicated by listing ID across all of them.

Do I need a Wallapop account?
No. This uses the same public search endpoint the Wallapop web app calls; no login, no seller-side access.

What happens if a query matches nothing?
The run succeeds with zero rows for that query and a status message saying what was searched — a genuine empty match is not treated as a failure.

Pricing

$0.20 per run plus $0.004 per unique listing scraped — $4.20 per 1,000 listings. A run that matches nothing costs only the start fee.

Wallapop Scraper on Apify


Built by Devil Scrapes. We rotate fingerprints, retry with backoff, and — as this one shows — we don't ship a search filter until we've proven it filters.

Top comments (0)