DEV Community

Devil Scrapes
Devil Scrapes

Posted on

A CONNECT 407 looked like a proxy-entitlement problem. It was a country pin with no group name.

Quick answer

Cloud QA on the Zalando Fashion Listings Scraper failed with CONNECT 407 and a message reading "requested exit country not in the owned group." That looks like a billing or entitlement problem — like the account simply doesn't have Germany. It wasn't. The Actor's proxy configuration pinned apifyProxyCountry: DE but named no proxy group at all, and on this platform a group-less proxy spec doesn't mean "any group" — it resolves to one specific pack, the datacenter one, which is a static US-only pool that cannot serve a German exit under any circumstances. The country pin was correct. It just needed a group that could actually honor it.

Why does "no group named" silently become "datacenter"?

Because a bare {"useApifyProxy": true} isn't a neutral default — Apify Proxy resolves an unspecified group to whichever pack is provisioned on the account, and here that's BUYPROXIES94952, a 27-IP static pack with US exits only. It works fine for targets that don't care what country the request comes from. It cannot produce a German IP no matter what apifyProxyCountry says, because "which pack" and "which country the pack can serve" are two separate constraints, and a spec that only pins the second one is trusting the first one to be right by accident.

Zalando makes this a correctness bug, not just a reachability one. It's a country-specific storefront — prices, stock, and even which products are listed vary by exit country — so a proxy that resolves to the wrong region wouldn't just risk a block, it would return real, well-formed, completely wrong data: a plausible price in the wrong currency's ballpark, a "the item is available" that's true in one country and false in the one the customer actually cares about. That's the same failure shape as shipping no proxy pin at all, just quieter, because nothing about the response looks broken.

How do you tell "wrong pack" apart from "no permission"?

By reading what actually failed. CONNECT 407 from Apify's proxy layer, with a message naming the exit country as the problem, means the tunnel was refused before any request reached Zalando — this is an infrastructure-layer rejection, not a Zalando anti-bot response and not an account-billing block. The fix lived entirely in .actor/input_schema.json and src/models.py: name apifyProxyGroups: ["RESIDENTIAL"] explicitly alongside the existing apifyProxyCountry: "DE" pin, in both the default and the prefill. Once the group is named, the same country pin that was failing to resolve now does exactly what it was always supposed to do.

A proxy error that blames "country not in owned group" is a package-selection problem, not a country problem — the country pin can be completely correct and still fail if nothing tells the platform which pool to pull that country from.

What the Actor gives you

One row per Zalando.de product card: SKU, name, brand, silhouette (SNEAKER, BOOTS, ANKLE_BOOTS, and so on), gender grouping, price and currency, item condition when not new, product URL, primary image, available sizes, and any merchandising flags shown on the card. You give it one or more category path segments — copy them straight from a Zalando category URL — and it pages through every category, with per-item fault isolation so one bad card doesn't sink the run.

Honest limitations 🚧

Category-path browsing only — Zalando's own /catalog/?q= keyword search endpoint returns a 404 on this site, so there's no free-text search mode here, only category traversal. Data reflects whatever Zalando's DE storefront serves the exit country; a non-DE proxy would return a genuinely different (and wrong-for-this-storefront) catalog, which is exactly why the country pin is load-bearing.

FAQ

Why does the default proxy configuration name a residential group instead of leaving proxy selection generic?
Because a group-less spec on this account resolves to a US-only datacenter pack that cannot serve a German exit — confirmed by a CONNECT 407 in cloud QA. Naming RESIDENTIAL alongside the DE country pin is what makes the pin actually work.

Can I scrape a non-German Zalando storefront?
Not in this version — v1 is Zalando.de only. Other country storefronts would need their own proxy pin and their own recon.

Do I need a Zalando account?
No — this scrapes Zalando's public category pages with no login.

What happens if a category path doesn't exist?
The affected category is skipped and logged; the run still succeeds with rows from every category path that did resolve.

Pricing

$0.20 per run plus $0.006 per product row — $6.20 per 1,000 results. A run that matches nothing costs only the start fee.

Zalando Fashion Listings Scraper on Apify


Built by Devil Scrapes. We rotate residential proxies through the right pool, pin the exit country the target actually needs, and retry with backoff — so a "not in owned group" error is something we catch before it reaches you.

Top comments (0)