Title
How to prove a scraping request is impossible (before you take the money)
Body
A client asked me: can you extract every voucher code from this site?
The honest answer was no, and the useful part of my job that week was proving it — with evidence, not opinion.
Here is the method. It generalises to almost any "can you get all the X" request.
Step 1: Enumerate what the client actually sees
First, unpack the front-end bundle and list the API surface.
15 chunks → 27 endpoint paths
Group them by module: auth, product, order, settlement, voucher.
The voucher module had exactly three endpoints:
POST /api/voucher/redeem
GET /api/voucher/detail
POST /api/voucher/status
There was no list endpoint. No search endpoint. No export.
That's the first piece of evidence, and it's structural: a client asking for "all vouchers" is asking for something the system never built a door to.
Step 2: Bound the capability by dependency, not by reading
Reading the code and concluding "I didn't find a list endpoint" is weak — maybe I missed it.
The stronger version: build the dependency graph.
The voucher module imports exactly two components (VoucherCard, VoucherRedeem), and those components call exactly two endpoints.
If the module's entire dependency closure contains no enumeration call, the capability does not exist. That's a proof about the graph, not a claim about my search thoroughness.
Step 3: Use error behaviour to infer the backend
This is the part most people skip, and it's the most informative.
I sent three kinds of input and recorded the response:
| Input | Response |
|---|---|
| Random 20-char string |
参数异常 (parameter error) |
| Valid token, truncated to 10 chars |
服务器开小差 (server error) |
| Valid token, rapid repeat |
安全受限 (security restricted) |
Three different error paths for three different inputs.
What this tells you: the server isn't doing a table lookup on the token. It's decrypting it, parsing a structure, and failing at different stages.
- Random string → decryption fails → "parameter error"
- Valid prefix → decryption partly succeeds, structure parse fails → "server error"
- Rate → a separate counter → "security restricted"
No lookup table means no enumeration. This is a much stronger claim than "I don't see a list API."
Step 4: Quantify the alternative (if it were enumerable)
Assume the token is 20 characters of base62. That's:
62^20 ≈ 7.0 × 10^35 keys
≈ 119 bits of entropy
At a generous 10,000 requests per second, expected time to hit one valid token is longer than the age of the universe by many orders of magnitude.
Say the number. "You can't" is an opinion. "119 bits, therefore no" is a calculation.
Step 5: Locate the signature so the client knows what is possible
The request carried X-Content-Security, 32 hex characters, generated by a WASM function called handleSecurity.
Finding it matters for two reasons:
- It tells the client their protection is real and I found where it lives — credibility
- It distinguishes "I can't do this" from "I didn't look"
I could reproduce the signature. I still couldn't enumerate vouchers, because signatures authenticate requests — they don't create endpoints that don't exist.
Conflating those two is the most common mistake in this kind of work.
The deliverable
The client paid for a document that said no. It contained:
- The endpoint inventory (27 paths, grouped)
- The dependency graph showing the voucher module's closure
- The three-way error behaviour table with my interpretation
- The entropy calculation
- The location of the signature function
- A recommendation: use the official partner API, or build the internal automation they actually needed
They came back with a second, smaller project that was doable.
Why write this down
Most freelancers in this space say yes and then quietly fail. The client finds out four weeks later, and nobody gets paid.
The alternative is to spend half a day producing evidence, deliver a clear no, and become the person they call for the next thing.
The half day is cheaper than the four weeks — for both sides.
A reusable checklist
For any "can you get all the X" request:
- Enumerate — what endpoints exist? (unpack the bundle)
- Bound — does the dependency closure of the feature contain an enumeration call? (graph, not grep)
- Infer — do different inputs produce different error paths? (if yes, it's decrypting, not looking up)
- Quantify — if enumerable, what's the entropy? (turn "no" into a number)
- Locate — where does the protection actually live? (proves you didn't just give up)
- Recommend — what can be built instead?
Step 6 is where the next contract comes from.
Top comments (0)