I built an uploader that could create a hundred products in an afternoon. Then it started failing after ten.
No error message worth reading. No rate-limit header. Just ten successes and then nothing, every time I tried a batch.
That day I learned a rule that is not in any tutorial: some platforms cap how much you can create, not how fast you can call the API. It is an anti-spam limit, and it resets on a rolling 24-hour clock.
Why the cap exists, and why it is fine
Once I stopped being annoyed, the logic was obvious. A store that lets one account dump a thousand listings overnight is a spam farm. A rolling daily cap keeps the marketplace usable.
So the fix was not to fight it. The fix was to design around it, which turned out to be better anyway.
The pattern I now use for everything
Instead of one big publish job, I run a small scheduled job:
- It reads a ledger file of what is already live.
- It uploads the next few items that are not.
- It records each new item the moment the API confirms it.
- It stops the moment the cap is hit and goes quiet.
- Tomorrow, it picks up exactly where it left off.
The ledger is the whole trick. Without it, a job that dies halfway cannot tell what it already did, and you get duplicates. With it, the job is idempotent: running it twice changes nothing.
What I got wrong first
- Trusting a summary endpoint. The product list call hid the files and covers, so it looked like nothing was attached. The detail call showed everything was fine. Never trust a summary to tell you the truth about a single item.
- Assuming a failure means a failure. The tenth call returned success and the eleventh returned a soft refusal. If I had retried blindly, I would have created duplicates.
- Publishing everything the moment it was ready. Spacing uploads across days is not slower in any way that matters. It is just calmer.
Why this is the boring part that decides everything
Nobody writes a blog post about a rate limit. But shipping 39 products instead of 10, without a single duplicate, is exactly the difference between a hobby and a small business.
The unglamorous version of automation is not clever code. It is a ledger, a schedule, and a rule that says: when in doubt, do nothing and try again tomorrow.
If you sell digital products, the free sampler here shows the output (transparent PNG, commercial-use, no email wall): Free cute sticker sampler
The scripts behind this, including the ledger-and-schedule uploader, are collected in The Pipeline Starter Kit.
Top comments (0)