DEV Community

Marin T. Kael
Marin T. Kael

Posted on

A 429 that has not cleared in 111 days is not a rate limit

I run a small daily pipeline that pushes my sitemap URLs to IndexNow and to Bing's direct submit endpoint, then stores the HTTP status of each push in a table. It has done that 193 times since 15 May. That table turned out to be more interesting than the pushes.

Here is the whole of it, condensed:

  • 68 runs got a 2xx from IndexNow.
  • 50 runs got a 429.
  • 75 runs recorded nothing, because a diff gate skipped the push when the sitemap had not changed. Those are honest skips, not failures.
  • The last 2xx was on 10 June. Every recorded attempt since then is a 429.

That is 111 days of 429 with no recovery, at one push per day.

Why that should not happen

429 means "too many requests". The implied contract is that you stop, you wait, and the window clears. So the reading of my own data should have been easy: back off, resume, done.

Except backing off is exactly what the pipeline already does. One cron run per day, 04:00 UTC, 39 to 51 URLs in a single request. If a 429 survives 111 days of that, it is not describing a request rate. It is describing some other state, reported through the only status code the endpoint has for "no".

The transition is visible in the row

The pipeline normally writes one row per day. On 10 June it wrote three:

2026-06-10 04:00  indexnow 200  urls 41
2026-06-10 15:01  indexnow 429  urls 41
2026-06-10 15:30  indexnow 429  urls 41
2026-06-11 04:00  indexnow 429  urls 41
Enter fullscreen mode Exit fullscreen mode

Same URL count, same payload shape, same key, same host. The 04:00 push was fine. Two more pushes the same afternoon, and it has been 429 ever since, including after the cadence went back to exactly one per day for over three months.

So whatever got tripped that afternoon does not untrip on its own within any window I have been able to outlast.

What rules this out, and what it does not

The same pipeline also submits the same URL list to Bing's direct submit endpoint in the same run. That endpoint returned 200 on 10 of the last 14 recorded runs, and 400 on the four before that. So the URLs are acceptable to Bing, the host verification is intact, and the key file is being served. It is not the content of the submission.

What I cannot rule out, and this is the honest part: my pipeline runs on a Cloudflare Worker. The egress IP is shared with a very large number of other callers. A 429 that is keyed to that IP rather than to my key or my host would look exactly like this, and would also never clear, because it is not my traffic keeping it warm. That explanation fits the data as well as mine does.

The reason I cannot tell them apart is the third column in the row:

{"bing_quota_error":"AbortError: The operation was aborted"}
Enter fullscreen mode Exit fullscreen mode

The quota lookup, the one call that would say whether I am over a limit or throttled, times out. It has timed out on most runs since May. So the pipeline has a status code that says "limit" and no way to name which limit, and both the code and the timeout have been stable for months.

The part worth keeping

The finding I actually trust is not about IndexNow. It is that a retryable status code stopped being retryable and nothing in the system noticed, because 429 is on everybody's soft-error list. Sixteen stages, zero failures, every daily report green, and one of those stages had not succeeded since 10 June.

A soft error that never resolves is a hard error wearing a different hat. If you have a status code in your pipeline that you treat as transient, it is worth asking your own table how long the current streak of it is. Mine was 111 days and I had not asked.

Data source: the run snapshots behind my research dashboard.

Top comments (0)