DEV Community

Cover image for Google indexing on Workers: 500s only a burst showed
Georgi Kalchev
Georgi Kalchev

Posted on Originally published at aiskills402.com

Google indexing on Workers: 500s only a burst showed

A site on Cloudflare Workers looked healthy page by page, yet half its pages failed under a burst. What caused it, how we fixed it, and a two-model test.

A site can answer every single request correctly and still fail on half its pages the moment something asks for many at once. On our own site the cause was ordinary: product pages pulled their data from an API that the same site publishes, and whose rate limit was written for outside callers, and a burst of page renders used it up. A one-by-one check never gets there. Only a parallel audit, plus a look at the live Worker log while it runs, shows it.

Read the full report on AISkills402: https://aiskills402.com/blog/google-indexing-cloudflare-500s-under-load?utm_source=devto&utm_medium=social

Top comments (0)