Before a site migration, someone has to verify that every old URL either still works or redirects correctly. One at a time, that's hours of work. The bulk tools (Screaming Frog, Ahrefs, SEMrush) cap their free tiers to push you toward a subscription.
So I built a bulk checker:
- Takes up to 100 URLs per run: pasted list, CSV upload (it detects URL columns automatically, whatever the header says), or a sitemap URL whose entries become the list
- Follows redirects manually instead of letting fetch auto-follow, so it records the status code and Location header at every hop (up to 20 deep)
- Flags common problems in plain English: chains of 3+ hops, 302 used for a permanent move, HTTPS → HTTP → HTTPS downgrades, 404s at the end of a redirect
- Streams results back as each URL finishes, with a concurrency pool of 10
- Exports a text report or CSV
The interesting engineering bit is SSRF protection. A server that fetches arbitrary URLs can be tricked into hitting internal resources, and a redirect is the classic way to do it (a public URL that 302s to 169.254.169.254, the cloud metadata address). Validating only the initial URL isn't enough, so every redirect hop is checked against private and reserved ranges before the request is made.
This one runs server-side, because browsers can't read cross-origin redirect chains. URLs aren't stored.
🔗 https://www.forgeplug.com/tools/bulk-url-checker
Curious what other checks would be useful in a migration audit, such as canonical or noindex detection.
Top comments (0)