Test your redirect map with bash and curl before you move a site

#webdev #devops #seo #bash

The DNS change is usually the calm part of a site move. The trouble shows up later: someone rebuilds the server config and half the redirect rules don't come back, or a rule left over from the old CMS stacks on top of a new one and every old URL now takes two hops to land.

I've done more than 700 site migrations, and the habit that saved me most often is a dull one. The redirect map lives in the repo as a CSV, and a small script tests it against the live site, the same way a test suite tests code. If the map can be re-run, a broken rule shows up in seconds instead of in Search Console three weeks later.

The map is one row per old URL

These are the columns I use:

old_url,new_url,status,match_type,clicks_12m,referring_domains,notes
https://old.example.com/about-us,https://www.example.com/about/,301,exact,380,12,Path renamed
https://old.example.com/blog/tag/widgets,https://www.example.com/blog/,301,merge,12,0,Thin tag page folded into the blog index
https://old.example.com/old-promo-2017,,410,removed,0,0,Expired offer. No equivalent page; do not send to home

status is 301 or 308 for pages that moved or were merged, and 410 or 404 for pages that are gone. match_type says whether the row is an exact match, a pattern rule, a merge or a removal. Pattern rows get expanded to real URLs before testing, because you can't curl a regex.

The two traffic columns come from Search Console and your backlink tool, and they decide which rows you check by hand first. A broken redirect on a page with 4,000 clicks a year hurts more than one on a dead tag page.

What the test asserts

For every 301 or 308 row, three things have to be true:

The first response is a permanent redirect. Google treats 301 and 308 as permanent and says they don't lose PageRank. A 302 or 307 tells it to keep showing the old URL in results.
The chain ends on new_url, and that page answers 200.
It takes one hop.

For 410 and 404 rows, the old URL has to answer with that code and must not redirect anywhere.

The one-hop rule is stricter than Google needs. Googlebot follows up to 10 redirects, but Google's own advice is to send people straight to the final URL. Two hops usually means an old rule and a new rule are both firing, and that is cheap to fix while you're still on staging.

The script

It needs only bash and curl.

#!/usr/bin/env bash
# check-redirects.sh: test a redirect map against a live site.
# Usage: ./check-redirects.sh redirect-map.csv
set -u
file="${1:?usage: $0 redirect-map.csv}"
pass=0; fail=0
while IFS=, read -r old new status _rest; do
  [ "$old" = "old_url" ] && continue
  [ -z "$old" ] && continue
  status="${status//$'\r'/}"; new="${new//$'\r'/}"
  first=$(curl -s -o /dev/null --max-time 20 --proto '=http,https' -w '%{http_code}' "$old")
  problems=""
  case "$status" in
    301|308)
      read -r last hops final < <(curl -s -o /dev/null -L --max-redirs 10 --max-time 30 --proto '=http,https' --proto-redir '=http,https' \
        -w '%{http_code} %{num_redirects} %{url_effective}\n' "$old")
      case "$first" in 301|308) ;; *) problems="$problems first-response=$first(want $status);";; esac
      [ "$last" = "200" ] || problems="$problems final-status=$last;"
      [ "$final" = "$new" ] || problems="$problems lands-on=$final(want $new);"
      [ "${hops:-0}" -le 1 ] || problems="$problems hops=$hops;"
      ;;
    410|404)
      [ "$first" = "$status" ] || problems="$problems status=$first(want $status);"
      ;;
    *) problems="unknown status column: $status;" ;;
  esac
  if [ -z "$problems" ]; then pass=$((pass+1)); echo "PASS $old"; else fail=$((fail+1)); echo "FAIL $old $problems"; fi
done < "$file"
echo "---"; echo "passed $pass, failed $fail"
[ "$fail" -eq 0 ]

Two curl calls per row. The first one doesn't follow redirects, so it sees exactly what the server answers. The second follows the chain with -L and reports the final status, the number of hops and the URL it ended on. A 308 passes a 301 row and the other way round, since both are permanent. Because the script exits 1 when any row fails, you can drop it into a CI job as it is.

A run against a test set

To show each failure, I built three rows on httpbin.org's redirect endpoint. The first is a correct 301, the second answers 302 where the map wants a 301, and the third is a 301 that lands on another 301 before it gets to the destination.

old_url,new_url,status,match_type
https://httpbin.org/redirect-to?url=https%3A%2F%2Fhttpbin.org%2Fget&status_code=301,https://httpbin.org/get,301,exact
https://httpbin.org/redirect-to?url=https%3A%2F%2Fhttpbin.org%2Fget&status_code=302,https://httpbin.org/get,301,exact
https://httpbin.org/redirect-to?url=https%3A%2F%2Fhttpbin.org%2Fredirect-to%3Furl%3Dhttps%253A%252F%252Fhttpbin.org%252Fget%26status_code%3D301&status_code=301,https://httpbin.org/get,301,exact

The run on 25 October 2026:

PASS https://httpbin.org/redirect-to?url=https%3A%2F%2Fhttpbin.org%2Fget&status_code=301
FAIL https://httpbin.org/redirect-to?url=https%3A%2F%2Fhttpbin.org%2Fget&status_code=302  first-response=302(want 301);
FAIL https://httpbin.org/redirect-to?url=https%3A%2F%2Fhttpbin.org%2Fredirect-to%3Furl%3Dhttps%253A%252F%252Fhttpbin.org%252Fget%26status_code%3D301&status_code=301  hops=2;
---
passed 1, failed 2

It took about 7 seconds and exited with code 1.

On a real move, the failures map to fixes like these:

first-response=302 means a temporary redirect somewhere in the config. Framework helpers often default to it. Express's res.redirect() sends a 302 unless you pass a status.
hops=2 usually means a legacy rule and a new rule are chained. If /a already went to /b and the new map sends /b to /c, map /a straight to /c.
lands-on= means the rule points somewhere other than the map says. Often it's the home page, which is the worst catch-all: Google can treat many unrelated URLs redirected to one page as soft 404s.

Keep it next to the config

Commit the CSV and the script beside the server or CDN config that holds the rules. Run the script once against staging before launch, again right after the DNS change, and again whenever someone touches the redirect config. The next person who rebuilds that config will find a test that tells them what they broke.

The rest of the migration process, all 40 checks from the first crawl to switching off the old server, with both files to download, is on MigrateWeb.

DEV Community