Scheduled broken-link and redirect checks that tell you the moment a link breaks — linkbruno.com
Moving a blog or docs site from one static site generator to another feels like a content migration. Copy the Markdown, port the templates, fix the front matter, done.
It's actually a URL migration. Jekyll, Hugo, Eleventy, Gatsby, Astro, Docusaurus and the rest all have different defaults for how a file path becomes a URL. _posts/2021-03-04-hello-world.md might be /2021/03/04/hello-world.html in one, /posts/hello-world/ in another and /blog/hello-world in a third. The new site builds perfectly, every internal link works, and every link anyone else ever made to you is broken.
Here's the checklist I'd use.
Before you start
1. Export the full list of current URLs. Not just the sitemap. Combine three sources: the old site's generated output (every .html file in the build folder), the sitemap, and the paths that actually get traffic from your analytics or server logs for the last year. Logs matter because they include old URLs that already redirect, and those redirects need to survive too.
cd _site && find . -name '*.html' | sed -e 's|^\.||' -e 's|/index\.html$|/|' | sort > ../old-urls.txt
2. Write down the old generator's URL rules. Permalink pattern, whether URLs end in / or .html, how tags and categories are paginated (/tags/go/page/2/ vs /tags/go/2/), how feeds are named (/feed.xml, /index.xml, /rss/). These are the details that differ between tools.
3. Note every non-page URL. RSS and Atom feeds, sitemap.xml, robots.txt, CNAME, _redirects or .htaccess, favicons, images with old paths, downloadable files. Feeds are the most painful to lose: feed readers don't follow your announcement, they just stop updating.
4. Freeze content during the migration, or at least keep a list of posts added after the export. Otherwise you'll compare against a list that's already out of date.
During the migration
5. Configure permalinks to match before you write redirects. Most generators can reproduce the old pattern. Hugo has permalinks in its config, Eleventy uses permalink in front matter or directory data, Astro follows the file structure (so name files to match), Jekyll has permalink: patterns. Reproducing the old URLs exactly is always better than redirecting to new ones.
6. Decide on trailing slashes once, and test it on the real host. /about vs /about/ vs /about.html depends on the generator's output (folders with index.html or flat files) and on the host's rules. Netlify, Vercel, GitHub Pages, Cloudflare Pages and S3 all behave differently with the same files. Build, deploy to a preview, and request both forms of a few URLs.
7. Preserve dates in URLs if the old ones had them. Date-based permalinks often come from the filename in one tool and from front matter in another. If the front matter dates are in a different timezone from the filenames, posts published near midnight move to a different day, and their URLs change by one day. Check a few posts that were published late in the evening.
8. Move redirects with the site. Old redirect rules lived somewhere: Jekyll's redirect_from plugin, a _redirects file, Nginx config. Collect them, translate them into the new host's format, and don't drop any. An old redirect you delete is a link you break.
The new build can pass every internal link check and still break every link from other sites. linkbruno.com checks your live URLs and redirects on a schedule after the switch, so an old path that quietly started returning 404 shows up in a report, not in a reader's comment.
Before you switch DNS
9. Diff the URL lists. Generate the same list from the new build and compare:
cd dist && find . -name '*.html' | sed -e 's|^\.||' -e 's|/index\.html$|/|' | sort > ../new-urls.txt
comm -23 old-urls.txt new-urls.txt > missing.txt
Every line in missing.txt needs either a fix to the permalink config or a redirect. Aim for an empty file.
10. Test the old URLs against the preview deployment. For each URL in the old list, request it on the preview and check it returns 200, or a single 301 to the right place:
while read -r p; do
printf '%s %s\n' "$(curl -s -o /dev/null -w '%{http_code}' "https://preview.example.com$p")" "$p"
done < old-urls.txt | grep -v '^200'
Redirect chains count as problems too. A redirect to a redirect is slower and fragile.
11. Check the internal links in the new build. Templates get rewritten, so navigation, tag links, "next post" links and image paths are new code. Run a link checker over the built output in CI before you deploy.
After the switch
12. Recheck the old URL list on the live site, then keep watching it. A day after the switch, run the same check against production. Then keep the old URL list on a scheduled link check for a few months: CDN caches, host-specific rules and forgotten redirects tend to surface late.
Don't forget pagination and taxonomy pages
Tag, category and archive pages are generated, not written, so they're easy to forget in the URL inventory. They're also the pages whose URLs differ most between generators: /tags/go/ vs /tag/go/, /page/2/ vs /2/, /categories/ vs /topics/. Search engines and other sites link to them less than to posts, but they're often in your own navigation. Include them in the old URL list, and decide explicitly whether to reproduce them or redirect them.
The short version
- Capture every old URL, including from logs.
- Reproduce the old permalinks instead of redirecting where possible.
- Test trailing slashes and feeds on the real host.
- Diff old and new URL lists before DNS changes.
- Keep checking the old URLs after launch with a scheduled checker such as linkbruno.com.
Related reading
- How to find broken links inside PDFs with Python (a script for a whole folder of whitepapers, manuals and invoices)
- Your static site generator can catch broken links at build time. Here's how in 6 popular ones
- Case study (hypothetical): moving from Windows to Linux hosting broke 1,900 links, all because of capital letters
- Moving docs from a GitHub wiki to a docs site? An 11-point checklist so the links survive
- How to check every URL in your sitemap.xml for 404s, redirect chains and timeouts (a Python script for CI)
Which generator migration broke the most URLs for you? I'd love to hear about the trailing-slash surprises in particular.
Top comments (0)