Checking hreflang Return Links Automatically With a Small Python Crawler

The most common hreflang bug is not a typo in a language code. It is a missing return link: page A says "my German version is B", but page B never points back to A. Search engines may then ignore the pair. On a site with a few hundred localised pages this is tedious to check by hand, so here is a small script that does it.

What we check

For every page in a list of URLs:

1. Fetch the HTML and read all <link rel="alternate" hreflang="..."> tags.
2. For each alternate, fetch that URL and confirm it lists the original page back.
3. Confirm each page includes a self-referencing hreflang entry.
4. Flag alternates that return a non-200 status or redirect.

The script

import requests, re
from collections import defaultdict
from html.parser import HTMLParser

class Alt(HTMLParser):
    def __init__(self):
        super().__init__(); self.alts = {}
    def handle_starttag(self, tag, attrs):
        a = dict(attrs)
        if tag == "link" and a.get("rel") == "alternate" and "hreflang" in a:
            self.alts[a["hreflang"].lower()] = a.get("href", "").strip()

cache = {}
def alternates(url):
    if url in cache: return cache[url]
    r = requests.get(url, timeout=20, allow_redirects=False,
                     headers={"User-Agent": "hreflang-check/1.0"})
    p = Alt()
    if r.status_code == 200: p.feed(r.text)
    cache[url] = (r.status_code, p.alts)
    return cache[url]

def check(urls):
    problems = defaultdict(list)
    for u in urls:
        status, alts = alternates(u)
        if status != 200:
            problems[u].append(f"status {status}"); continue
        if u not in alts.values():
            problems[u].append("no self-reference")
        for lang, target in alts.items():
            if target == u: continue
            t_status, t_alts = alternates(target)
            if t_status != 200:
                problems[u].append(f"{lang} -> {target} returns {t_status}")
            elif u not in t_alts.values():
                problems[u].append(f"{lang} -> {target} has no return link")
    return problems

if __name__ == "__main__":
    urls = [l.strip() for l in open("urls.txt") if l.strip()]
    for page, issues in check(urls).items():
        for i in issues: print(page, "|", i)

Put one URL per line in urls.txt, ideally the canonical URLs from your sitemap, and run it.

Notes from using it

• Trailing slashes matter. https://example.com/de/ and https://example.com/de are different strings. Normalise in your CMS rather than in the checker, so the check reflects what crawlers see.
• Don't follow redirects silently. An alternate that 301s is a real problem worth flagging, which is why allow_redirects=False is set.
• x-default should exist once per cluster and point to a sensible fallback, usually a language selector or the main market page.
• Rate limit if you run this against production: a short time.sleep between requests is polite.

Extending it

The same parser can compare canonical tags with hreflang targets (they should agree), or export a CSV for your content team. For a broader overview of hreflang setup, including URL structure and sitemaps, see this hreflang guide for multilingual websites.

Published by PandaWM, an overseas marketing team based in Hong Kong.