DEV Community

saripalli shanmukha kiran sagar
saripalli shanmukha kiran sagar

Posted on

Free Python script: check if a website looks outdated (stdlib only, robots.txt-respecting)

#python #webdev #beginners #freelancing

If you sell web design or maintenance to small businesses, you probably have a list of prospects — but not all of them need you yet. A homepage that scores high on a few simple, observable "outdated" signals is a much better warm-up than one that just looks… fine.

This is a drop-in Python script that checks a homepage for five of those signals, prints a rough 0–100 "outdated" score and the reasons, and does it all with the standard library. No third-party libraries, no config files. It also respects robots.txt (via urllib.robotparser) and collects no personal data.

How it scores

For each URL you pass on the command line, it fetches only the homepage (follows the final redirect, reads up to 2 MB) and adds points for:

  • No HTTPS (the final URL is not https://) — +25
  • No mobile viewport <meta> tag — +25
  • A footer copyright year that's at least 2 years older than this year — +20 (e.g. (c) 2019 in 2026)
  • jQuery < 3 detected in the page — +15
  • Legacy markup: inline <table> layout with width/bgcolor, a <font> tag, a <marquee>, or a .swf Flash reference — +15

Capped at 100. It prints one line per site with the score, or a dash (-/100) if robots.txt disallows fetching the URL — so you can stream a list through it and spot the highest-scoring prospects.

It does NOT crawl internal links, does NOT collect personal data, and does NOT modify anything on the target site.

Usage 

python3 outdated_site_check.py https://example.com https://another.com
Enter fullscreen mode Exit fullscreen mode

Example output:

https://example.com 0/100   looks reasonably modern
https://oldshop.com 65/100  no HTTPS; no mobile viewport meta tag; footer copyright says 2019
Enter fullscreen mode Exit fullscreen mode

The script

Save this as-is (it is idempotent and needs no config).

#!/usr/bin/env python3
"""
Quick outdated-website checker (homepage only, standard library only).

Usage:  python3 outdated_site_check.py https://example.com https://another.com

Prints a rough 0-100 "outdated" score and the reasons, so web designers can
qualify redesign prospects from a list they already have.

It fetches one page per site, respects robots.txt, and collects no personal data.
"""
import datetime
import re
import sys
import urllib.parse
import urllib.request
import urllib.robotparser

UA = "outdated-site-check/1.0"


def allowed(url):
    p = urllib.parse.urlparse(url)
    rp = urllib.robotparser.RobotFileParser(f"{p.scheme}://{p.netloc}/robots.txt")
    try:
        rp.read()
        return rp.can_fetch(UA, url)
    except Exception:
        return True


def check(url):
    if not url.startswith("http"):
        url = "https://" + url
    if not allowed(url):
        return None, ["robots.txt disallows fetching; skipped"]
    req = urllib.request.Request(url, headers={"User-Agent": UA})
    with urllib.request.urlopen(req, timeout=15) as r:
        final = r.geturl()
        html = r.read(2_000_000).decode("utf-8", "ignore")

    score, why = 0, []
    year = datetime.date.today().year

    if not final.startswith("https://"):
        score += 25; why.append("no HTTPS")
    if not re.search(r'<meta[^>]+name=["\']viewport', html, re.I):
        score += 25; why.append("no mobile viewport meta tag")
    yrs = [int(y) for y in re.findall(r"(?:©|&copy;|copyright)\D{0,20}((?:19|20)\d{2})", html, re.I)]
    if yrs and max(yrs) <= year - 2:
        score += 20; why.append(f"footer copyright says {max(yrs)}")
    jq = re.search(r"jquery[.-]?(\d)\.(\d+)", html, re.I)
    if jq and int(jq.group(1)) < 3:
        score += 15; why.append(f"old jQuery {jq.group(1)}.{jq.group(2)}")
    if re.search(r"<table[^>]+(width|bgcolor)=|<font\b|<marquee\b|\.swf\b", html, re.I):
        score += 15; why.append("table/font-tag layout or Flash")
    return min(score, 100), why or ["looks reasonably modern"]


if __name__ == "__main__":
    for u in sys.argv[1:] or ["https://example.com"]:
        try:
            s, why = check(u)
            print(f"{u}\t{'-' if s is None else s}/100\t" + "; ".join(why))
        except Exception as e:
            print(f"{u}\terror\t{e}")
Enter fullscreen mode Exit fullscreen mode

Feedback welcome — if a signal you care about is missing, tell me and I'll add it.

Note: I also built an Apify actor that collects the same kinds of signals at scale (header checks, tech signals, content freshness, link health) and outputs a prioritized list of pages worth revisiting. The actor is open-source and runs on Apify free: https://apify.com/shanmukhkiransagar/website-redesign-signals

Top comments (0)