DEV Community

appierun
appierun

Posted on

Audit Image Weight Across Your Sitemap with Python (Standard Library Only): Oversized Images, Missing Width/Height, Lazy Loading, CSV Report

Image weight creeps up quietly. Someone uploads a 3000 pixel PNG straight from a design tool and the CMS shows it at 600 pixels wide. A theme update adds loading="lazy" to every image, including the hero at the top of the page. A gallery block drops the width and height attributes, so the text jumps down as each photo arrives. None of this breaks a page. It just makes it slower and jumpier, one upload at a time.

This post builds an image weight auditor in Python using only the standard library (urllib, urllib.robotparser, html.parser, xml.etree, struct, csv, concurrent.futures). It reads your sitemap or sitemap index, fetches every page, collects each <img>, and checks every unique image once. It gets the file size from a HEAD request's Content-Length and falls back to downloading the file when the server refuses HEAD or sends no length. It reads the real pixel size from the first bytes of PNG, GIF, JPEG and WebP files, compares that with the declared width, flags images with no width/height, and checks how loading="lazy" is used. Output is a CSV with one row per image per page, a console summary with the heaviest pages, and an exit code you can use in CI.

web.dev's Browser-level image lazy loading for the web and Optimize Cumulative Layout Shift (showing "Last updated: Feb 7, 2025"), Chrome for Developers' Properly size images Lighthouse page, and Google Search Central's image SEO best practices (showing "Last updated 2026-03-02 UTC") were opened on 17 Nov 2026, and every statement attributed to them below comes from those pages. Appie free-tool descriptions are as of 17 Nov 2026, checked on the tool pages that day. The script was tested on Python 3.13.5 against a local fixture site with planted problems; the output below is real, from runs on 17 Nov 2026.

Disclosure: Appie.run publishes this article. Two free tools mentioned near the end are ours. The script does not depend on them.

Where this fits

This is another script in a small standard-library series. An earlier one found images missing alt text across a sitemap, which is about what an image means. This one is about what an image costs: bytes, pixels, layout space and loading order. The most recent post measured click depth for every sitemap page. Same approach throughout: parse what the server returns, report what is there, and leave the judgment to a person.

What the guidance actually says (and what it does not)

Size. Google's image SEO page says "images are often the largest contributor to overall page size, which can make pages slow and expensive to load," and asks you to "apply the latest image optimization and responsive image techniques." The web.dev lazy loading article cites the HTTP Archive: images "usually take up more bandwidth than any other resource," and "At the 90th percentile, sites send over 5 MB of images on desktop and mobile."

Pixels versus display size. Lighthouse's Properly size images page explains how it decides an image is too big: it "compares the size of the rendered image against the size of the actual image," accounting for device pixel ratio, and fails the image "If the rendered size is at least 4KiB smaller than the actual size." The page now carries a warning that this audit "has moved into the Improve image delivery insight as of Lighthouse 13." The principle is unchanged: "your page should never serve images that are larger than the version that's rendered on the user's screen."

Width and height. The Optimize CLS article lists "Images without dimensions" first among "The most common causes of a poor CLS," and says: "Always include width and height size attributes on your images and video elements. Alternatively, reserve the required space with CSS aspect-ratio or similar." Modern browsers use those attributes to set a default aspect ratio before the image loads, so width: 100%; height: auto in CSS still works with them.

Lazy loading. The lazy loading article describes loading="lazy" as deferring an image "until it reaches a calculated distance from the viewport," and gives a clear caution: "Don't lazy-load images that are likely to be in-viewport when the page loads, especially LCP images." It also notes that without width and height an image defaults to 0x0 pixels, so in a gallery "the browser might decide that all of them fit inside the viewport at the start" and load everything.

What none of these pages give is a kilobyte limit for a single image. The script's default of 200 KB is my own starting line, not a Google or Lighthouse number. Change it with --max-kb and keep it the same between runs. The pixel check uses 2x the declared width by default (--max-scale) so that images exported for high density screens are not flagged. And the script cannot know where the fold is. It does not render the page, so it treats the first two images in the HTML as "probably above the fold" (--eager-first, default 2, because a header logo is usually image one and the hero is image two).

How the audit works

  1. Sitemap URLs. It reads a urlset or a sitemap index (recursively, gzip supported) and keeps every <loc>. robots.txt is respected for page fetches.
  2. Images per page. It parses each page's HTML and collects every <img> in document order. Markup inside <script>, <style>, <template> and <noscript> is ignored, since lazy-load plugins often put a duplicate <img> in <noscript>. If src is empty or a data: placeholder and data-src is set (the classic JavaScript lazy-load pattern), the data-src URL is audited and the loading column says data-src.
  3. One probe per unique image. A logo used on 500 pages is fetched once. The probe sends HEAD and reads Content-Length. If HEAD returns 405 or 501, or the response has no length, it downloads the file in 64 KB chunks and counts the bytes. The size_from column tells you which happened.
  4. Pixel size from the header bytes. When the size came from HEAD, it sends a GET with Range: bytes=0-65535 and reads at most 64 KB, which is enough to find the dimensions in PNG, GIF, WebP and almost every JPEG. If the server ignores Range, the script still stops reading after 64 KB. --no-dimensions skips this step for a HEAD-only run.
  5. Checks per image tag. Size, pixels and attributes are compared per tag, because the same file can be fine on one page and oversized on another.

What the script flags

  • heavy: Meaning: File is larger than --max-kb (default 200 KB); What to check: Compress, switch format, or resize
  • oversized_pixels: Meaning: Pixel width is more than --max-scale (default 2) times the declared width, and the tag has no srcset or <picture>; What to check: Export at the size it is shown, or add srcset
  • missing_dimensions: Meaning: No numeric width and height attributes, and no inline style with aspect-ratio or both width and height; What to check: Add the attributes from the real pixel size
  • lazy_above_fold: Meaning: One of the first --eager-first images has loading="lazy"; What to check: Remove it from the hero; web.dev says not to lazy-load in-viewport images
  • not_lazy: Meaning: An image after the first --eager-first has no loading="lazy" (and no data-src); What to check: Add loading="lazy" if it starts below the fold
  • image_http_404 (or other code) / image_fetch_failed: Meaning: The image did not load; What to check: Fix the path or remove the tag
  • inline_or_empty_src: Meaning: src is empty or a data: URI with no data-src; What to check: Check what the tag is for
  • http_404 / blocked_by_robots / fetch_failed: Meaning: The page itself did not load or is disallowed; What to check: Fix the page or remove it from the sitemap

Two limits to know up front. The script sees only the HTML your server returns, so images injected by JavaScript and CSS background images are not in the report. And a class-based CSS rule that sets aspect-ratio on images will not be seen, so missing_dimensions may list images that are actually fine. Google's image page also notes that Google "doesn't index CSS images," which is one more reason to use real <img> tags for content images.

The script

Save as image_weight.py. No pip install needed.

#!/usr/bin/env python3
"""Audit image weight across a sitemap (standard library only).

Reads a sitemap or sitemap index, fetches every page, collects <img> tags and
checks each image for: file size over a limit (HEAD Content-Length, or a full
download when HEAD gives no size), pixel width far above the declared width,
missing width/height, and loading="lazy" use. Writes a CSV and prints a summary.
"""
import argparse
import concurrent.futures as cf
import csv
import gzip
import re
import struct
import sys
import urllib.error
import urllib.parse
import urllib.request
import urllib.robotparser
import xml.etree.ElementTree as ET
from html.parser import HTMLParser

UA = "image-weight-audit/1.0 (+https://example.com/bot)"
SKIP_TAGS = {"script", "style", "template", "noscript"}
PROBE_BYTES = 65536


def fetch(url, method="GET", timeout=20, headers=None, limit=None):
    """Return (status, headers, body_bytes, final_url). body is read up to limit."""
    h = {"User-Agent": UA}
    h.update(headers or {})
    req = urllib.request.Request(url, method=method, headers=h)
    try:
        with urllib.request.urlopen(req, timeout=timeout) as r:
            body = b"" if method == "HEAD" else (r.read(limit) if limit else r.read())
            return r.status, r.headers, body, r.geturl()
    except urllib.error.HTTPError as e:
        return e.code, e.headers, b"", url
    except Exception as e:  # DNS, timeout, refused
        return None, {}, str(e).encode(), url


def stream_size(url, timeout=30, cap=50 * 1024 * 1024):
    """Download a file in chunks and count bytes. Keeps the first chunk for sniffing."""
    req = urllib.request.Request(url, headers={"User-Agent": UA})
    try:
        with urllib.request.urlopen(req, timeout=timeout) as r:
            head, total = b"", 0
            while True:
                chunk = r.read(65536)
                if not chunk:
                    break
                if len(head) < PROBE_BYTES:
                    head += chunk[: PROBE_BYTES - len(head)]
                total += len(chunk)
                if total > cap:
                    break
            return r.status, r.headers, head, total
    except urllib.error.HTTPError as e:
        return e.code, e.headers, b"", None
    except Exception:
        return None, {}, b"", None


# ---------- sitemap ----------

def strip_ns(tag):
    return tag.rsplit("}", 1)[-1]


def sitemap_urls(url, seen=None):
    seen = seen if seen is not None else set()
    if url in seen:
        return []
    seen.add(url)
    status, _, body, _ = fetch(url)
    if status != 200:
        raise RuntimeError(f"sitemap {url} returned {status}")
    if body[:2] == b"\x1f\x8b":
        body = gzip.decompress(body)
    root = ET.fromstring(body)
    locs = [e.text.strip() for e in root.iter() if strip_ns(e.tag) == "loc" and e.text]
    if strip_ns(root.tag) == "sitemapindex":
        out = []
        for child in locs:
            out.extend(sitemap_urls(child, seen))
        return out
    return locs


# ---------- HTML ----------

class ImgParser(HTMLParser):
    def __init__(self, base):
        super().__init__(convert_charrefs=True)
        self.base = base
        self.skip = 0
        self.picture = 0
        self.images = []

    def handle_starttag(self, tag, attrs):
        a = {k.lower(): (v or "") for k, v in attrs}
        if tag in SKIP_TAGS:
            self.skip += 1
            return
        if self.skip:
            return
        if tag == "base" and a.get("href"):
            self.base = urllib.parse.urljoin(self.base, a["href"])
        elif tag == "picture":
            self.picture += 1
        elif tag == "img":
            src, loading = a.get("src", ""), a.get("loading", "").strip().lower()
            if (not src or src.startswith("data:")) and a.get("data-src"):
                src = a["data-src"]  # JS lazy-load placeholder pattern
                loading = loading or "data-src"
            self.images.append({
                "src": src,
                "abs": "" if src.startswith("data:") or not src else urllib.parse.urljoin(self.base, src),
                "width": a.get("width", ""),
                "height": a.get("height", ""),
                "style": a.get("style", ""),
                "loading": loading,
                "srcset": "srcset" in a or bool(self.picture),
                "in_picture": bool(self.picture),
            })

    def handle_endtag(self, tag):
        if tag in SKIP_TAGS and self.skip:
            self.skip -= 1
        elif tag == "picture" and self.picture:
            self.picture -= 1


def declared_px(value):
    m = re.fullmatch(r"\s*(\d+)(?:px)?\s*", value or "")
    return int(m.group(1)) if m else None


def style_sets_size(style):
    s = style.lower().replace(" ", "")
    return "aspect-ratio:" in s or ("width:" in s and "height:" in s)


# ---------- image headers ----------

def image_dimensions(data):
    """Pixel (width, height) from the first bytes of PNG, GIF, JPEG or WebP."""
    try:
        if data[:8] == b"\x89PNG\r\n\x1a\n":
            return struct.unpack(">II", data[16:24])
        if data[:6] in (b"GIF87a", b"GIF89a"):
            return struct.unpack("<HH", data[6:10])
        if data[:4] == b"RIFF" and data[8:12] == b"WEBP":
            kind = data[12:16]
            if kind == b"VP8X":
                w = int.from_bytes(data[24:27], "little") + 1
                h = int.from_bytes(data[27:30], "little") + 1
                return w, h
            if kind == b"VP8L":
                b = int.from_bytes(data[21:25], "little")
                return (b & 0x3FFF) + 1, ((b >> 14) & 0x3FFF) + 1
            if kind == b"VP8 ":
                w, h = struct.unpack("<HH", data[26:30])
                return w & 0x3FFF, h & 0x3FFF
        if data[:2] == b"\xff\xd8":
            i = 2
            while i + 9 < len(data):
                if data[i] != 0xFF:
                    i += 1
                    continue
                marker = data[i + 1]
                if marker in (0xD8, 0x01) or 0xD0 <= marker <= 0xD7:
                    i += 2
                    continue
                length = struct.unpack(">H", data[i + 2:i + 4])[0]
                if marker in (0xC0, 0xC1, 0xC2, 0xC3, 0xC5, 0xC6, 0xC7,
                              0xC9, 0xCA, 0xCB, 0xCD, 0xCE, 0xCF):
                    h, w = struct.unpack(">HH", data[i + 5:i + 9])
                    return w, h
                i += 2 + length
    except (struct.error, IndexError):
        pass
    return None


def probe_image(url, want_dims=True):
    """HEAD for size; fall back to a full download when HEAD has no size."""
    info = {"status": None, "bytes": None, "size_from": "", "type": "", "px": None}
    status, headers, _, _ = fetch(url, method="HEAD")
    length = headers.get("Content-Length") if headers else None
    if status == 200 and length and length.isdigit():
        info.update(status=200, bytes=int(length), size_from="head",
                    type=headers.get("Content-Type", "").split(";")[0])
        if want_dims:
            s, _, head, _ = fetch(url, headers={"Range": f"bytes=0-{PROBE_BYTES - 1}"},
                                  limit=PROBE_BYTES)
            if s in (200, 206):
                info["px"] = image_dimensions(head)
        return info
    # HEAD refused (405/501), failed, or sent no Content-Length: download it.
    s, h, head, total = stream_size(url)
    info["status"] = s
    if s == 200 and total is not None:
        info.update(bytes=total, size_from="download")
        info["type"] = h.get("Content-Type", "").split(";")[0] if h else ""
        info["px"] = image_dimensions(head)
    return info


# ---------- main ----------

def audit_page(url, robots):
    if robots and not robots.can_fetch(UA, url):
        return url, "blocked_by_robots", []
    status, headers, body, final = fetch(url)
    if status != 200:
        return url, f"http_{status}" if status else "fetch_failed", []
    ctype = headers.get("Content-Type", "") if headers else ""
    charset = re.search(r"charset=([\w-]+)", ctype)
    p = ImgParser(final)
    p.feed(body.decode(charset.group(1) if charset else "utf-8", "replace"))
    return url, "", p.images


def main():
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("sitemap")
    ap.add_argument("--out", default="image_weight.csv")
    ap.add_argument("--max-kb", type=float, default=200, help="flag images heavier than this")
    ap.add_argument("--max-scale", type=float, default=2.0,
                    help="flag pixel width above declared width x this")
    ap.add_argument("--eager-first", type=int, default=2,
                    help="first N images per page are treated as above the fold")
    ap.add_argument("--no-dimensions", action="store_true",
                    help="HEAD only, skip reading pixel sizes")
    ap.add_argument("--workers", type=int, default=4)
    ap.add_argument("--no-robots", action="store_true")
    args = ap.parse_args()

    try:
        pages = list(dict.fromkeys(sitemap_urls(args.sitemap)))
    except Exception as e:
        print(f"Could not read sitemap: {e}", file=sys.stderr)
        return 2

    robots = None
    if not args.no_robots:
        parts = urllib.parse.urlsplit(args.sitemap)
        robots = urllib.robotparser.RobotFileParser(f"{parts.scheme}://{parts.netloc}/robots.txt")
        try:
            robots.read()
        except Exception:
            robots = None

    with cf.ThreadPoolExecutor(args.workers) as ex:
        page_results = list(ex.map(lambda u: audit_page(u, robots), pages))

    unique = sorted({img["abs"] for _, _, imgs in page_results for img in imgs if img["abs"]})
    with cf.ThreadPoolExecutor(args.workers) as ex:
        probes = dict(zip(unique, ex.map(lambda u: probe_image(u, not args.no_dimensions), unique)))

    rows, flag_counts, page_bytes = [], {}, {}
    max_bytes = args.max_kb * 1024
    for page, page_error, imgs in page_results:
        if page_error:
            rows.append({"page": page, "flags": page_error})
            flag_counts[page_error] = flag_counts.get(page_error, 0) + 1
            continue
        counted = set()
        for n, img in enumerate(imgs, 1):
            flags = []
            pr = probes.get(img["abs"], {})
            dw, dh = declared_px(img["width"]), declared_px(img["height"])
            px = pr.get("px")
            if not img["abs"]:
                flags.append("inline_or_empty_src")
            elif pr.get("status") != 200:
                flags.append(f"image_http_{pr.get('status')}" if pr.get("status") else "image_fetch_failed")
            else:
                if pr["bytes"] is not None and pr["bytes"] > max_bytes:
                    flags.append("heavy")
                if px and dw and not img["srcset"] and px[0] > dw * args.max_scale:
                    flags.append("oversized_pixels")
                if img["abs"] not in counted and pr["bytes"]:
                    page_bytes[page] = page_bytes.get(page, 0) + pr["bytes"]
                    counted.add(img["abs"])
            if not (dw and dh) and not style_sets_size(img["style"]):
                flags.append("missing_dimensions")
            if n <= args.eager_first and img["loading"] == "lazy":
                flags.append("lazy_above_fold")
            elif n > args.eager_first and img["loading"] not in ("lazy", "data-src"):
                flags.append("not_lazy")
            for f in flags:
                flag_counts[f] = flag_counts.get(f, 0) + 1
            rows.append({
                "page": page, "position": n, "image": img["abs"] or img["src"][:40],
                "kb": f"{pr['bytes'] / 1024:.1f}" if pr.get("bytes") is not None else "",
                "size_from": pr.get("size_from", ""), "type": pr.get("type", ""),
                "pixels": f"{px[0]}x{px[1]}" if px else "",
                "declared": f"{img['width'] or '-'}x{img['height'] or '-'}",
                "loading": img["loading"] or "(none)",
                "srcset": "yes" if img["srcset"] else "",
                "flags": ";".join(flags),
            })

    fields = ["page", "position", "image", "kb", "size_from", "type", "pixels",
              "declared", "loading", "srcset", "flags"]
    with open(args.out, "w", newline="", encoding="utf-8") as fh:
        w = csv.DictWriter(fh, fieldnames=fields)
        w.writeheader()
        w.writerows(rows)

    n_imgs = sum(len(i) for _, _, i in page_results)
    total = sum(p["bytes"] or 0 for p in probes.values() if p.get("bytes"))
    print(f"Pages in sitemap: {len(pages)}")
    print(f"Image tags: {n_imgs} ({len(unique)} unique URLs, {total / 1024:.0f} KB total)")
    sources = {}
    for p in probes.values():
        if p.get("size_from"):
            sources[p["size_from"]] = sources.get(p["size_from"], 0) + 1
    print("Size source: " + ", ".join(f"{k} {v}" for k, v in sorted(sources.items())))
    print("\nFlags:")
    for f, c in sorted(flag_counts.items(), key=lambda kv: (-kv[1], kv[0])):
        print(f"  {f:<22}{c}")
    if not flag_counts:
        print("  none")
    print("\nHeaviest pages (unique image bytes):")
    for page, b in sorted(page_bytes.items(), key=lambda kv: -kv[1])[:5]:
        print(f"  {b / 1024:>8.1f} KB  {page}")
    heavy = sorted((p["bytes"], u) for u, p in probes.items()
                   if p.get("bytes") and p["bytes"] > max_bytes)
    if heavy:
        print(f"\nImages over {args.max_kb:.0f} KB:")
        for b, u in reversed(heavy):
            print(f"  {b / 1024:>8.1f} KB  {u}")
    print(f"\nCSV written to {args.out} ({len(rows)} rows)")
    return 1 if flag_counts else 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

A few implementation notes:

  • Why HEAD first. On a site with thousands of images, downloading every file to measure it is slow and costs the host bandwidth. Content-Length from HEAD is free. The download fallback is there because some CDNs and image proxies reject HEAD or stream responses with no length.
  • Why only the first 64 KB for pixels. Every supported format puts its dimensions in the first few bytes, except JPEG, where the size sits in the frame header after any metadata segments. A camera JPEG with a large embedded thumbnail can push that past 64 KB; those rows show an empty pixels column rather than a wrong number.
  • SVG has no pixel size to compare, so only its byte size is checked. A 100 KB SVG icon is usually an auto-traced bitmap and worth a look even under the limit.
  • Responsive images (srcset or inside <picture>) are never flagged oversized_pixels, because the browser picks the candidate. The src fallback is still weighed, so a heavy fallback still shows up as heavy.

Testing it against a local fixture

I built a small shop site for a made-up outdoor gear brand, "Larkspur," with a sitemap index pointing at one child sitemap (six URLs). Every page has a 120x32 SVG logo in the header as image one. Test images were generated with Pillow by the fixture builder (gradient plus noise, so they compress like photos); the audit script itself uses only the standard library. The planted problems:

  • Home page: a 1600x900 JPEG hero with dimensions and no lazy loading (correct), three 400x300 WebP thumbnails with loading="lazy" (correct). The hero is about 500 KB.
  • Product page: the hero has loading="lazy". A 3000x2000 PNG detail shot is declared as 600x400. Two gallery images have no width or height. A <noscript> copy of the detail image is there to check it is ignored.
  • Blog post: images on /cdn/, where the server answers HEAD with 405, so sizes must come from a download. A 2400x1600 photo declared at 1200x800 (exactly 2x, so it should not be flagged as oversized). A map GIF without loading="lazy". A JavaScript lazy-load image with a data: placeholder and data-src. An image with srcset.
  • About page: a team photo on /streamed/, where the server sends no Content-Length at all, with loading="lazy" and only an inline aspect-ratio style. A link to a deleted image (404).
  • Sitemap: /drafts/spring-sale is disallowed in robots.txt, and /old-catalog returns 404.

The fixture is served by a small handler that maps clean URLs to .html files and simulates the two awkward image hosts:

"""Serve ./site with clean URLs. /cdn/ refuses HEAD (405); /streamed/ sends no Content-Length."""
import functools
import http.server
import os


class Handler(http.server.SimpleHTTPRequestHandler):
    def map_path(self):
        path = self.path.split("?")[0].split("#")[0]
        path = path.rstrip("/") or "/"
        if os.path.isdir("site" + path):
            path = path.rstrip("/") + "/index.html"
        elif not os.path.splitext(path)[1]:
            path += ".html"
        self.path = path

    def do_HEAD(self):
        if self.path.startswith("/cdn/"):
            self.send_error(405)
            return
        if self.path.startswith("/streamed/"):
            self.send_response(200)
            self.send_header("Content-Type", "image/jpeg")
            self.end_headers()
            return
        self.map_path()
        super().do_HEAD()

    def do_GET(self):
        if self.path.startswith("/streamed/"):
            with open("site" + self.path, "rb") as f:
                data = f.read()
            self.send_response(200)
            self.send_header("Content-Type", "image/jpeg")
            self.send_header("Connection", "close")
            self.end_headers()
            self.wfile.write(data)
            self.close_connection = True
            return
        self.map_path()
        super().do_GET()

    def log_message(self, *args):
        pass


http.server.ThreadingHTTPServer(
    ("127.0.0.1", 8796),
    functools.partial(Handler, directory="site")).serve_forever()
Enter fullscreen mode Exit fullscreen mode

Run the server, then from another terminal:

python3 image_weight.py http://127.0.0.1:8796/sitemap.xml --out image_weight.csv
echo "exit code: $?"
Enter fullscreen mode Exit fullscreen mode

Real output on Python 3.13.5:

Pages in sitemap: 6
Image tags: 18 (13 unique URLs, 15912 KB total)
Size source: download 3, head 9

Flags:
  heavy                 6
  lazy_above_fold       2
  missing_dimensions    2
  blocked_by_robots     1
  http_404              1
  image_http_404        1
  not_lazy              1
  oversized_pixels      1

Heaviest pages (unique image bytes):
   12145.5 KB  http://127.0.0.1:8796/products/tent
    2436.4 KB  http://127.0.0.1:8796/blog/ridge-trail
    1164.9 KB  http://127.0.0.1:8796/about
     629.2 KB  http://127.0.0.1:8796/

Images over 200 KB:
   11578.6 KB  http://127.0.0.1:8796/img/tent-detail.png
    1964.9 KB  http://127.0.0.1:8796/cdn/ridge-sunrise.jpg
    1164.8 KB  http://127.0.0.1:8796/streamed/team.jpg
     500.9 KB  http://127.0.0.1:8796/img/hero-trail.jpg
     361.6 KB  http://127.0.0.1:8796/img/tent-hero.jpg

CSV written to image_weight.csv (20 rows)
exit code: 1
Enter fullscreen mode Exit fullscreen mode

The product and about page rows from image_weight.csv:

page,position,image,kb,size_from,type,pixels,declared,loading,srcset,flags
http://127.0.0.1:8796/products/tent,1,http://127.0.0.1:8796/img/logo.svg,0.1,head,image/svg+xml,,120x32,(none),,
http://127.0.0.1:8796/products/tent,2,http://127.0.0.1:8796/img/tent-hero.jpg,361.6,head,image/jpeg,1200x800,1200x800,lazy,,heavy;lazy_above_fold
http://127.0.0.1:8796/products/tent,3,http://127.0.0.1:8796/img/tent-detail.png,11578.6,head,image/png,3000x2000,600x400,lazy,,heavy;oversized_pixels
http://127.0.0.1:8796/products/tent,4,http://127.0.0.1:8796/img/tent-side.jpg,102.5,head,image/jpeg,800x600,-x-,lazy,,missing_dimensions
http://127.0.0.1:8796/products/tent,5,http://127.0.0.1:8796/img/tent-door.jpg,102.7,head,image/jpeg,800x600,-x-,lazy,,missing_dimensions
http://127.0.0.1:8796/about,1,http://127.0.0.1:8796/img/logo.svg,0.1,head,image/svg+xml,,120x32,(none),,
http://127.0.0.1:8796/about,2,http://127.0.0.1:8796/streamed/team.jpg,1164.8,download,image/jpeg,2000x1333,-x-,lazy,,heavy;lazy_above_fold
http://127.0.0.1:8796/about,3,http://127.0.0.1:8796/img/founders-2019.jpg,,,,,600x400,lazy,,image_http_404
Enter fullscreen mode Exit fullscreen mode

What it found, against what was planted:

  • All planted problems were caught. The 11.3 MB PNG (tent-detail.png) is both heavy and oversized_pixels: 3000 pixels wide in a 600 pixel slot. It alone makes the product page the heaviest on the site at 12,145.5 KB (about 11.9 MB) of unique image bytes.
  • The two lazy heroes (product hero and the about page team photo) are lazy_above_fold. The team photo has no width or height attributes, but its inline aspect-ratio style reserves the space, so it was correctly not flagged as missing_dimensions.
  • The fallbacks worked. size_from shows download for both /cdn/ images (HEAD returned 405) and for the team photo (no Content-Length), and their pixel sizes were still read.
  • The 2x photo on the blog was flagged heavy (1.9 MB) but not oversized_pixels, because 2400 is not more than 2 x 1200. Whether that is right depends on your audience; --max-scale 1.5 would flag it.
  • Not flagged, as intended: the logo (image one on every page), the home page hero (eager, with dimensions), the lazy thumbnails, the data-src image (the loading column says data-src), and the <noscript> duplicate, which never appears. The srcset image on the blog is heavy because its 361.6 KB fallback is, but not oversized_pixels.
  • The 404 image, the 404 page and the robots-blocked page each got their own row.

After fixing the fixture

A second script re-exported the big images as WebP at their display width (1200 or 1600 pixels), removed loading="lazy" from the two heroes and added fetchpriority="high" to the product hero, added width and height to the gallery and team photo, added loading="lazy" to the map, pointed the srcset fallback at the 800 pixel file, removed the broken image, and dropped the two dead URLs from the sitemap. Same command, real output:

Pages in sitemap: 4
Image tags: 17 (13 unique URLs, 1821 KB total)
Size source: download 3, head 10

Flags:
  heavy                 3

Heaviest pages (unique image bytes):
     657.8 KB  http://127.0.0.1:8796/products/tent
     613.4 KB  http://127.0.0.1:8796/
     447.2 KB  http://127.0.0.1:8796/blog/ridge-trail
     205.5 KB  http://127.0.0.1:8796/about

Images over 200 KB:
     485.1 KB  http://127.0.0.1:8796/img/hero-trail-1600.webp
     320.1 KB  http://127.0.0.1:8796/img/tent-hero-1200.webp
     205.4 KB  http://127.0.0.1:8796/streamed/team-1200.webp

CSV written to image_weight_after.csv (17 rows)
exit code: 1
Enter fullscreen mode Exit fullscreen mode

Total image bytes went from 15,912 KB to 1,821 KB, and the product page from 12,145.5 KB to 657.8 KB. Three images are still over 200 KB. That is partly the fixture: noise-filled test images compress far worse than real photos. It is also the normal end state of an audit. The heroes are the largest images on the site and the ones people see first, so this is where a person decides between compressing harder and setting a higher line for heroes. Here is the same site with --max-kb 500 and --no-dimensions (a HEAD-only run with download fallback):

Pages in sitemap: 4
Image tags: 17 (13 unique URLs, 1821 KB total)
Size source: download 3, head 10

Flags:
  none

Heaviest pages (unique image bytes):
     657.8 KB  http://127.0.0.1:8796/products/tent
     613.4 KB  http://127.0.0.1:8796/
     447.2 KB  http://127.0.0.1:8796/blog/ridge-trail
     205.5 KB  http://127.0.0.1:8796/about

CSV written to image_weight_500.csv (17 rows)
exit code: 0
Enter fullscreen mode Exit fullscreen mode

Exit code 0, so this is the setting you could hold the site to in CI.

Running it on your own sitemap

python3 image_weight.py https://example.com/sitemap.xml --workers 2 --out image_weight.csv
Enter fullscreen mode Exit fullscreen mode

Options worth knowing:

  • --max-kb 150 changes the size line. Pick one and keep it fixed so CSVs from different weeks compare.
  • --max-scale 1.5 is stricter on pixels. Leave it at 2 if a lot of your traffic is on high density phones.
  • --eager-first 1 if your pages have no image logo, or 3 if a header logo, a badge and a hero all come first.
  • --no-dimensions skips the ranged GET and is the lightest run you can do. Sizes still fall back to a download where HEAD gives nothing, so on a CDN that rejects HEAD the run downloads those images in full.
  • --workers 2 keeps the load polite. Images on a third-party CDN are fetched too, so a big site means a lot of requests.

Sort the CSV by kb, then filter flags. Images shared across templates (a hero background, a footer badge) show up on every page, so fix those first: one export fixes hundreds of rows.

If you only want to check the sitemap side before running the audit, two free tools on our site can help. The Sitemap URL Extractor takes pasted sitemap XML or a sitemap URL and returns one URL per line from urlset loc tags; for a sitemap index it lists the child sitemap locations, and you run it again on a child to get its page URLs. The XML Sitemap Validator checks the root element, absolute URLs, lastmod, changefreq, priority, duplicates and the 50,000 URL limit, and gives a score that starts at 100. Both ask for a captcha before each run. Neither looks at images; that is what the script is for.

Fixing what it finds

  1. Start with oversized_pixels. Exporting at the displayed width (or 2x for high density screens) is usually the biggest single saving. Lighthouse's page names responsive images, image CDNs and SVG for icons as the main strategies, and Google's image page shows srcset and <picture> with a src fallback.
  2. Then heavy. Re-encode in a modern format and compress. Google Search supports BMP, GIF, JPEG, PNG, WebP, SVG and AVIF in img src, per its image page, so WebP and AVIF are safe choices.
  3. Fix lazy_above_fold before not_lazy. A lazy hero delays the image people see first. web.dev says to use the browser's default eager loading for images visible on load, "especially for LCP images," and to raise priority with fetchpriority="high" if needed.
  4. Add width and height everywhere. Take them from the pixels column (or the display size, keeping the aspect ratio). Keep height: auto in your CSS so the image still scales.
  5. Then add loading="lazy" to images further down. Keep each image's width and height, so lazy images in a gallery do not all count as in the viewport at 0x0.
  6. Rerun and diff. Keep each CSV. A page that jumps from 600 KB to 6 MB between deploys is usually one new upload.

Extending it

  • Weigh every srcset candidate. Parse srcset into URL and width pairs and probe each, so you can see the bytes a phone actually downloads.
  • Check the rendered size. For an exact Lighthouse-style comparison you need layout, which means a headless browser. The script's declared-width check is a cheap proxy.
  • Add response headers. Cache-Control and Content-Type are already on the HEAD response. Flag images with no caching or with a type that does not match the file extension (Google's image page recommends they match).
  • Fail CI on regressions only. Compare kb per image URL with the previous run and fail when a page's total grows by more than a set percentage.

FAQ

What is a good maximum size for one image?
None of the pages cited here give one. The script defaults to 200 KB as a starting line. Heroes and product photos often need more than icons and thumbnails, so some teams run the script twice with different limits, or filter the CSV by page section.

Why use HEAD instead of downloading everything?
Content-Length from HEAD gives the size without transferring the file, which matters when a sitemap references thousands of images. The script downloads only when HEAD is refused or has no length, and the size_from column shows which images that was.

Does Content-Length match what the browser downloads?
It is the size of the response the server sent to the script. With compression or an image CDN that serves WebP or AVIF based on the browser's Accept header, a browser can receive a different file. Send the same Accept header your users' browsers send if you want to measure what they get.

Should every image have loading="lazy"?
No. web.dev says not to lazy-load images that are likely to be in the viewport on load, especially the LCP image. That is why the script flags lazy images among the first two and only suggests loading="lazy" for the rest.

Do width and height stop images from being responsive?
No. The Optimize CLS article explains that browsers use the attributes to compute an aspect ratio, and CSS like width: 100%; height: auto still sizes the image to its container.

Can I run it in CI?
Yes. It exits 1 when anything is flagged, 0 when clean and 2 when the sitemap cannot be read. Run it against staging with the limits you settled on.

Wrap-up

Image weight is one of the few performance problems you can audit from a sitemap without a browser. A HEAD request gives the bytes, the first few kilobytes of the file give the pixels, and the HTML gives the declared size and the loading attribute. Put those together per page and the CSV shows exactly which upload made which page heavy, which hero is waiting on lazy loading, and which gallery is still shifting text around. Fix the template images first, settle on your limits, and rerun after every redesign.

RG, Founder, Appie.run

Top comments (1)

Collapse
 
michael1980 profile image
Toolmaster •

Solid write-up, and the fixture with a HEAD-405 host and a no-Content-Length host is a nice touch. Two edge cases where the pixel check ends up wrong or empty:

  1. EXIF orientation. The JPEG SOF marker stores dimensions before rotation. A portrait phone photo with Orientation 6 or 8 is stored landscape, and browsers apply the orientation by default (image-orientation: from-image), so the script compares the wrong axis with the declared width. Tag 0x0112 in the APP1/Exif segment is inside the same 64 KB probe; for values 5–8, swap width and height.
  2. AVIF. The post recommends it and Google supports it, but image_dimensions returns None for it, so AVIF rows always have an empty pixels column. The size sits in the ispe property box inside meta, normally in the first few hundred bytes, so the existing probe is enough.

Smaller one: data-srcset (the lazy-load plugin variant) isn't treated as srcset, so those images can be flagged oversized_pixels even though the browser picks a smaller candidate once the plugin swaps it in.