26 of 114 sitemaps gave every URL the same lastmod. One repeated a millisecond 312 times.

I read 30,353 <loc> entries out of 114 sitemaps. On 26 of those domains, every single URL carried the same <lastmod> value. One domain repeated an identical timestamp — down to the millisecond — 312 times.

That is not a freshness signal. That is a build log line that wandered into an XML file.

Here is the count, the two ways it happens, and the ledger that fixes it without inventing dates you cannot defend.

What I actually collected

For each domain I fetched the root sitemap.xml, followed child sitemaps one level deep, and parsed every <url> block for its <loc> and <lastmod>. No page crawling yet — this is purely what the site tells a crawler before it fetches anything.

114 domains, 30,353 URLs:

• 36 domains ship no lastmod at all — 9,718 URLs.
• 65 domains date every URL. 13 date only part of the sitemap, leaving 5,177 URLs undated.
• 26 domains put a single value across every URL they date — 3,904 URLs.
• 13 of those 26 share the literal string 2026-10-14 — 2,714 URLs on 13 different hostnames.

"Sitemap has lastmod" sounds like a checked box. What the parser sees is one value repeated N times, and a parser can tell the difference between a date and a constant.

The test is a counter

You do not need a crawler to catch this. You need Counter.

import re
from collections import Counter

URL_BLOCK = re.compile(r"<url>(.*?)</url>", re.I | re.S)
LOC = re.compile(r"<loc>\s*(.*?)\s*</loc>", re.I | re.S)
LASTMOD = re.compile(r"<lastmod>\s*(.*?)\s*</lastmod>", re.I | re.S)

def lastmod_profile(xml: str) -> dict:
    values, urls = [], []
    for block in URL_BLOCK.findall(xml):
        loc = LOC.search(block)
        if not loc:
            continue
        urls.append(loc.group(1).strip())
        lm = LASTMOD.search(block)
        values.append(lm.group(1).strip() if lm else "")

    present = [v for v in values if v]
    counts = Counter(present)
    top_value, top_n = counts.most_common(1)[0] if counts else ("", 0)
    return {
        "urls": len(urls),
        "with_lastmod": len(present),
        "distinct": len(counts),
        "top_value": top_value,
        "top_share": round(top_n / len(present), 3) if present else 0.0,
    }

Run it on a sitemap that is edited by humans and distinct tracks the number of editing sessions. Run it on a collapsed one and distinct is 1 whether the site has 40 URLs or 1,114. The discriminator is top_share == 1.0 with more than a handful of URLs — cheap enough to assert in CI on every deploy.

Failure mode one: frozen at the first render

One domain, 312 URLs, every one stamped 2026-07-18T12:19:02.782Z.

Milliseconds are the tell. A human editing a page does not produce .782. A file write does — and this one still matches the filesystem: the same page's HTTP Last-Modified is Sat, 18 Jul 2026 12:19:02 GMT. The sitemap copied the mtime of the build output, then never updated it.

Sampling twelve pages from that domain found three distinct Last-Modified values spanning 105 days, including a homepage rebuilt on 31 October. The sitemap still tells Google that homepage last changed on 18 July. The content had moved on; the timestamp had not. That is the worst combination, because a stale date on a page that did change is a page Google has no reason to revisit — a 2026 guide to buying Bitcoin in the US on that domain is a clean example of a page whose bytes have not moved since the July build.

The usual cause is generating lastmod from os.path.getmtime() of the rendered file. That works exactly once: the next build rewrites every file, and if your pipeline is smarter than a touch, the mtime stops advancing at all — which is exactly how you end up frozen in July.

Failure mode two: stamped with the build date, not the content date

The opposite failure looks healthier and is just as useless. One domain has 1,114 URLs, but only 3 of them carry a lastmod at all — all three stamped 2026-10-31, while two sampled pages from the same domain return Last-Modified: Thu, 15 Oct 2026 13:34:24 GMT.

Partial coverage is its own bug, and it hides the collapse: the counter sees one distinct value on three URLs and reports a perfectly informative ratio. It is not informative. It is three URLs out of 1,114, forward-dated by 16 days, sitting on a sitemap that dates almost nothing.

The full-coverage version of the same failure is easier to see. Thirteen domains in this set produced byte-identical 2026-10-14 across 2,714 URLs — 941 of them on a single host. That is what a shared template deploy looks like from the outside: one build, many hostnames, one timestamp smeared over all of them. A domain outside that group went further and put 2026-10-07T00:00:00.000Z — midnight, zero microseconds — on all 66 of the URLs it dates, including its FAQ page, which is the sort of page that changes roughly never. The other 524 URLs in that sitemap have no date at all.

Both failure modes trip the same counter. They are not the same bug, and the fix is not the same: one needs the timestamp to advance, the other needs it to stop advancing in lockstep.

The server already disagrees with the sitemap

There is a free second opinion on every response: the Last-Modified header. I paired it with lastmod only where the sitemap actually dated that exact URL — 115 pages across 60 domains. They disagreed by more than a day on 85 of them: 62 headers newer than the sitemap claim, 23 older, 30 within a day.

Domain	Dated URLs	Sitemap says	Page Last-Modified	Gap
openaistocknews.com	41 / 41	2025-07-01	2026-11-02	489 days stale
cryptobullforecast.com	116 / 116	2026-06-01	2026-10-31	152 days stale
coinatlashub.com	312 / 312	2026-07-18	2026-10-31	105 days stale
nextcryptopump.com	270 / 270	2026-10-14	2026-06-02	134 days ahead

The first row is a domain still advertising July 2025 as its last change while serving files written in November 2026. That is not a rounding error in a hint field; it is a site telling a crawler, in a machine-readable format, that sixteen months of work did not happen.

The last row is the inverse, and it is the one people miss: the sitemap claims a modification 134 days after the file it describes. Nothing was edited on 14 October — a batch job ran and stamped its own clock across 270 entries.

The inverse case is just as easy to find on a bigger sitemap. On a domain whose 327 dated entries all claim 14 October, three sampled market pages carried Last-Modified: Mon, 02 Nov 2026 18:35:15 GMT — nineteen days of edits the sitemap does not admit to. Pages like that domain's list of highest-probability crypto forecasts are regenerated on a schedule, and the only field that could have told a crawler so is pinned to a single value.

What lastmod is and is not

lastmod is not a ranking factor. It is a scheduling hint: it lets a crawler spend a recrawl budget on pages that moved instead of pages that did not. That is why a constant is worse than nothing. A constant carries zero bits, so the crawler falls back to its own heuristics — and the published guidance is that lastmod is only used when it is consistently and verifiably accurate. Inconsistency does not degrade one entry; it teaches the crawler to discount the field for the whole host.

Note the asymmetry in the numbers above: 36 domains shipped no lastmod at all. That is a legitimate, documented choice. Nobody is penalised for omitting it. The only unrecoverable option is the fabricated one, because once a crawler has evidence that your dates are decorative, restoring trust takes far longer than deleting the field would have.

The fix: hash what you render, publish when the hash changed

Do not compute lastmod at render time. Compute it once, when the content actually changes, and store the answer.

import hashlib, re, sqlite3, datetime

STRIP = re.compile(r"<(script|style|noscript)[^>]*>.*?</\1>", re.I | re.S)
MAIN = re.compile(r"<(article|main)[^>]*>(.*?)</\1>", re.I | re.S)
TAGS = re.compile(r"<[^>]+>")
ENTITIES = re.compile(r"&[a-z#0-9]+;", re.I)

def content_fingerprint(html: str) -> str:
    """Hash the part of the page a reader would notice changing."""
    main = MAIN.search(html)
    body = main.group(2) if main else html
    text = TAGS.sub(" ", STRIP.sub(" ", body))
    words = re.findall(r"[a-z0-9']+", ENTITIES.sub(" ", text).lower())
    return hashlib.sha256(" ".join(words).encode()).hexdigest()

DB = sqlite3.connect("lastmod.db")
DB.execute("""CREATE TABLE IF NOT EXISTS page (
    url TEXT PRIMARY KEY,
    fingerprint TEXT NOT NULL,
    changed_at TEXT NOT NULL
)""")

def record(url: str, html: str, now: datetime.datetime) -> str:
    fp = content_fingerprint(html)
    row = DB.execute("SELECT fingerprint, changed_at FROM page WHERE url = ?",
                     (url,)).fetchone()
    if row is None:
        # first sighting: the honest value is the publish date, not today
        changed_at = now.isoformat(timespec="seconds")
        DB.execute("INSERT INTO page VALUES (?, ?, ?)", (url, fp, changed_at))
    elif row[0] != fp:
        changed_at = now.isoformat(timespec="seconds")
        DB.execute("UPDATE page SET fingerprint = ?, changed_at = ? WHERE url = ?",
                   (fp, changed_at, url))
    else:
        changed_at = row[1]          # unchanged: keep the old date
    DB.commit()
    return changed_at

Three details matter more than the code.

Fingerprint the content, not the file. A template change to the footer, the cookie banner, or a nav link rewrites every byte of every page. Hashing the <main>/<article> container after stripping scripts and styles means cosmetic deploys do not touch a single lastmod. If your pages have no semantic container, hash the inline text minus a fixed list of chrome selectors — but pick something, because hashing the whole document guarantees permanent churn.

Clamp to reality. lastmod must never exceed the time you actually published. A build that backfills archives should write the archive's real date, not the deploy's. If you do not know the date, omit <lastmod> for that URL — the protocol allows per-URL omission, and crawlers handle missing values far better than wrong ones.

One row per canonical URL. If two paths canonicalise to one, they share one fingerprint row. Otherwise the same content gets two clocks and you have rebuilt the bug in a different shape.

A CI check that fails when the field collapses

The counter from earlier is the assertion. Run it against the generated sitemap before deploy:

def check_informative(profile: dict, min_distinct_ratio: float = 0.02) -> list[str]:
    problems = []
    total, distinct = profile["with_lastmod"], profile["distinct"]
    if total == 0:
        return problems                      # omitting lastmod is allowed
    if distinct == 1 and total > 5:
        problems.append(f"{total} URLs, 1 distinct lastmod: {profile['top_value']}")
    elif distinct / total < min_distinct_ratio:
        problems.append(f"{distinct} distinct values over {total} URLs")
    if profile["top_share"] >= 0.98 and total > 20:
        problems.append(f"{profile['top_share']:.0%} of URLs share one timestamp")
    return problems

A healthy content site fails none of these. A site where every page really was published in one import will fail them honestly, which is the correct outcome: those pages have one real date, and the sitemap should say so once rather than pretend to be a change feed.

Add a coverage assertion too, because the 1,114-URL domain above would pass the collapse check on three dated URLs: with_lastmod / urls should be either 1.0 or 0.0, never 0.27. And pair a sample against the response header — abs(Last-Modified - lastmod) < 1 day — which is the only one of the three checks that catches a build stamp applied to everything while the build runs often.

What this does not fix

lastmod will not get a page indexed. If the page is noindex, canonicalised elsewhere, or orphaned, an accurate timestamp changes nothing — and in this same fleet, none of 180 sampled pages carried a noindex at all, which means the timestamp was the only freshness signal on offer and 26 domains spent it on a constant.

The countermeasure is one function, one table, and one test. What it buys is narrow and real: when a page changes, the crawl budget goes there. 30,353 URLs is a lot of budget to spend on a number that never moves.