A news sitemap looks like a normal sitemap with a few extra tags. It isn't. It has its own rules, and if you break them Google quietly ignores the file instead of telling you. No error in Search Console, no warning — the articles just don't show up in the News tab or the Top stories carousel.
I write for an Indian news site and wanted a check I could run before every deploy. This post is the ~100 lines of Python I ended up with, the four rules worth enforcing, and a stdlib bug I hit on the way.
The rules that actually matter
Google's news sitemap spec is short. The parts that cause real failures:
1. The 48-hour window. A news sitemap should only contain articles published in the last two days. Older entries aren't an error exactly — they're just dead weight that Google skips. If your sitemap is full of last month's articles, the handful of fresh ones get lost in it.
2. The 1,000-entry limit. Go over and the file is rejected outright. High-volume publishers need to split into several sitemaps behind an index.
3. <news:publication_date> must be a real timestamp, in the future-proof sense. Timezone included. A date in the future gets the entry dropped.
4. Every <url> needs a <news:news> block. Easy to break when the same template generates both your regular and news sitemaps.
Everything else — keywords, genres, stock tickers — is optional and mostly ignored now.
The script
import re
import sys
import urllib.request
import xml.etree.ElementTree as ET
from collections import Counter
from datetime import datetime, timedelta, timezone
NS = {
"s": "http://www.sitemaps.org/schemas/sitemap/0.9",
"n": "http://www.google.com/schemas/sitemap-news/0.9",
}
WINDOW_HOURS = 48
TITLE_MAX = 110
Parsing is plain ElementTree. The only wrinkle is that the news tags live in their own namespace, so every lookup needs the prefix:
def parse(xml_bytes):
root = ET.fromstring(xml_bytes)
items = []
for url in root.findall("s:url", NS):
news = url.find("n:news", NS)
if news is None:
items.append({"loc": url.findtext("s:loc", "", NS), "news": False})
continue
items.append({
"loc": url.findtext("s:loc", "", NS),
"news": True,
"title": news.findtext("n:title", "", NS),
"keywords": news.findtext("n:keywords", "", NS),
"published": parse_date(news.findtext("n:publication_date", "", NS)),
})
return items
Then the checks, each one returning a line of human-readable output rather than a stack trace:
def audit(items):
now = datetime.now(timezone.utc)
cutoff = now - timedelta(hours=WINDOW_HOURS)
problems = []
missing_news = [i for i in items if not i["news"]]
if missing_news:
problems.append(f"{len(missing_news)} <url> entries have no <news:news> block")
dated = [i for i in items if i.get("published")]
stale = [i for i in dated if i["published"] < cutoff]
if stale:
oldest = min(i["published"] for i in stale)
problems.append(
f"{len(stale)} articles are older than {WINDOW_HOURS}h "
f"(oldest {oldest.isoformat()}) — Google ignores these"
)
future = [i for i in dated if i["published"] > now + timedelta(minutes=5)]
if future:
problems.append(f"{len(future)} articles have a publication_date in the future")
long_titles = [i for i in items if len(i.get("title") or "") > TITLE_MAX]
if long_titles:
problems.append(f"{len(long_titles)} titles are over {TITLE_MAX} characters")
dupe_locs = [u for u, c in Counter(i["loc"] for i in items).items() if c > 1]
if dupe_locs:
problems.append(f"{len(dupe_locs)} duplicate <loc> values")
if len(items) > 1000:
problems.append(f"{len(items)} entries — Google's limit is 1,000")
return problems, dated
The date bug
This is the part that cost me twenty minutes. My first version did the obvious thing:
datetime.fromisoformat(raw_date)
And it blew up:
ValueError: Invalid isoformat string: '2026-11-05T08:20:41.28+00:00'
Look closely at the fractional seconds: .28. Two digits. Before Python 3.11, fromisoformat only accepted exactly 3 or 6 digits there — it was written to round-trip isoformat() output, not to parse arbitrary ISO-8601. The sitemap was perfectly valid; the parser was picky.
If you're on 3.11+ you'll never see this. Plenty of us aren't, and a CI box on 3.9 failing on one article out of 178 is a miserable thing to debug. A tolerant parser is six lines:
def parse_date(raw):
if not raw:
return None
raw = raw.strip().replace("Z", "+00:00")
raw = re.sub(r"\.(\d{1,6})",
lambda m: "." + m.group(1).ljust(6, "0"),
raw, count=1)
try:
return datetime.fromisoformat(raw)
except ValueError:
return None
Pad the fraction to six digits, normalise a trailing Z, and return None instead of raising so one malformed entry doesn't take the whole run down.
Running it
Against a live news sitemap — Peepals, which publishes across seven sections and is a convenient test subject because it refreshes several times a day:
$ python news_sitemap_audit.py https://peepalsglobal.com/news-sitemap.xml
https://peepalsglobal.com/news-sitemap.xml
178 entries
window covered : 44.2h
title length : 48-88 chars
keywords : 5.01 per article, 866 distinct
No problems found.
44.2 hours of coverage, comfortably inside the 48-hour window, and 178 entries against a 1,000 limit. That's what a healthy file looks like.
The summary line is there on purpose. "No problems found" tells you nothing about whether the sitemap is actually doing its job, but 44 hours, 178 entries, titles 48–88 characters tells you the publishing pipeline is alive and the window is being respected.
One number did surprise me: 866 distinct keyword tags across 178 articles, averaging 5.01 per article. Near-zero reuse. Whether <news:keywords> still influences anything is doubtful — Google deprecated the meta keywords equivalent years ago — but it's a cheap signal that each article is being tagged individually rather than by template.
Now the failure case. Point it at a regular sitemap by mistake, which is exactly the kind of config slip that goes unnoticed for months:
$ python news_sitemap_audit.py https://peepalsglobal.com/sitemap-main.xml
https://peepalsglobal.com/sitemap-main.xml
3824 entries
PROBLEMS
- 3824 <url> entries have no <news:news> block
- 3824 entries — Google's limit for a news sitemap is 1,000
Both real problems, caught in one run.
Wiring it into CI
The script exits 1 when it finds problems, so it drops straight into a pipeline:
- name: Audit news sitemap
run: python news_sitemap_audit.py https://example.com/news-sitemap.xml
A word of warning from experience: run it after deploy, not before. The sitemap you want to check is the one your readers and Googlebot actually get, not the one in your build directory. And if your site is behind a CDN, add a cache-busting query string or you'll audit a stale copy for an hour.
What I'd add next
-
Fetch each
<loc>and compare the<news:title>to the<title>on the page. Mismatches between sitemap and page are a common cause of articles not appearing. - Check the canonical. An article whose canonical points elsewhere shouldn't be in the news sitemap at all.
- Track the window over time. If coverage drops from 44 hours to 6, something upstream broke — probably worth an alert before anyone notices traffic falling.
The whole thing is stdlib, no dependencies, and it runs in under a second. If you're running a news site and you've never actually looked at what's in that file, it's worth the ten minutes.
Written by Aniket Patil. I write about India's news, technology and markets at Peepals — our technology section is where most of this kind of thing ends up.
Top comments (1)
The two digit fractional seconds catch is a good one. The same parser quirk shows up in a lot of CI boxes pinned to older Python, and returning None instead of raising is the right call for an audit script. One check worth adding next to the 48 hour window: compare publication_date against the article's own datePublished in its JSON-LD. When the sitemap timestamp and the on-page date drift apart, that is often the reason a fresh story is skipped for Top stories.