Diff two Google SERPs in Python without mistaking churn for rank movement

Your monitor says four positions changed overnight for “espresso scale.” A merchandising team wants to know which competitor appeared. An analyst wants to know which existing URLs moved. A diff built around list indices gives both teams a pile of changed rows.

Use the URL as the identity and treat the position as a property of that URL. Then you can report the new page, the departed page, and the moves as separate events from the same pair of result sets.

Pick a depth both snapshots cover

Serper.live returns url and position in organic[] plus the effective query, country, interface language, and device on the response. Store the requested limit beside the payload. Compare the same context and check that both snapshots contain every organic position through your chosen depth. If a top-five response contains four rows, leave that interval out of the automated comparison and inspect the coverage. Serper.live search response contract

This example uses exact returned URLs. If your feed exposes tracking variants or redirects, validate a URL normalization rule against actual pairs before using it to collapse identities. Keep the raw URL for inspection.

Diff the URLs and positions

Save this as serp_diff.py and run python3 serp_diff.py with Python 3.11 or later. The fixture includes the fields used by the diff. In your collector, pass the stored API response and the limit originally requested.

from datetime import datetime

CONTEXT = ("query", "country", "language", "device")


def top_urls(response, depth):
    rows = [row for row in response["organic"] if row["position"] <= depth]
    if [row["position"] for row in rows] != list(range(1, depth + 1)):
        raise ValueError(f"Need a complete, ordered top {depth}")

    positions = {row["url"]: row["position"] for row in rows}
    if len(positions) != depth:
        raise ValueError("Duplicate URL in the comparison window")
    return positions


def serp_diff(before, after, depth):
    old = before["response"]
    new = after["response"]
    if any(old[field] != new[field] for field in CONTEXT):
        raise ValueError("The search contexts differ")
    if old["requestId"] == new["requestId"]:
        raise ValueError("The snapshots refer to the same request")
    if datetime.fromisoformat(old["fetchedAt"]) >= datetime.fromisoformat(new["fetchedAt"]):
        raise ValueError("Pass snapshots in chronological order")
    if not 1 <= depth <= min(before["requested_limit"], after["requested_limit"]):
        raise ValueError("The requested limits must cover the comparison depth")

    previous = top_urls(old, depth)
    current = top_urls(new, depth)
    return {
        "entered": [(url, rank) for url, rank in current.items() if url not in previous],
        "left": [(url, rank) for url, rank in previous.items() if url not in current],
        "moved": [
            (url, previous[url], rank)
            for url, rank in current.items()
            if url in previous and previous[url] != rank
        ],
    }


def snapshot(day, urls):
    return {
        "requested_limit": 5,
        "response": {
            "requestId": f"fixture-{day}",
            "fetchedAt": f"2026-10-{day:02d}T09:00:00Z",
            "query": "espresso scale",
            "country": "US",
            "language": "en",
            "device": "mobile",
            "organic": [
                {"position": rank, "url": f"https://{url}"}
                for rank, url in enumerate(urls, 1)
            ],
        },
    }


before = snapshot(5, [
    "review.example/scale-review",
    "store.example/scale",
    "roundup.example/best-scales",
    "barista.example/guide",
    "compare.example/scales",
])
after = snapshot(6, [
    "store.example/scale",
    "lab.example/timer-scales",
    "roundup.example/best-scales",
    "compare.example/scales",
    "barista.example/guide",
])

for event, rows in serp_diff(before, after, depth=5).items():
    print(event, rows)

entered [('https://lab.example/timer-scales', 2)]
left [('https://review.example/scale-review', 1)]
moved [('https://store.example/scale', 2, 1), ('https://compare.example/scales', 5, 4), ('https://barista.example/guide', 4, 5)]

The review left the observed top five. Four URLs stayed, including one that held rank 3. Put the comparison depth and snapshot IDs in the event you store so a teammate can reopen the evidence and see exactly which window changed.

For a scheduled monitor, diff each new observation against the previous complete one in the same context. If the previous scheduled run had incomplete data, show the older comparison timestamp alongside the event. “Changed since Monday's complete check” gives an operator a usable interval.

This tutorial was drafted with AI assistance. The two SERPs are local fixtures.