Compare Competitor Domains Across SERP Snapshots with Python

A competitor list is usually too static.

Search results are not.

If you track a set of keywords every week, you can learn which domains keep appearing in front of your audience. Some will be direct competitors. Some will be directories, review sites, forums, media pages, or content hubs that influence the buying journey.

This post shows a small Python pattern for comparing competitor domains across SERP snapshots.

The collection layer can be a SERP API such as TalorData SERP API. The analysis layer is a simple domain frequency report.

The goal

Given multiple SERP snapshots, we want to answer:

• Which domains appear most often?
• Which domains appear for the highest-priority keywords?
• Which domains are new this week?
• Which domains should be reviewed by the SEO or growth team?

This is not a full competitive intelligence system. It is a lightweight way to turn search results into a review list.

Example snapshot shape

Assume each snapshot contains one keyword and its top organic results:

{
  "keyword": "serp api for seo monitoring",
  "checked_at": "2026-07-08T09:00:00Z",
  "results": [
    {
      "position": 1,
      "title": "Example result title",
      "url": "https://example.com/page"
    }
  ]
}

The exact response depends on your SERP API provider. Normalize the data before building the competitor report.

Extract domains from URLs

Start with a small helper:

from urllib.parse import urlparse


def domain_from_url(url: str) -> str:
    parsed = urlparse(url)
    return parsed.netloc.lower().replace("www.", "")

This is intentionally simple. If you need production-level handling of subdomains and public suffixes, add a proper domain parsing library later.

Load snapshots and count domains

import json
from collections import Counter, defaultdict
from pathlib import Path
from urllib.parse import urlparse


SNAPSHOT_DIR = Path("serp_snapshots")


def domain_from_url(url: str) -> str:
    return urlparse(url).netloc.lower().replace("www.", "")


def load_snapshots() -> list[dict]:
    snapshots = []
    for path in SNAPSHOT_DIR.glob("*.json"):
        with path.open("r", encoding="utf-8") as file:
            snapshots.append(json.load(file))
    return snapshots


def count_domains(snapshots: list[dict]) -> Counter:
    counter = Counter()

    for snapshot in snapshots:
        for result in snapshot.get("results", []):
            url = result.get("url")
            if not url:
                continue
            counter[domain_from_url(url)] += 1

    return counter


if __name__ == "__main__":
    snapshots = load_snapshots()
    domain_counts = count_domains(snapshots)

    for domain, count in domain_counts.most_common(20):
        print(f"{domain}: {count}")

This gives you the most visible domains across your tracked keyword set.

Add keyword context

Domain frequency alone can be misleading.

A domain appearing once for a high-intent keyword may matter more than a domain appearing five times for low-priority research terms.

Add keyword-level context:

def domain_keyword_map(snapshots: list[dict]) -> dict[str, set[str]]:
    mapping = defaultdict(set)

    for snapshot in snapshots:
        keyword = snapshot.get("keyword")
        for result in snapshot.get("results", []):
            url = result.get("url")
            if keyword and url:
                mapping[domain_from_url(url)].add(keyword)

    return mapping

Now your report can show:

example.com: appeared for 8 keywords
  - serp api for seo monitoring
  - search data api
  - competitor serp tracking

That is easier to review than a raw count.

Compare this week with last week

To detect new competitor domains, compare two sets:

def domains_in_snapshots(snapshots: list[dict]) -> set[str]:
    domains = set()
    for snapshot in snapshots:
        for result in snapshot.get("results", []):
            url = result.get("url")
            if url:
                domains.add(domain_from_url(url))
    return domains


def new_domains(previous_week: list[dict], current_week: list[dict]) -> set[str]:
    previous_domains = domains_in_snapshots(previous_week)
    current_domains = domains_in_snapshots(current_week)
    return current_domains - previous_domains

A weekly report can then include:

• most frequent domains
• new domains this week
• domains appearing for priority keywords
• domains that need manual classification

Do not label everything as a competitor

This is important.

Not every visible domain is a direct competitor.

You may find:

• direct product competitors
• affiliate lists
• review sites
• marketplaces
• documentation pages
• forums
• media articles
• comparison pages
• unrelated pages that need filtering

The first report should classify domains manually before the automation starts making conclusions.

Where the SERP API fits

The SERP API collects the search result snapshots. The domain analysis turns those snapshots into a competitor review list.

For workflows that need structured search result data, TalorData can provide the collection layer. Your code can own the domain grouping, classification rules, and weekly review format.

Final note

If you monitor competitors through SERPs, do you group by exact URL, domain, subdomain, or page type first?