Build a Keyword Intent Classifier from SERP Page Patterns

Keyword intent is easy to guess and hard to verify.

Two keywords can look similar in a spreadsheet, but their result pages can tell very different stories:

• one SERP is mostly product pages
• one SERP is mostly how-to guides
• one SERP is full of comparison pages
• one SERP has local packs or marketplace pages
• one SERP has documentation and developer tools

If you classify intent from keyword text alone, you are often classifying the phrase you see, not the demand Google is showing.

This post walks through a lightweight keyword intent classifier that uses SERP page patterns as input. A SERP API such as TalorData SERP API can provide structured result data; the classifier logic stays in your own code.

The classifier idea

Instead of asking what a keyword sounds like, ask what kind of pages are ranking for it.

For example:

• "best proxy API" may show review pages, vendor pages, and comparison posts
• "how to scrape SERP results" may show tutorials and documentation
• "serp api pricing" may show pricing pages and vendor pages
• "keyword rank tracker" may show product pages and listicles

The result page gives you a second signal.

A simple label set

Keep the first version small:

• informational
• commercial
• transactional
• navigational
• mixed
• unclear

You can always split labels later. A smaller label set makes review easier.

Features to extract from the SERP

Start with signals that are visible in ordinary organic results:

• title words
• URL path words
• result position
• repeated page formats
• common modifiers such as "best", "review", "pricing", "tutorial", "docs"

You do not need a large model for the first pass. A transparent rule system is often easier to debug.

Example input

This example uses a normalized SERP shape. Adjust the fields to match your provider response.

{
  "query": "serp api pricing",
  "results": [
    {
      "position": 1,
      "title": "SERP API Pricing",
      "url": "https://example.com/pricing",
      "snippet": "Compare plans for search result data access."
    },
    {
      "position": 2,
      "title": "Best SERP APIs Compared",
      "url": "https://example.com/blog/best-serp-apis",
      "snippet": "A comparison of APIs for search result collection."
    }
  ]
}

Build a small feature extractor

import re
from collections import Counter
from urllib.parse import urlparse

COMMERCIAL_TERMS = {"best", "compare", "comparison", "review", "alternative", "pricing", "plans"}
TRANSACTIONAL_TERMS = {"buy", "signup", "trial", "demo", "pricing", "checkout"}
INFORMATIONAL_TERMS = {"how", "guide", "tutorial", "what", "why", "docs", "documentation", "example"}


def tokenize(text: str) -> set[str]:
    return set(re.findall(r"[a-z0-9-]+", text.lower()))


def path_tokens(url: str) -> set[str]:
    path = urlparse(url).path.replace("-", " ").replace("_", " ")
    return tokenize(path)


def score_result(result: dict) -> Counter:
    text = " ".join([result.get("title", ""), result.get("snippet", "")])
    tokens = tokenize(text) | path_tokens(result.get("url", ""))

    scores = Counter()
    scores["commercial"] += len(tokens & COMMERCIAL_TERMS)
    scores["transactional"] += len(tokens & TRANSACTIONAL_TERMS)
    scores["informational"] += len(tokens & INFORMATIONAL_TERMS)
    return scores

This is not meant to be perfect. It is meant to be readable enough that your SEO or content team can challenge the rules.

Aggregate scores across the SERP

Top-ranking pages should usually count more than lower results.

def classify_serp(serp: dict) -> dict:
    total = Counter()

    for result in serp["results"][:10]:
        position = result.get("position", 10)
        weight = max(1, 11 - position)
        total.update({
            intent: score * weight
            for intent, score in score_result(result).items()
        })

    if not total:
        return {"label": "unclear", "scores": {}, "reason": "No intent signals found"}

    top_label, top_score = total.most_common(1)[0]
    second_score = total.most_common(2)[1][1] if len(total) > 1 else 0

    if top_score == 0:
        label = "unclear"
    elif second_score and top_score / second_score < 1.4:
        label = "mixed"
    else:
        label = top_label

    return {"label": label, "scores": dict(total), "reason": f"Top SERP patterns leaned {label}"}

Add page-type hints

You can improve the first pass by detecting page types:

def page_type(url: str, title: str) -> str:
    text = f"{url} {title}".lower()

    if "/pricing" in text or "pricing" in title.lower():
        return "pricing_page"
    if "/docs" in text or "documentation" in title.lower():
        return "documentation"
    if "best" in title.lower() or "compare" in title.lower():
        return "comparison"
    if "how to" in title.lower() or "guide" in title.lower():
        return "guide"
    return "unknown"

For many keywords, page-type counts are easier to explain than raw text matching.

Keep a review queue

Do not auto-accept every label.

Send these cases to review:

• mixed
• unclear
• low score difference between top labels
• high-value keywords
• keywords where SERP page types changed since the last crawl

The review queue matters because intent classification is a content decision, not just a parsing task.

Store evidence with each label

When you save the label, also save:

• query
• location
• capture date
• top result titles
• top result URLs
• detected page types
• classifier scores
• reviewer override, if any

That evidence is useful when someone asks why a keyword was labeled commercial instead of informational.

Where the SERP API fits

The SERP API is the collection layer. The classifier is your interpretation layer.

One possible workflow:

1. Pull SERP results for a keyword list
2. Normalize titles, URLs, snippets, and result positions
3. Extract page-pattern signals
4. Assign a first-pass intent label
5. Queue uncertain keywords for review
6. Save the final label with SERP evidence

If you need structured search result data for this kind of workflow, TalorData can support the SERP collection step while your classifier owns the labeling logic.

What I would not automate

I would avoid fully automating final labels for strategic pages, content format decisions, or search intent changes after major SERP shifts.

A classifier should reduce spreadsheet work. It should not hide judgment.

Final pattern

Treat SERP patterns as evidence, not truth.

The useful version of this classifier is not the one that claims perfect accuracy. It is the one that gives your team a repeatable starting point and shows enough evidence for a human to correct it.

If you have built something similar, I would be curious how you separate commercial, informational, and mixed intent in your own workflow.