DEV Community: Elowen

Turn SERP Results into a Competitor Market Map by Domain Category

Elowen — Tue, 11 Aug 2026 03:20:31 +0000

A competitor list tells you who you already know. A SERP market map can show which types of sites repeatedly control discovery across a topic: product companies, publishers, directories, communities, marketplaces, or review sites.

The useful output is not another ranking export. It is a table that answers:

Which domains appear across multiple related queries?
Which domain categories occupy the most search visibility?
Which sites rank broadly, and which only own one narrow intent?
Where is the market crowded with direct vendors, and where do intermediaries dominate?

This walkthrough builds that output from several SERPs, rather than treating one keyword as the whole market.

1. Define a query set around buyer tasks

Start with queries that represent different ways someone explores the market. For a search data category, that might look like this:

QUERIES = [
    "serp api",
    "google search api",
    "serp data for seo tools",
    "search api for ai agents",
    "competitor rank tracking api",
]

The query set matters more than the size of the script. If every query has the same intent, the map will exaggerate one part of the market. Include category discovery, use-case, comparison, and implementation queries when they are relevant.

2. Collect structured organic results

The function below sends each query to TalorData SERP API and keeps the fields needed for analysis.

from urllib.parse import urlparse
import requests

ENDPOINT = "https://serpapi.talordata.net/serp/v1/request"


def normalize_domain(url):
    host = urlparse(url).netloc.lower()
    return host.removeprefix("www.")


def fetch_results(query):
    response = requests.post(
        ENDPOINT,
        headers={
            "Authorization": "Bearer <TALORDATA_TOKEN>",
            "Content-Type": "application/x-www-form-urlencoded",
        },
        data={
            "engine": "google",
            "q": query,
            "gl": "us",
            "hl": "en",
            "device": "desktop",
            "json": "2",
        },
        timeout=60,
    )
    response.raise_for_status()

    rows = []
    for result in response.json().get("organic", []):
        url = result.get("link") or result.get("url") or ""
        if not url:
            continue
        rows.append({
            "query": query,
            "position": result.get("position"),
            "domain": normalize_domain(url),
            "title": result.get("title", ""),
            "url": url,
        })
    return rows

Keep the country, language, device, and collection date consistent. Otherwise, changes in search context can look like differences in market structure.

3. Classify domains by market role

Domain classification needs judgment. A transparent lookup table is a good starting point because a reviewer can correct it.

CATEGORY_BY_DOMAIN = {
    "example-vendor.com": "direct_vendor",
    "example-directory.com": "directory",
    "example-media.com": "publisher",
    "example-community.com": "community",
}


def classify_domain(domain):
    return CATEGORY_BY_DOMAIN.get(domain, "unclassified")

Do not silently force every unknown domain into a convenient category. Keep unclassified visible, review the most frequent unknowns, and update the lookup table. That makes the map auditable instead of pretending the categories are objective facts.

4. Aggregate visibility across queries

A simple position-weighted score gives higher-ranked results more influence while still rewarding repeated appearances.

from collections import defaultdict


def position_weight(position):
    if not isinstance(position, int) or position < 1:
        return 0
    return 1 / position


all_rows = []
for query in QUERIES:
    all_rows.extend(fetch_results(query))

domain_stats = defaultdict(lambda: {
    "appearances": 0,
    "queries": set(),
    "visibility_score": 0.0,
})

for row in all_rows:
    stats = domain_stats[row["domain"]]
    stats["appearances"] += 1
    stats["queries"].add(row["query"])
    stats["visibility_score"] += position_weight(row["position"])

market_map = []
for domain, stats in domain_stats.items():
    market_map.append({
        "domain": domain,
        "category": classify_domain(domain),
        "appearances": stats["appearances"],
        "query_coverage": len(stats["queries"]),
        "visibility_score": round(stats["visibility_score"], 3),
    })

market_map.sort(
    key=lambda row: (row["query_coverage"], row["visibility_score"]),
    reverse=True,
)

This is not a universal visibility metric. It is a practical comparison score for one controlled query set. Keep the formula documented so future snapshots remain comparable.

5. Produce two review tables

The domain table shows recurring sites:

domain,category,appearances,query_coverage,visibility_score
example-vendor.com,direct_vendor,4,3,1.583
example-media.com,publisher,3,3,0.867
example-directory.com,directory,2,2,0.750

Then aggregate the same rows by category:

category_stats = defaultdict(lambda: {
    "domains": set(),
    "appearances": 0,
    "visibility_score": 0.0,
})

for row in market_map:
    stats = category_stats[row["category"]]
    stats["domains"].add(row["domain"])
    stats["appearances"] += row["appearances"]
    stats["visibility_score"] += row["visibility_score"]

category_map = [
    {
        "category": category,
        "domain_count": len(stats["domains"]),
        "appearances": stats["appearances"],
        "visibility_score": round(stats["visibility_score"], 3),
    }
    for category, stats in category_stats.items()
]

The category table reveals whether discovery is controlled mainly by vendors or by intermediaries such as publishers and directories. That distinction can change your content and partnership strategy.

6. Add the limits to the report

A SERP market map is a search snapshot, not a complete company database. Record the query set, collection date, location, language, device, category definitions, and scoring formula beside the output. Also keep the original URLs so someone can inspect why a domain was included.

For a first pass, TalorData SERP API provides structured Google results that can feed this workflow. New accounts include 500 responses, which can support a focused multi-query map before you expand the query set.

The main shift is simple: stop reading the SERP as ten isolated rankings. Aggregate related searches, classify the domains by role, and turn repeated visibility into a market structure you can review.

Evaluate a SERP API with a Practical Accuracy, Latency, and Cost Checklist

Elowen — Mon, 10 Aug 2026 01:45:36 +0000

Choosing a SERP API is usually framed as a feature comparison: supported engines, locations, pricing, response format, and maybe JavaScript rendering. Those things matter, but they are not enough if the API will run inside a production SEO workflow, AI agent, market research pipeline, or competitor monitoring system.

A more useful evaluation looks at three things together:

accuracy: does the returned SERP match the search context you asked for?
latency: is the response time predictable enough for your workflow?
cost: how many useful outputs do you get per request, retry, and stored snapshot?

This post outlines a practical checklist and a small test harness you can adapt before committing to a SERP data provider.

Start with fixed test cases

Do not evaluate an API with random one-off queries. Build a small test matrix that reflects your real use case.

TEST_CASES = [
    {
        "name": "commercial_us_desktop",
        "q": "serp api pricing",
        "gl": "us",
        "hl": "en",
        "device": "desktop",
        "location": "United States",
    },
    {
        "name": "local_mobile",
        "q": "emergency plumber near me",
        "gl": "us",
        "hl": "en",
        "device": "mobile",
        "location": "Austin, Texas, United States",
    },
    {
        "name": "informational_ai_search",
        "q": "ai search visibility monitoring",
        "gl": "us",
        "hl": "en",
        "device": "desktop",
        "location": "United States",
    },
]

A good test matrix should include the types of searches your workflow actually needs: local, commercial, informational, mobile, desktop, and any country or language settings that matter.

Make the request measurable

For a SERP API test, store the request parameters, response time, and basic response shape for every run. Here is a minimal example using TalorData SERP API.

import os
import time
import requests

ENDPOINT = "https://serpapi.talordata.net/serp/v1/request"
TOKEN = os.environ["TALORDATA_TOKEN"]


def run_serp_request(case):
    payload = {
        "engine": "google",
        "q": case["q"],
        "gl": case["gl"],
        "hl": case["hl"],
        "device": case["device"],
        "location": case["location"],
        "json": "2",
    }

    started = time.perf_counter()
    response = requests.post(
        ENDPOINT,
        headers={
            "Authorization": "Bearer <TALORDATA_TOKEN>",
            "Content-Type": "application/x-www-form-urlencoded",
        },
        data=payload,
        timeout=60,
    )
    elapsed_ms = round((time.perf_counter() - started) * 1000)

    data = response.json()

    return {
        "case_name": case["name"],
        "q": case["q"],
        "gl": case["gl"],
        "hl": case["hl"],
        "device": case["device"],
        "location": case["location"],
        "status_code": response.status_code,
        "elapsed_ms": elapsed_ms,
        "organic_count": len(data.get("organic", [])),
        "has_paa": bool(data.get("people_also_ask")),
        "has_knowledge": bool(data.get("knowledge")),
        "request_params": data.get("request_params", {}),
        "search_metadata": data.get("search_metadata", {}),
    }

In a real script, read the token from an environment variable or secret manager. Keep <TALORDATA_TOKEN> as a placeholder in shared examples.

Accuracy checklist

Accuracy is not just whether the API returns JSON. Check whether the result matches the search context.

Use a checklist like this:

Accuracy checks
- Does the response include organic results for normal queries?
- Are location, country, language, and device parameters stored or echoed back?
- Do top domains look plausible for the query intent?
- Does a local query produce local-looking results?
- Are expected SERP features present when relevant, such as people_also_ask or knowledge?
- Are result URLs, titles, positions, and snippets stable enough to review?
- Can the same request be repeated later with the same parameters?

For accuracy testing, do not rely only on a single query. One strange SERP does not prove the provider is wrong, and one clean SERP does not prove the provider is right. Look for patterns across the test matrix.

Latency checklist

Latency should be measured in the workflow where the API will be used. A dashboard, batch report, and real-time agent do not have the same tolerance.

Latency checks
- Median response time across the test matrix
- Slowest response time in the test run
- Difference between local and broad queries
- Difference between desktop and mobile requests
- Timeout behavior in your client code
- Whether retries are needed for your workflow
- Whether the workflow can run asynchronously

For batch SEO reporting, a slower request may be acceptable if the output is reliable. For an agent that answers a user in real time, predictable response time may matter more.

Cost checklist

The lowest request price is not always the lowest workflow cost. Cost should include failed attempts, retries, duplicate checks, and the number of requests required to produce one useful output.

Cost checks
- Requests needed per report
- Requests needed per monitored keyword-location pair
- Retry rate during normal runs
- Duplicate calls caused by missing caching or poor scheduling
- Storage cost for keeping snapshots
- Human review time saved by structured output
- Cost per useful row, not only cost per request

For example, a local SEO monitor may need keywords x locations x devices requests per snapshot. A content gap workflow may need requests across many related queries. A RAG or agent workflow may only need a few searches, but latency and relevance may be more important.

Build an evaluation table

After running the test matrix, reduce the output into a review table.

case_name
query
location
device
status_code
elapsed_ms
organic_count
has_paa
has_knowledge
accuracy_note
latency_note
cost_note
pass_for_workflow

A row might look like this:

case_name: local_mobile
query: emergency plumber near me
location: Austin, Texas, United States
device: mobile
status_code: 200
elapsed_ms: 1840
organic_count: 9
has_paa: true
accuracy_note: local service and directory results visible
latency_note: acceptable for batch monitoring
cost_note: one request per keyword-location-device snapshot
pass_for_workflow: yes for weekly local SEO report

This format is easier to discuss than a generic vendor comparison. It ties the API evaluation to the workflow you actually need to run.

A practical conclusion

A SERP API should be evaluated by the quality of the workflow it enables. If the data is structured, the request context is preserved, the latency fits the use case, and the cost model still makes sense after retries and snapshots, then the API is much easier to trust in production.

If you want to test this kind of checklist with structured Google SERP data, TalorData SERP API can be used with parameters such as q, location, gl, hl, device, and json=2. New accounts can use 500 included responses to run a small evaluation matrix before expanding the test.

Build a People Also Ask Research Workflow with SERP API Data

Elowen — Sat, 08 Aug 2026 03:41:12 +0000

People Also Ask data is easy to screenshot and hard to reuse. A few questions in a browser tab can help with brainstorming, but they do not become a reliable research asset until they are collected, normalized, deduplicated, and grouped by intent.

This walkthrough shows a small workflow for turning People Also Ask results into a content research table. The output is a list of user questions with source query, location, intent group, and review notes.

The output we want

The final table should answer these questions:

Which user questions appear across several related searches?
Which questions are informational, commercial, comparative, or troubleshooting-oriented?
Which questions already map to existing content?
Which questions deserve a new section, FAQ, article, or product page update?

A minimal output can look like this:

question
normalized_question
source_query
location
intent_group
content_action
notes

The important part is that the question is not just copied into a document. It is attached to enough context for later review.

Request SERP data with PAA included

TalorData SERP API can return structured Google SERP data, including people_also_ask. A request can be made with the same SERP endpoint used for other Google result checks.

curl -X POST 'https://serpapi.talordata.net/serp/v1/request' \
  -H 'Authorization: Bearer <TALORDATA_TOKEN>' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'engine=google' \
  -d 'q=serp api pricing' \
  -d 'gl=us' \
  -d 'hl=en' \
  -d 'device=desktop' \
  -d 'json=2'

For research, run this across a cluster of related queries instead of a single keyword. A single SERP can be noisy, but repeated patterns across multiple SERPs are usually more useful.

Start with a query set

QUERIES = [
    "serp api pricing",
    "google search api for seo",
    "serp api for ai agents",
    "search results api comparison",
]

CONTEXT = {
    "gl": "us",
    "hl": "en",
    "device": "desktop",
    "location": "United States",
}

Keep this query set focused. If the set mixes unrelated topics, the deduped PAA list will become difficult to interpret.

Extract and normalize questions

Different response shapes can be handled defensively. The goal is to collect question-like text from the people_also_ask section without assuming every item has the same shape.

import re


def normalize_question(text: str) -> str:
    text = text.strip().lower()
    text = re.sub(r"\s+", " ", text)
    text = re.sub(r"[?!.]+$", "", text)
    return text


def extract_paa_questions(response_json, source_query, context):
    rows = []

    for item in response_json.get("people_also_ask", []):
        question = item.get("question") or item.get("title") or ""
        question = question.strip()
        if not question:
            continue

        rows.append({
            "question": question,
            "normalized_question": normalize_question(question),
            "source_query": source_query,
            "location": context.get("location", ""),
            "gl": context.get("gl", ""),
            "hl": context.get("hl", ""),
            "device": context.get("device", ""),
        })

    return rows

This keeps the extraction step small. Do not assign intent yet. First collect the evidence.

Deduplicate across queries

A question may appear with small wording differences. Start with exact normalization, then review close duplicates manually or with a separate similarity step.

def dedupe_questions(rows):
    grouped = {}

    for row in rows:
        key = row["normalized_question"]
        if key not in grouped:
            grouped[key] = {
                "question": row["question"],
                "normalized_question": key,
                "source_queries": set(),
                "locations": set(),
            }

        grouped[key]["source_queries"].add(row["source_query"])
        grouped[key]["locations"].add(row["location"])

    output = []
    for item in grouped.values():
        output.append({
            "question": item["question"],
            "normalized_question": item["normalized_question"],
            "source_queries": ", ".join(sorted(item["source_queries"])),
            "locations": ", ".join(sorted(item["locations"])),
            "query_count": len(item["source_queries"]),
        })

    return sorted(output, key=lambda row: row["query_count"], reverse=True)

query_count is useful because repeated questions are usually better candidates for content review than one-off questions.

Add intent groups

A lightweight intent classifier is enough for the first pass. It does not need to be perfect; it needs to make the review faster.

def classify_intent(question: str) -> str:
    q = question.lower()

    if any(term in q for term in ["pricing", "cost", "free", "paid"]):
        return "commercial"
    if any(term in q for term in ["vs", "compare", "alternative", "best"]):
        return "comparison"
    if any(term in q for term in ["how", "what is", "why", "guide"]):
        return "informational"
    if any(term in q for term in ["not working", "error", "fix", "limit"]):
        return "troubleshooting"

    return "mixed_or_review"

This is deliberately simple. The output should be reviewed by a human before it becomes a content plan.

Map questions to content actions

The final step is to decide what to do with each question.

def suggest_action(intent_group, query_count):
    if query_count >= 3 and intent_group == "commercial":
        return "review product or pricing page"
    if query_count >= 2 and intent_group == "comparison":
        return "consider comparison section or article"
    if intent_group == "informational":
        return "map to FAQ or educational article"
    if intent_group == "troubleshooting":
        return "map to docs or support content"
    return "manual review"

The goal is not to automate editorial judgment. The goal is to make the research table structured enough that editorial judgment can happen faster.

Example final row

Question: How do I get Google search results from an API?
Source queries: google search api for seo, serp api for ai agents
Query count: 2
Intent group: informational
Content action: map to FAQ or educational article
Note: Review whether existing docs explain authentication, parameters, and output fields clearly.

That row is much more useful than a raw PAA export. It tells the team why the question matters and where it might belong.

If you are building a PAA research workflow, TalorData SERP API can provide structured SERP data that includes people_also_ask. New accounts can use the included 500 responses to test a small query cluster before expanding the research workflow.

Create a Location-Aware SERP Monitor for Local SEO Campaigns

Elowen — Fri, 07 Aug 2026 03:55:31 +0000

A local SEO rank check becomes more useful when it keeps the location context attached to every result. Without that context, a report can say that a page ranks position 4 for a keyword, but it cannot explain where that position was observed, which market changed, or whether two locations are showing different competitors.

This post walks through a small location-aware SERP monitor that stores keyword, location, language, country, rank, URL, and snapshot time in one table. The goal is not to build a full SEO platform. The goal is to create a repeatable output that a local SEO team can inspect every week.

What the monitor should answer

For each keyword, the monitor should answer a few practical questions:

What does Google return for this keyword in each target location?
Which URL from our domain appears, if any?
What position was observed?
Which competitors are visible in the same location?
Did the result change compared with the previous snapshot?

The important part is that the location is not a note in a file name. It is a first-class field in the output.

Minimal table design

A useful local rank table can start with these columns:

snapshot_date
keyword
location_label
gl
hl
device
rank
result_title
result_url
result_domain
matched_own_domain
is_own_result
serp_top_domains
notes

This structure keeps the ranking evidence and the local context together. If the same keyword is checked for New York, Austin, and Toronto, those become three separate rows instead of one vague metric.

Requesting a location-aware SERP

TalorData SERP API accepts Google SERP parameters such as q, location, gl, hl, device, and json=2. A simple request can look like this:

curl -X POST 'https://serpapi.talordata.net/serp/v1/request' \
  -H 'Authorization: Bearer <TALORDATA_TOKEN>' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'engine=google' \
  -d 'q=emergency plumber near me' \
  -d 'location=Austin, Texas, United States' \
  -d 'gl=us' \
  -d 'hl=en' \
  -d 'device=desktop' \
  -d 'json=2'

If you want to build this into a script, keep the keyword and location list outside the request function. That makes it easier to add new cities without touching the API logic.

Example keyword and location config

KEYWORDS = [
    "emergency plumber near me",
    "water heater repair",
    "drain cleaning service",
]

LOCATIONS = [
    {
        "label": "Austin, TX",
        "location": "Austin, Texas, United States",
        "gl": "us",
        "hl": "en",
    },
    {
        "label": "Dallas, TX",
        "location": "Dallas, Texas, United States",
        "gl": "us",
        "hl": "en",
    },
    {
        "label": "Toronto, ON",
        "location": "Toronto, Ontario, Canada",
        "gl": "ca",
        "hl": "en",
    },
]

This format also makes the final report more readable. The report can show Austin, TX while the request still sends the full location value.

Normalize organic results

The SERP response can include an organic array. For monitoring, normalize it into flat rows before doing any comparison.

from urllib.parse import urlparse

OWN_DOMAIN = "example.com"


def domain_from_url(url: str) -> str:
    host = urlparse(url).netloc.lower()
    return host[4:] if host.startswith("www.") else host


def normalize_organic(snapshot_date, keyword, location_cfg, response_json):
    rows = []

    for item in response_json.get("organic", []):
        url = item.get("link") or item.get("url") or ""
        domain = domain_from_url(url) if url else ""
        position = item.get("position") or item.get("rank")

        rows.append({
            "snapshot_date": snapshot_date,
            "keyword": keyword,
            "location_label": location_cfg["label"],
            "gl": location_cfg["gl"],
            "hl": location_cfg["hl"],
            "rank": position,
            "result_title": item.get("title", ""),
            "result_url": url,
            "result_domain": domain,
            "matched_own_domain": OWN_DOMAIN,
            "is_own_result": domain == OWN_DOMAIN or domain.endswith("." + OWN_DOMAIN),
        })

    return rows

Keep this normalization boring. The monitor is easier to debug when every row represents one observed organic result in one location.

Reduce rows into a weekly monitor

Once you have normalized rows, you can produce a smaller monitoring table for stakeholders. For example, you may only need the first own-domain result and the top visible competitor domains.

def summarize_location(keyword, location_label, rows):
    location_rows = [
        row for row in rows
        if row["keyword"] == keyword and row["location_label"] == location_label
    ]

    own_rows = [row for row in location_rows if row["is_own_result"]]
    own_rows = sorted(own_rows, key=lambda row: row["rank"] or 999)

    top_domains = []
    for row in sorted(location_rows, key=lambda row: row["rank"] or 999)[:10]:
        domain = row["result_domain"]
        if domain and domain not in top_domains:
            top_domains.append(domain)

    first_own = own_rows[0] if own_rows else None

    return {
        "keyword": keyword,
        "location_label": location_label,
        "own_rank": first_own["rank"] if first_own else None,
        "own_url": first_own["result_url"] if first_own else "",
        "top_domains": ", ".join(top_domains[:5]),
        "note": "Own result not found in observed organic results" if not first_own else "",
    }

This gives you a compact output without losing the evidence. If a ranking drops in one city, you can go back to the normalized rows and inspect what changed.

Add change detection

A local SEO monitor becomes more useful when it compares the current snapshot with the previous one.

def compare_rank(previous_rank, current_rank):
    if previous_rank is None and current_rank is None:
        return "not_visible"
    if previous_rank is None:
        return "newly_visible"
    if current_rank is None:
        return "lost_visibility"
    if current_rank < previous_rank:
        return "improved"
    if current_rank > previous_rank:
        return "declined"
    return "unchanged"

The final report should not only say rank = 7. It should say something like declined from 3 to 7 in Austin, TX, or newly visible in Toronto, ON.

What to watch for

Location-aware monitoring has a few practical traps:

Do not mix desktop and mobile results in the same comparison.
Do not compare different countries without keeping gl visible.
Do not hide the observed location behind a generic market label.
Do not treat one city result as a national ranking.
Do not overwrite old snapshots; append new ones.

For local SEO, the audit trail matters. The team needs to know when a snapshot was taken, what parameters were used, and what result was actually observed.

Final output

A simple weekly output can look like this:

Date: 2026-08-07
Keyword: emergency plumber near me
Location: Austin, TX
Device: desktop
Own rank: 5
Own URL: https://example.com/austin/emergency-plumbing
Top visible domains: yelp.com, angi.com, example.com, localcompetitor.com
Change: declined from 3 to 5
Note: review local landing page title and competing directory pages

That is much more useful than a single average position. It tells the team where the observation came from and what to inspect next.

If you are building this kind of monitor, TalorData SERP API can provide structured Google SERP data for location-aware checks. New accounts can start with 500 responses, which is enough to test a small keyword-location matrix before expanding the monitor.

Build a SERP Content Gap Report with Rankings and Missing Topics

Elowen — Thu, 06 Aug 2026 03:40:42 +0000

A content gap report is more useful when it shows what is missing, where the evidence came from, and which URLs are already ranking.

A keyword list alone cannot do that. A ranking position alone cannot do that either. To make the report actionable, you need the SERP context around the query: titles, URLs, snippets, domains, and the topics competitors appear to cover.

This walkthrough shows a small pattern for building a SERP content gap report with TalorData SERP API.

What the report should answer

The first version should answer five questions:

Which pages rank for the query?
Which domains appear repeatedly?
What topics are visible in titles and snippets?
What topics are missing from our target page?
Which gaps are worth reviewing for a content update?

This is not a full content strategy platform. It is a structured research output that can support a human review.

Request the SERP data

Use the SERP API endpoint with a safe token placeholder in examples:

curl -X POST 'https://serpapi.talordata.net/serp/v1/request' \
  -H 'Authorization: Bearer <TALORDATA_TOKEN>' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'engine=google' \
  -d 'q=serp api content gap analysis' \
  -d 'num=10' \
  -d 'json=2'

The first field to inspect is organic.

Normalize organic results

A compact result row is easier to compare than a full response.

from urllib.parse import urlparse


def domain_from_url(url: str | None) -> str | None:
    if not url:
        return None
    return urlparse(url).netloc.replace("www.", "")


def normalize_organic(query: str, data: dict) -> list[dict]:
    rows = []

    for item in data.get("organic", []):
        link = item.get("link")
        rows.append(
            {
                "query": query,
                "position": item.get("position"),
                "title": item.get("title"),
                "url": link,
                "domain": domain_from_url(link),
                "snippet": item.get("description"),
            }
        )

    return rows

This gives the report a consistent base.

Add simple topic extraction

You do not need a complex NLP system for the first pass. Start with a controlled list of expected topics.

TOPICS = [
    "pricing",
    "api documentation",
    "integration",
    "examples",
    "accuracy",
    "locations",
    "ai overview",
    "people also ask",
]


def covered_topics(text: str) -> list[str]:
    lower = text.lower()
    return [topic for topic in TOPICS if topic in lower]

Apply it to the title and snippet:

def add_topic_coverage(rows: list[dict]) -> list[dict]:
    enriched = []

    for row in rows:
        text = f"{row.get('title') or ''} {row.get('snippet') or ''}"
        enriched.append(
            {
                **row,
                "visible_topics": covered_topics(text),
            }
        )

    return enriched

This is intentionally simple. It creates a reviewable first report instead of a black-box score.

Compare against your target page

The report becomes useful when you compare SERP patterns with your own page.

Create a target page record:

target_page = {
    "url": "https://example.com/serp-api",
    "covered_topics": ["pricing", "api documentation", "examples"],
}

Then compute missing topics:

def report_missing_topics(rows: list[dict], target_topics: list[str]) -> list[str]:
    serp_topics = set()

    for row in rows:
        serp_topics.update(row.get("visible_topics", []))

    return sorted(serp_topics - set(target_topics))

This creates a concrete review item: topics competitors visibly cover that your target page may not.

Create the report rows

A minimal report can include:

query
position
domain
url
title
visible_topics
gap_note

Example row:

{
  "query": "serp api content gap analysis",
  "position": 3,
  "domain": "example.com",
  "url": "https://example.com/page",
  "title": "SERP API Guide",
  "visible_topics": ["pricing", "examples"],
  "gap_note": "Competitor page visibly covers examples. Review whether target page covers this clearly."
}

The goal is not to let the script decide the final content update. The goal is to prepare evidence for review.

Final thought

A useful content gap report should not stop at rankings. It should show URLs, SERP context, visible topics, and a clear review path.

That makes the output easier for SEO, content, and product teams to use together.

If you want to test this with live Google results, TalorData gives new accounts 500 responses to build a small SERP content gap workflow before scaling it.

Handle Timeouts, Empty Results, and Retries in SERP API Workflows

Elowen — Wed, 05 Aug 2026 03:01:45 +0000

A SERP API workflow usually works well in a demo because the happy path is simple: send a query, get results, parse the response, move on.

The real workflow becomes more interesting when a request times out, the result is empty, or a retry creates duplicate output. If those states are not handled separately, the pipeline becomes hard to trust.

This post shows a small reliability pattern for workflows that call TalorData SERP API: use explicit timeouts, classify empty results, retry only when it makes sense, and log enough context for review.

The states to separate

Do not treat every non-happy path as the same failure.

At minimum, separate these states:

request succeeded with organic results
request succeeded but returned no organic results
request timed out
request failed before a usable response was returned
parsing failed after a response was received
workflow skipped the query intentionally

This separation matters because each state needs a different action.

An empty result may need review. A timeout may deserve a retry. A parsing issue probably needs a code fix. A skipped query should not be retried at all.

Basic request wrapper

Here is a small Python wrapper using requests.

import os
import time
from dataclasses import dataclass
from typing import Any

import requests


API_URL = "https://serpapi.talordata.net/serp/v1/request"


@dataclass
class SerpRunResult:
    status: str
    query: str
    attempts: int
    organic_count: int = 0
    data: dict[str, Any] | None = None
    note: str | None = None

Use a placeholder token in docs and examples. In a real app, read the token from an environment variable or secret manager.

def request_serp(query: str, timeout_seconds: int = 30) -> dict[str, Any]:
    token = os.environ["TALORDATA_TOKEN"]

    response = requests.post(
        API_URL,
        headers={
            "Authorization": f"Bearer {token}",
            "Content-Type": "application/x-www-form-urlencoded",
        },
        data={
            "engine": "google",
            "q": query,
            "num": "10",
            "json": "2",
        },
        timeout=timeout_seconds,
    )

    response.raise_for_status()
    return response.json()

If you are documenting the header directly, keep it safe:

Authorization: Bearer <TALORDATA_TOKEN>

Classify empty results

An empty result is not automatically the same as a failed request.

def classify_success(query: str, data: dict[str, Any], attempts: int) -> SerpRunResult:
    organic = data.get("organic", [])

    if not organic:
        return SerpRunResult(
            status="empty_result",
            query=query,
            attempts=attempts,
            organic_count=0,
            data=data,
            note="Request succeeded, but no organic results were available for this query.",
        )

    return SerpRunResult(
        status="succeeded",
        query=query,
        attempts=attempts,
        organic_count=len(organic),
        data=data,
    )

This distinction helps downstream nodes decide what to do next.

A report may show empty results as a review item. A retry queue may ignore them unless the query is business-critical.

Add limited retry with backoff

Retries should be limited and visible.

def fetch_with_retry(query: str, max_attempts: int = 3) -> SerpRunResult:
    last_note = None

    for attempt in range(1, max_attempts + 1):
        try:
            data = request_serp(query)
            return classify_success(query, data, attempt)
        except requests.Timeout:
            last_note = "Request timed out."
        except requests.RequestException:
            last_note = "Request failed before a usable response was returned."
        except ValueError:
            return SerpRunResult(
                status="parse_failed",
                query=query,
                attempts=attempt,
                note="Response was received, but JSON parsing failed.",
            )

        if attempt < max_attempts:
            time.sleep(2 ** (attempt - 1))

    return SerpRunResult(
        status="failed_after_retries",
        query=query,
        attempts=max_attempts,
        note=last_note,
    )

The key is not the exact backoff formula. The key is that retry behavior is bounded and recorded.

Decide what each status should do

A workflow becomes easier to maintain when each status has an action.

ACTIONS = {
    "succeeded": "store_results",
    "empty_result": "store_review_item",
    "parse_failed": "send_to_engineering_review",
    "failed_after_retries": "queue_for_later_review",
}

For example:

succeeded: normalize and store organic results
empty_result: store the query, timestamp, and request settings for review
parse_failed: inspect the parser and response shape
failed_after_retries: preserve the failed query without blocking the whole run

Log the context

A useful log record should include:

query
status
attempts
organic_count
timestamp
request settings
next action

If the workflow runs in batches, also include a run_id and query_id.

That makes it possible to answer later: did the workflow fail, return no results, or skip a query intentionally?

Final thought

Reliable search workflows do not retry everything blindly.

They classify what happened, retry only where retrying is useful, and preserve enough context for a human to inspect the edge cases.

If you want to test this reliability pattern with live Google results, TalorData gives new accounts 500 responses to build a small SERP API workflow before scaling it.

Make LangChain Web Search Auditable with Source URLs and Timestamps

Elowen — Tue, 04 Aug 2026 02:03:30 +0000

A LangChain agent with web search can answer with fresher context, but freshness is not enough.

When the answer looks wrong, you need to know what the agent searched, which URLs it used, when the search happened, and which results were passed into the final prompt. Without that trace, debugging becomes guesswork.

This walkthrough shows a small pattern for making a LangChain search step auditable with source URLs and timestamps using TalorData SERP API.

Basic setup

The confirmed LangChain package is langchain-talordata.

pip install langchain-talordata

Set the package environment variable:

export TALOR_API_KEY="<TALORDATA_TOKEN>"

A common starting point:

from langchain_talordata import TalorSerpTool

search_tool = TalorSerpTool.from_env()

The package reads TALOR_API_KEY. The value should be your TalorData token.

What should be logged

A useful search trace should answer five questions:

What did the user ask?
What query did the agent search?
When did the search happen?
Which URLs were returned?
Which URLs were selected for the final answer?

That means the log should preserve both the tool call and the selected evidence.

A simple trace shape

Start with a small structure:

from dataclasses import dataclass, asdict
from datetime import datetime, timezone
from typing import Any


@dataclass
class SearchTrace:
    user_request: str
    generated_query: str
    searched_at: str
    selected_sources: list[dict[str, Any]]
    result_count: int

The timestamp should be created when the search result is fetched, not after the final answer is generated.

def utc_now() -> str:
    return datetime.now(timezone.utc).isoformat()

Shape the search results

Do not pass the entire SERP response into the final prompt by default. Keep a compact source object.

def compact_source(item: dict) -> dict:
    return {
        "position": item.get("position"),
        "title": item.get("title"),
        "url": item.get("link"),
        "snippet": item.get("description"),
    }

For many agent tasks, the first useful source fields are position, title, URL, and snippet.

Build the search wrapper

Your exact wrapper depends on how your LangChain app calls tools, but the boundary should be clear:

def search_with_trace(user_request: str, generated_query: str) -> tuple[list[dict], dict]:
    searched_at = utc_now()

    result = search_tool.invoke(generated_query)

    organic = result.get("organic", []) if isinstance(result, dict) else []
    selected_sources = [compact_source(item) for item in organic[:5]]

    trace = SearchTrace(
        user_request=user_request,
        generated_query=generated_query,
        searched_at=searched_at,
        selected_sources=selected_sources,
        result_count=len(organic),
    )

    return selected_sources, asdict(trace)

If your tool call returns a different object shape, adapt the extraction layer. The key idea is the same: preserve the generated query, timestamp, and selected URLs.

Use the trace in the final answer

The final prompt should receive only selected sources, not the entire raw response.

Example prompt input:

User request:
{user_request}

Search query:
{generated_query}

Search timestamp:
{searched_at}

Selected sources:
{selected_sources}

Answer the user using the selected sources. If the sources are weak or incomplete, say so.

This gives the model a clearer boundary between evidence and reasoning.

Store the trace

For a prototype, writing JSON lines is enough:

import json


def append_trace(path: str, trace: dict) -> None:
    with open(path, "a", encoding="utf-8") as file:
        file.write(json.dumps(trace, ensure_ascii=False) + "\n")

In production, this trace may belong in a database, observability system, or internal evaluation table.

What this helps debug

A source trace helps answer:

Did the agent search the wrong query?
Did the SERP return weak results?
Did the source selection remove something important?
Did the answer rely on inference instead of evidence?
Was the answer generated from fresh or old search context?

That is much better than trying to inspect a final answer with no search record.

Final thought

Adding web search to an agent is useful. Making the search auditable is what makes it maintainable.

Start by logging the generated query, timestamp, selected URLs, and result count. That small trace will save a lot of debugging time later.

If you want to test this pattern with live Google results, TalorData gives new accounts 500 responses to build and inspect a small source-logging workflow.

Design a Reliable SERP Change-Detection Pipeline in n8n

Elowen — Mon, 03 Aug 2026 09:41:58 +0000

A SERP change-detection workflow should not be judged by how many alerts it can send.

It should be judged by whether the team can trust the alert, inspect the evidence, and decide what to do next.

That requires more than a scheduled API call. A reliable n8n pipeline needs a snapshot model, change categories, alert rules, failure handling, and a review path.

This article outlines a practical architecture using TalorData SERP API as the structured SERP data source.

The pipeline goal

The pipeline should answer five questions:

What was searched?
What did the SERP look like at that time?
What changed since the previous snapshot?
Which changes are important enough to review?
Can we audit why an alert fired?

If the workflow cannot answer those questions, it may still collect data, but it is not yet reliable monitoring.

Recommended n8n structure

A stable version can use this structure:

Schedule Trigger
Keyword Source
SERP API Request
Raw Snapshot Storage
Normalized Snapshot Storage
Previous Snapshot Lookup
Change Classifier
Alert Rule Evaluator
Notification Node
Review Log

The separation is important. Collection, classification, and alerting should not be collapsed into one opaque node.

Snapshot model

Store the raw response when possible, but make normalized records the main comparison layer.

A normalized organic result can contain:

snapshot_id
query_id
query
fetched_at
position
title
link
description

The snapshot_id groups all results from one run. The query_id keeps repeated checks tied to a stable monitored query.

This gives the pipeline a reliable comparison base.

Change categories

Do not treat all changes the same.

Useful categories include:

new_result
removed_result
position_changed
title_changed
description_changed

For many teams, new_result, removed_result, and meaningful position movement are the first useful categories.

Title and description changes are worth logging, but they may not always deserve immediate alerts.

Alert rules

Alert rules should be tied to action.

Examples:

a tracked domain disappears from the first page
a competitor enters the top five
a result moves more than five positions
several URLs change for the same query in one run
a monitored URL is replaced by a directory, forum, or comparison page

The rule should explain why the alert exists. If no one would act on it, it should probably be logged rather than sent.

Failure handling

Failure handling matters because a failed run can create false conclusions.

For example, if today's request fails and the workflow compares an empty result against yesterday's SERP, it may incorrectly report that every URL disappeared.

A reliable workflow should:

mark failed runs separately
avoid diffing against empty failed responses
keep retry count visible
preserve the previous valid snapshot
log request parameters for review

This protects the team from false alerts.

Review log

Every alert should create a reviewable record.

Useful fields:

alert_id
query_id
snapshot_id
change_type
previous_value
current_value
rule_triggered
created_at
review_status
review_note

This makes the workflow useful beyond the notification. A team can review which alert rules were helpful and which ones created noise.

Operational principle

Separate these three layers:

Detected changes
Meaningful alerts
Human decisions

Detected changes are data. Meaningful alerts are filtered signals. Human decisions are what the team does after review.

Keeping those layers separate makes the n8n pipeline easier to maintain.

TalorData SERP API can provide the structured Google results needed for this kind of pipeline. A new account includes 500 responses, which is enough to test snapshot storage, diff logic, and alert thresholds on a small keyword set.

Batch Google SERP Queries with n8n and Form Data

Elowen — Wed, 29 Jul 2026 02:46:54 +0000

A single SERP query is useful for debugging. A keyword list is where the workflow becomes useful for research.

If you have 20, 50, or 100 keywords, you do not want to paste each one into a search tool by hand. You want a repeatable workflow that accepts a list, runs each query, saves the result, and gives you a structured output for review.

This tutorial shows a simple n8n pattern: submit keywords through a form, loop through them, call TalorData SERP API, and return normalized SERP records.

Workflow shape

The first version has six parts:

Form Trigger
Code node to split keyword input
Split In Batches node
HTTP Request node
Code node to normalize organic results
Storage or output node

The goal is not to build a full research platform. The goal is to make batch SERP collection repeatable.

Step 1: Create a form input

Use an n8n Form Trigger or any form-like input source.

Create a textarea field:

keywords

Example input:

serp api
best serp api
google search api
ai search visibility

Keep one keyword per line. That makes parsing simple and reduces accidental duplicates.

Step 2: Split the keyword list

Add a Code node after the form.

const raw = $json.keywords || "";

const keywords = raw
  .split("\n")
  .map((keyword) => keyword.trim())
  .filter(Boolean);

const seen = new Set();
const uniqueKeywords = [];

for (const keyword of keywords) {
  const key = keyword.toLowerCase();
  if (seen.has(key)) continue;
  seen.add(key);
  uniqueKeywords.push(keyword);
}

return uniqueKeywords.map((keyword) => ({
  json: {
    keyword,
  },
}));

This gives n8n one item per keyword.

Step 3: Add Split In Batches

Add a Split In Batches node.

Start with a small batch size:

A batch size of one is slower, but it is easier to debug. Once the workflow is stable, you can adjust the cadence based on your operational needs.

Step 4: Add the HTTP Request node

Use a POST request.

URL:

https://serpapi.talordata.net/serp/v1/request

Headers:

Authorization: Bearer <TALORDATA_TOKEN>
Content-Type: application/x-www-form-urlencoded

Body parameters:

engine=google
q={{ $json.keyword }}
num=10
json=2

This returns structured search data for each keyword. For the first output, focus on the organic field.

Step 5: Normalize the response

Add another Code node after the HTTP Request node.

const keyword = $json.request_params?.q || $json.keyword;
const organic = $json.organic || [];

return organic.map((item) => ({
  json: {
    keyword,
    position: item.position,
    title: item.title,
    link: item.link,
    description: item.description,
  },
}));

This turns each SERP into rows that can be written to a sheet, database, or CSV-like output.

Step 6: Store the output

For a first version, use Google Sheets or a database node.

Useful columns:

run_id
keyword
position
title
link
description
fetched_at

The run_id matters. Without it, you cannot easily compare one research run with another.

Step 7: Add a lightweight run summary

After all items finish, create a summary:

keywords submitted
unique keywords processed
results saved
failed keywords
run_id

That summary helps you know whether the workflow completed cleanly.

What to improve next

Once the basic batch flow works, the next improvements are straightforward:

add a status column for each keyword
store raw JSON beside normalized rows
extract domains from result URLs
group results by domain
add People Also Ask questions for intent research
compare one run against a previous run

The important thing is to keep batch collection boring and traceable.

A keyword list becomes much more useful when every query produces the same kind of structured output.

If you want to test a small batch workflow with live Google results, TalorData gives new accounts 500 responses to validate the process before scaling it.

Parse People Also Ask Results in a Dify Workflow

Elowen — Tue, 28 Jul 2026 01:48:07 +0000

People Also Ask data is useful for content research, but raw questions are not ready to use by default.

If you pass the entire SERP response into a Dify LLM node, the prompt gets noisy fast. A better workflow is to call a SERP API, extract only the people_also_ask field, clean it, deduplicate it, and pass a compact question list into the next node.

This walkthrough shows a simple Dify workflow pattern using TalorData SERP API as the search data source.

Workflow shape

The workflow has four steps:

Start node: accepts a keyword or topic.
HTTP Request node: calls the SERP API.
Code node: extracts and deduplicates People Also Ask questions.
LLM node: groups the questions into content angles or brief inputs.

The important part is the middle layer. Do not make the LLM parse a full SERP response if you only need questions.

Step 1: Create the input

Create a workflow input variable:

query

Example input:

serp api pricing

Keep the first version to one query. Multi-query workflows are easier after the response shape is stable.

Step 2: Add the HTTP Request node

Use a POST request.

URL:

https://serpapi.talordata.net/serp/v1/request

Headers:

Authorization: Bearer <TALORDATA_TOKEN>
Content-Type: application/x-www-form-urlencoded

Body parameters:

engine=google
q={{ query }}
json=2

If your Dify workspace uses a different variable reference format, adapt the q value to the variable syntax shown in your workflow editor.

Step 3: Inspect the field you need

The field to inspect is:

people_also_ask

For this workflow, the rest of the SERP response is secondary. You may keep organic results for later context, but the first parser should focus on questions.

A clean intermediate result can look like this:

[
  { "question": "What is a SERP API?" },
  { "question": "How do I use Google Search data in an AI workflow?" }
]

Step 4: Deduplicate questions

People Also Ask questions can be repetitive. Some differ only by casing, punctuation, or small wording changes.

In a Code node, normalize the question before deduplication:

const paa = $json.people_also_ask || [];
const seen = new Set();
const questions = [];

for (const item of paa) {
  const raw = item.question || item.title || "";
  const question = raw.trim();

  if (!question) continue;

  const key = question
    .toLowerCase()
    .replace(/[?!.]/g, "")
    .replace(/\s+/g, " ");

  if (seen.has(key)) continue;

  seen.add(key);
  questions.push({ question });
}

return [
  {
    json: {
      query: $json.request_params?.q,
      questions,
      question_count: questions.length,
    },
  },
];

This gives the LLM a compact list instead of a full API response.

Step 5: Ask the LLM to group, not invent

The next LLM node should transform the extracted questions. It should not invent a new list from scratch.

A prompt can be simple:

You are helping create a content brief from real People Also Ask questions.

Input:
{{ questions }}

Group these questions into 3-5 search-intent clusters. Keep the original questions visible under each cluster. Do not add questions that are not present in the input.

This keeps the workflow grounded in the PAA data.

Step 6: Return a useful output

A practical output format:

Topic: SERP API pricing

Cluster 1: API basics
- What is a SERP API?
- How does a SERP API work?

Cluster 2: Use cases
- How do AI agents use search data?
- How can SEO teams automate SERP monitoring?

Cluster 3: Evaluation
- What should I compare when choosing a SERP API?

That output is easier to turn into a brief than a raw list of questions.

What to log

During testing, log:

input query
response timestamp if available
number of PAA items returned
number of deduplicated questions
final grouped clusters

If a brief looks weak, you can inspect whether the issue came from the query, the SERP response, the parser, or the LLM grouping step.

Final thought

A Dify workflow does not need to hand the entire SERP response to an LLM. For content research, it is often better to extract one useful field, shape it carefully, and then let the LLM organize it.

If you want to test this with live search data, new TalorData accounts include 500 responses for a small Dify PAA workflow.

5 Failure Modes When Adding TalorData Search to a LangChain Agent

Elowen — Mon, 27 Jul 2026 02:03:20 +0000

Adding web search to a LangChain agent looks simple at first.

Install a tool, pass the user query, return the results, and let the agent reason over fresh search context. That is often enough for a demo. It is not always enough for a reliable workflow.

When an agent can search, new failure modes appear: weak queries, noisy results, oversized context, missing logs, and unclear source handling.

This post walks through five issues I check when adding TalorData SERP API search to a LangChain agent.

Basic setup

The confirmed LangChain package is langchain-talordata.

pip install langchain-talordata

Set the token as an environment variable. The package reads TALOR_API_KEY.

export TALOR_API_KEY="<TALORDATA_TOKEN>"

A common starting point is:

from langchain_talordata import TalorSerpTool

search_tool = TalorSerpTool.from_env()

Now the harder part starts: deciding when and how the agent should use the tool.

1. The agent sends a weak query

A user may ask:

What changed in the market this week?

That is not a useful search query by itself. If the agent passes it directly to a SERP tool, the result may be broad, vague, or impossible to evaluate.

A better pattern is to make query generation explicit:

def build_search_query(user_goal: str, topic: str) -> str:
    return f"{topic} market trends recent updates"

For agent workflows, I usually log both values:

the original user request
the generated search query

That gives you something to inspect when the answer is weak.

2. The agent treats every result as equal

SERP results are structured, but they still need filtering.

The agent should not blindly pass every title and snippet into the final prompt.

For a first version, filter to a compact result shape:

def compact_result(item: dict) -> dict:
    return {
        "position": item.get("position"),
        "title": item.get("title"),
        "link": item.get("link"),
        "description": item.get("description"),
    }

Then decide what is useful for the task:

top organic results for ranking context
People Also Ask questions for user intent
AI Overview content for generated SERP context
metadata for debugging and traceability

A search tool should return evidence, not a pile of text.

3. The context gets too large

The fastest way to make an agent unreliable is to pass too much unranked search context into the prompt.

Instead of giving the LLM the entire response, shape the output before it reaches the agent.

Example:

def shape_search_context(data: dict, limit: int = 5) -> list[dict]:
    organic = data.get("organic", [])[:limit]
    return [compact_result(item) for item in organic]

This gives the agent a smaller, inspectable context window.

If the user needs deeper research, the agent can run a second search or ask for clarification. It should not start by flooding the prompt.

4. No one logs the search call

If the agent gives a bad answer, you need to know what it searched.

At minimum, log:

user request
generated query
search parameters
timestamp
result count
selected result links

This is especially important when live search is involved. The web changes, so the same question may produce different evidence later.

Without logs, debugging becomes guesswork.

5. The final answer hides the source boundary

Search data can support an answer, but the agent still needs to explain what it used.

A practical final response should separate:

what the search results showed
what the agent inferred
what remains uncertain

That makes the answer easier to review and safer to improve.

A small checklist

Before shipping a LangChain search tool, I check:

Is query generation visible?
Is the SERP response filtered before the LLM sees it?
Are organic, people_also_ask, or ai_overview fields used intentionally?
Is the search call logged?
Can the final answer point back to selected source URLs?
Is there a fallback when the search result is weak?

Adding search is not just about giving the agent more information. It is about giving the agent better evidence and making that evidence reviewable.

If you want to test this with live Google results, TalorData gives new accounts 500 responses to build and debug a small LangChain search workflow.

Build an n8n Workflow to Track Organic Result Changes

Elowen — Sat, 25 Jul 2026 02:29:56 +0000

Manual SERP checks do not scale well. You search a keyword, look at the top results, maybe take a screenshot, and then repeat the same thing tomorrow. The problem is not the first check. The problem is remembering exactly what changed.

For a small monitoring workflow, I like starting with n8n: one scheduled trigger, one API request, one normalization step, and one comparison step. This walkthrough builds a simple n8n workflow that calls TalorData SERP API, extracts organic results, and compares the latest snapshot with a previous snapshot.

Workflow shape

The first version has five parts:

Schedule Trigger
Set node for query settings
HTTP Request node for SERP data
Code node to normalize organic results
Code node to compare current and previous results

You can add storage, Slack, email, or Google Sheets later. The important part is to make the change logic clear before adding notifications.

Step 1: Create the query settings

Add a Set node after the Schedule Trigger. Create fields like:

query = google search api
engine = google
num = 10

If you plan to monitor multiple queries, keep the first workflow to one query until the data shape is stable. Batch mode is easier after the single-query version works.

Step 2: Add the HTTP Request node

Use a POST request.

https://serpapi.talordata.net/serp/v1/request

Headers:

Authorization: Bearer <TALORDATA_TOKEN>
Content-Type: application/x-www-form-urlencoded

Body parameters:

engine=google
q={{ $json.query }}
num={{ $json.num }}
json=2

The response should be JSON. For this workflow, the main field we care about is organic.

Step 3: Normalize organic results

Add a Code node after the HTTP Request node. Keep a compact shape that is easy to compare:

const organic = $json.organic || [];

return organic.map((item) => ({
  json: {
    position: item.position,
    title: item.title,
    link: item.link,
    description: item.description,
  },
}));

This avoids comparing the entire SERP response. For change detection, a small normalized record is easier to debug.

Step 4: Store the previous snapshot

For a real workflow, you need somewhere to keep the previous run. Good first options are Google Sheets, Airtable, Postgres, or n8n static data for a small prototype.

The snapshot should include:

query
fetched_at
position
title
link
description

Step 5: Compare current and previous results

Create a key from the URL, then check whether each URL is new, removed, or moved.

const current = $input.all().map((item) => item.json);

// Replace this with records loaded from your storage node.
const previous = [];

const previousByLink = new Map(previous.map((item) => [item.link, item]));
const currentByLink = new Map(current.map((item) => [item.link, item]));

const changes = [];

for (const item of current) {
  const old = previousByLink.get(item.link);

  if (!old) {
    changes.push({ type: "new_result", link: item.link, title: item.title, position: item.position });
    continue;
  }

  if (old.position !== item.position) {
    changes.push({ type: "position_changed", link: item.link, title: item.title, previous_position: old.position, current_position: item.position });
  }
}

for (const item of previous) {
  if (!currentByLink.has(item.link)) {
    changes.push({ type: "removed_result", link: item.link, title: item.title, previous_position: item.position });
  }
}

return changes.map((change) => ({ json: change }));

Avoid noisy alerts

Do not send a notification for every tiny movement. Useful rules might be: alert when a new domain enters the top 10, alert when a tracked competitor moves into the top 5, alert when your page disappears from the first page, or log minor position changes without notifying anyone.

Most bugs in this type of workflow are data-shape bugs, not API-call bugs. Inspect the raw response, normalized output, previous snapshot, generated changes, and final alert payload before adding notifications.

An n8n SERP monitor does not need to be complicated. Start by making the search snapshot repeatable, then make the diff readable, then decide which changes deserve attention.

If you want to test the same workflow with live Google results, new TalorData accounts include 500 responses for a small first monitor.