DEV Community: Elowen

Build a Minimal SERP Snapshot Script for Keyword Checks

Elowen — Thu, 02 Jul 2026 09:18:02 +0000

If you check the same keyword manually every few days, you will eventually run into the same problem: you cannot reliably compare what changed.

A browser tab gives you a result page.

A snapshot gives you something you can store, diff, review, and discuss with a team.

This post shows a minimal pattern for saving SERP snapshots from a SERP API. The goal is not to build a full SEO platform. The goal is to create a small, repeatable starting point.

What the script should do

For a first version, keep the job narrow:

accept a keyword
call a SERP API
extract the top organic results
save the snapshot with a timestamp
keep the raw response optional for debugging

That is enough to answer a basic question later:

"What did the search results look like when we checked this keyword?"

A simple snapshot shape

I like to save a normalized object before thinking about dashboards or alerts.

{
  "keyword": "serp api for seo monitoring",
  "checked_at": "2026-07-01T09:00:00Z",
  "results": [
    {
      "position": 1,
      "title": "Example result title",
      "url": "https://example.com/page",
      "snippet": "Short result summary..."
    }
  ]
}

The exact fields will depend on the SERP API you use. If you use Talor Data or another SERP API provider, map this shape to the response fields from the provider docs instead of assuming every API returns the same schema.

Minimal Python example

This example is intentionally generic. Replace the endpoint, auth format, parameters, and response field names with the SERP API you use.

import json
import os
from datetime import datetime, timezone

import requests


SERP_API_ENDPOINT = os.environ["SERP_API_ENDPOINT"]
SERP_API_KEY = os.environ["SERP_API_KEY"]


def fetch_serp(keyword: str) -> dict:
    response = requests.get(
        SERP_API_ENDPOINT,
        headers={"Authorization": f"Bearer {SERP_API_KEY}"},
        params={
            "q": keyword,
            "engine": "google",
            "location": "United States",
        },
        timeout=30,
    )
    response.raise_for_status()
    return response.json()


def normalize_results(raw: dict) -> list[dict]:
    # Adjust this path to match your SERP API response schema.
    organic_results = raw.get("organic_results", [])

    normalized = []
    for index, item in enumerate(organic_results[:10], start=1):
        normalized.append(
            {
                "position": item.get("position", index),
                "title": item.get("title"),
                "url": item.get("link") or item.get("url"),
                "snippet": item.get("snippet"),
            }
        )

    return normalized


def save_snapshot(keyword: str, results: list[dict]) -> str:
    checked_at = datetime.now(timezone.utc).isoformat()
    snapshot = {
        "keyword": keyword,
        "checked_at": checked_at,
        "results": results,
    }

    safe_keyword = keyword.lower().replace(" ", "_").replace("/", "_")
    filename = f"serp_snapshot_{safe_keyword}_{checked_at[:10]}.json"

    with open(filename, "w", encoding="utf-8") as file:
        json.dump(snapshot, file, indent=2, ensure_ascii=False)

    return filename


if __name__ == "__main__":
    keyword = "serp api for seo monitoring"
    raw = fetch_serp(keyword)
    results = normalize_results(raw)
    path = save_snapshot(keyword, results)
    print(f"Saved {len(results)} results to {path}")

What to store first

Do not start with every possible field.

Start with the fields you can explain:

keyword
checked timestamp
location or market
position
title
URL
snippet
result type if your API provides it

Once this is reliable, you can add SERP features, ads, People Also Ask, local packs, or AI search modules depending on your monitoring goals.

Why this is useful

The first benefit is not automation.

The first benefit is repeatability.

When someone asks why traffic changed, you can look back at saved search result snapshots instead of trying to recreate an old SERP from memory.

A SERP API such as Talor Data SERP API can act as the collection layer behind this workflow, while your script owns the parts that are specific to your team: storage, naming, review, and comparison.

Final note

If you already save SERP snapshots, I would be interested in how you structure them. Do you store only normalized fields, or do you keep the raw API response as well?

How to Build a Google Rank Tracker with TalorData SERP API

Elowen — Mon, 29 Jun 2026 06:38:10 +0000

Tracking where your domain ranks for a list of keywords on Google sounds like a simple one-afternoon project. And it is—until you try to scrape google.com yourself.

After a few hundred requests from the same IP, you start getting the consent page, then a CAPTCHA, then nothing at all. Rotating proxies and headless browsers work for a while, but you end up spending more time keeping the scraper alive than actually using the data.

Easier path: hand that problem to a SERP API and keep your code focused on the rank itself.

In this tutorial, we'll build a complete Google rank tracker in Python using TalorData SERP API—from a single request to a full tracking system with SQLite persistence.

What We're Building

A Python script that:

Takes a list of keywords and a target domain
Queries Google via TalorData SERP API
Finds the target domain's position in organic results
Stores historical rankings in SQLite
Retrieves ranking history for trend visualization

Why TalorData?

TalorData provides a multi-engine SERP API that returns structured search results from Google, Bing, Yandex, and DuckDuckGo through a single endpoint.

Feature	Details
Free tier	1,000 free API responses, no credit card required
Pricing	~$0.25–$1.00 per 1,000 successful requests
Engines	Google, Bing, Yandex, DuckDuckGo
Output	Structured JSON (or raw HTML)
Geo-targeting	Location-specific results

Cost comparison: SerpAPI charges ~$10 per 1,000 requests, while TalorData is ~$0.25–$1.00. For 50 keywords tracked daily, that's about $1.50/month vs $15/month.

Step 1: Get Your API Token

Sign up at talordata.com
Navigate to the dashboard and grab your Bearer Token (API token)
Export it as an environment variable:

export TALORDATA_TOKEN="your_api_token_here"

Or hardcode it (not recommended for production):

TOKEN = "your_api_token_here"

Step 2: Send Your First Request

TalorData's SERP API accepts POST requests with form-encoded parameters. Here's the minimal curl example:

curl -X POST "https://serpapi.talordata.net/serp/v1/request" \
  -H "Authorization: Bearer $TALORDATA_TOKEN" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "engine=google" \
  -d "q=serp api" \
  -d "device=desktop" \
  -d "location=United States" \
  -d "num=10" \
  -d "json=1"

Key parameters:

Parameter	Description
`engine`	Search engine: `google`, `bing`, `yandex`, `duckduckgo`
`q`	Search query/keyword
`device`	`desktop` or `mobile`
`location`	Geo-target (e.g., "United States")
`num`	Number of results to return (max 100)
`json`	`1` or `2` for structured JSON output

The response includes top-level keys like:

organic — organic search results (where the rank lives)
sponsored_results — paid ads
people_also_ask — related questions
pagination — next page info
search_information — metadata about the search

Step 3: Parse Rankings in Python

Here's a complete Python function that sends a query and extracts the target domain's position:

import os
import requests

ENDPOINT = "https://serpapi.talordata.net/serp/v1/request"
TOKEN = os.environ.get("TALORDATA_TOKEN")

def search(query: str, location: str = "United States", num: int = 10) -> dict:
    """Send a search request to TalorData SERP API and return the JSON response."""
    resp = requests.post(
        ENDPOINT,
        headers={
            "Authorization": f"Bearer {TOKEN}",
            "Content-Type": "application/x-www-form-urlencoded",
        },
        data={
            "engine": "google",
            "q": query,
            "device": "desktop",
            "location": location,
            "num": num,
            "json": 1,
        },
        timeout=60,
    )
    resp.raise_for_status()
    return resp.json()


def find_rank(data: dict, target_domain: str) -> int | None:
    """
    Find the rank (position) of a target domain in the organic search results.
    Returns None if the domain is not found.
    """
    for item in data.get("organic", []):
        link = item.get("link", "")
        # Extract domain from link for matching
        if target_domain.lower() in link.lower():
            return item.get("position")
    return None


# Example usage
if __name__ == "__main__":
    results = search("serp api")

    # Print top 5 results
    for r in results.get("organic", [])[:5]:
        print(f'{r["position"]:>2} {r.get("display_link", "")}')
        print(f'    {r.get("title", "")}')

    # Find where talordata.com ranks
    rank = find_rank(results, "talordata.com")
    print(f"\nTalorData rank for 'serp api': {rank if rank else 'Not found'}")

Expected output (actual results will vary):

 1 https://serpapi.com
    SerpApi: Google Search API
 2 https://brightdata.com/products/serp-api
    SERP API - SERP Scraper API - Free Trial
 3 https://dataforseo.com/APIs
    SERP API You Can Trust
 4 https://serper.dev
    Serper - The World's Fastest and Cheapest Google Search API
 5 https://github.com/serpapi/google-search-results-python
    serpapi/google-search-results-python

TalorData rank for 'serp api': Not found

Each entry in organic contains:

position — the rank (1 = first result)
title — page title
link — full URL
display_link — breadcrumb URL shown under the title
description — snippet text

Note: For domain matching, use the host of link rather than display_link, as display_link is Google's rendered version and may differ.

Step 4: Build the Full Rank Tracker

Now let's build a complete tracker that:

Tracks multiple keywords
Stores historical data in SQLite
Retrieves history for trend analysis

Complete Code

import os
import sqlite3
from datetime import datetime
from typing import Optional

import requests

ENDPOINT = "https://serpapi.talordata.net/serp/v1/request"


class RankTracker:
    def __init__(self, api_token: str, db_path: str = "rank_tracker.db"):
        self.api_token = api_token
        self.conn = sqlite3.connect(db_path)
        self._init_db()

    def _init_db(self) -> None:
        """Initialize the SQLite database table."""
        cursor = self.conn.cursor()
        cursor.execute("""
            CREATE TABLE IF NOT EXISTS rankings (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                keyword TEXT NOT NULL,
                domain TEXT NOT NULL,
                rank INTEGER,
                checked_at TIMESTAMP NOT NULL
            )
        """)
        self.conn.commit()

    def check_rank(self, keyword: str, target_domain: str, location: str = "United States") -> Optional[int]:
        """
        Query Google for a keyword and return the rank of the target domain.
        Returns None if the domain is not found in the top results.
        """
        headers = {
            "Authorization": f"Bearer {self.api_token}",
            "Content-Type": "application/x-www-form-urlencoded",
        }
        data = {
            "engine": "google",
            "q": keyword,
            "device": "desktop",
            "location": location,
            "num": 100,  # Get top 100 results
            "json": 1,
        }

        resp = requests.post(ENDPOINT, headers=headers, data=data, timeout=60)
        resp.raise_for_status()
        results = resp.json()

        for item in results.get("organic", []):
            link = item.get("link", "")
            if target_domain.lower() in link.lower():
                return item.get("position")

        return None

    def save_rank(self, keyword: str, domain: str, rank: Optional[int]) -> None:
        """Save a ranking record to the database."""
        cursor = self.conn.cursor()
        cursor.execute(
            "INSERT INTO rankings (keyword, domain, rank, checked_at) VALUES (?, ?, ?, ?)",
            (keyword, domain, rank, datetime.utcnow()),
        )
        self.conn.commit()

    def track_keywords(self, keywords: list[str], domain: str, location: str = "United States") -> list[dict]:
        """
        Track multiple keywords for a single domain.
        Returns a list of results with keyword, rank, and timestamp.
        """
        results = []
        for keyword in keywords:
            rank = self.check_rank(keyword, domain, location)
            self.save_rank(keyword, domain, rank)
            results.append({
                "keyword": keyword,
                "rank": rank,
                "checked_at": datetime.utcnow().isoformat(),
            })
            print(f"{keyword}: {rank if rank else 'Not found'}")
        return results

    def get_history(self, keyword: str, domain: str, limit: int = 30) -> list[tuple]:
        """
        Get historical rankings for a specific keyword and domain.
        Returns a list of (rank, checked_at) tuples, most recent first.
        """
        cursor = self.conn.cursor()
        cursor.execute(
            "SELECT rank, checked_at FROM rankings "
            "WHERE keyword = ? AND domain = ? "
            "ORDER BY checked_at DESC LIMIT ?",
            (keyword, domain, limit),
        )
        return cursor.fetchall()

    def get_latest_rank(self, keyword: str, domain: str) -> Optional[int]:
        """Get the most recent rank for a keyword and domain."""
        cursor = self.conn.cursor()
        cursor.execute(
            "SELECT rank FROM rankings "
            "WHERE keyword = ? AND domain = ? "
            "ORDER BY checked_at DESC LIMIT 1",
            (keyword, domain),
        )
        row = cursor.fetchone()
        return row[0] if row else None

    def close(self) -> None:
        """Close the database connection."""
        self.conn.close()


# Example usage
if __name__ == "__main__":
    API_TOKEN = os.environ.get("TALORDATA_TOKEN")
    if not API_TOKEN:
        raise ValueError("Please set TALORDATA_TOKEN environment variable")

    tracker = RankTracker(API_TOKEN)

    # Keywords to track
    keywords = [
        "serp api",
        "google search api",
        "best serp api 2026",
        "seo rank tracker",
        "search engine results api",
    ]

    # Your domain
    domain = "talordata.com"

    # Run the tracker
    print("Tracking keywords...")
    tracker.track_keywords(keywords, domain)

    # Check history for one keyword
    print("\nHistory for 'serp api':")
    history = tracker.get_history("serp api", domain, limit=10)
    for rank, checked_at in history:
        print(f"  {checked_at}: rank {rank if rank else 'Not found'}")

    tracker.close()

What This Code Does

check_rank() — Sends a request to TalorData, iterates through organic results, and returns the position where the target domain appears.
save_rank() — Stores each check with a timestamp in SQLite.
track_keywords() — Loops through a list of keywords, checks each one, saves the results, and prints progress.
get_history() — Retrieves historical rankings for a specific keyword/domain pair, ordered by date (most recent first).
get_latest_rank() — Quick lookup for the most recent rank without fetching all history.

Step 5: Visualize the Data (Optional)

With historical data in SQLite, you can easily visualize ranking trends. Here's a quick example using matplotlib:

import matplotlib.pyplot as plt
from datetime import datetime

def plot_rank_history(tracker, keyword: str, domain: str):
    history = tracker.get_history(keyword, domain, limit=30)
    if not history:
        print("No history found.")
        return

    # Reverse to plot chronological (oldest to newest)
    history.reverse()
    dates = [datetime.fromisoformat(str(ts).replace(" ", "T")) for _, ts in history]
    ranks = [r if r is not None else 101 for r, _ in history]  # 101 = not found

    plt.figure(figsize=(10, 5))
    plt.plot(dates, ranks, marker="o")
    plt.gca().invert_yaxis()  # Lower rank is better
    plt.xlabel("Date")
    plt.ylabel("Rank")
    plt.title(f"Ranking History: '{keyword}'")
    plt.grid(True, alpha=0.3)
    plt.show()

# Usage
# plot_rank_history(tracker, "serp api", "talordata.com")

Step 6: Automate with a Scheduler

To run this automatically (e.g., daily), you can use:

Option 1: Cron job (Linux/macOS)

# Run daily at 9 AM
0 9 * * * cd /path/to/project && python rank_tracker.py

Option 2: GitHub Actions
Create .github/workflows/rank-track.yml:

name: Track Rankings
on:
  schedule:
    - cron: '0 9 * * *'  # Daily at 9 AM UTC
jobs:
  track:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: pip install requests
      - run: python rank_tracker.py
        env:
          TALORDATA_TOKEN: ${{ secrets.TALORDATA_TOKEN }}

Cost Analysis

Let's calculate the monthly cost for tracking 50 keywords daily:

Metric	Value
Keywords	50
Checks per day	50
Checks per month	50 × 30 = 1,500
TalorData price	~$0.25–$1.00 per 1,000 requests
Monthly cost	~$1.35–$1.50

Compare this to building and maintaining your own scraper:

Approach	Monthly Cost	Maintenance Time
Self-built scraper	$20–50+ (proxies)	Several hours/week
TalorData SERP API	~$1.50	0

Supported Search Engines

TalorData supports multiple search engines through the same endpoint:

Engine	`engine` parameter
Google	`google`
Bing	`bing`
Yandex	`yandex`
DuckDuckGo	`duckduckgo`

To switch engines, simply change the engine parameter in your request:

data = {
    "engine": "bing",  # or "yandex", "duckduckgo"
    "q": keyword,
    # ... other parameters
}

Common Pitfalls and Tips

Domain matching: Use the host of link rather than display_link for reliable matching.
Rate limiting: TalorData charges per successful request—failed requests are not billed. Still, cache results for stable keywords to save costs.
Location matters: Rankings vary significantly by geographic location. Always specify location for consistent results.
Free tier: You get 1,000 free requests after signing up—more than enough to test and build your MVP.

Next Steps

Once your rank tracker is running, consider these enhancements:

Multi-domain tracking — Monitor competitors alongside your own domain
Alert system — Send email/Slack notifications when rankings change significantly
Dashboard — Build a web UI with Streamlit or Flask to visualize trends
Multi-engine tracking — Check rankings on Bing and DuckDuckGo too
SERP feature extraction — Track "People Also Ask" questions and featured snippets for content ideas

Summary

In ~100 lines of Python, we built a complete Google rank tracker that:

Fetches search results via TalorData SERP API (no scraping, no CAPTCHAs)
Extracts target domain positions from organic results
Persists historical data to SQLite
Retrieves ranking history for analysis

Why this works:

TalorData handles the hard parts (proxies, parsing, rate limits)
You focus on the data and what it means
Cost is minimal (~$1.50/month for 50 keywords daily)
1,000 free requests to get started

Get started today: Sign up at talordata.com, grab your free 1,000 requests, and start tracking your rankings in minutes.

Have questions? Drop them in the comments below. Happy tracking!

Build an AI-Powered SEO Content Brief Generator with Python and SERP API

Elowen — Sat, 27 Jun 2026 06:53:05 +0000

Ever spent hours researching what to write for your next blog post? You manually check the top 10 results, analyze what's working, and figure out how to make something better. What if you could automate that entire process with a few lines of code?

Today, we're going to build an AI-powered SEO content brief generator that automatically pulls the top 10 search results for your keyword and uses GPT to create a complete content brief. It takes about 10 minutes, and we'll be using Python and TalorData's SERP API.

Why Automate This?

Creating a good content brief used to take me 30–60 minutes per keyword. I'd:

Open incognito mode and search Google
Manually copy/paste titles and summaries into a document
Read through everything to identify common themes
Outline a structure that could compete

That's a lot of repetitive work that a computer can do for us. With automation, you can generate a complete content brief in under 10 seconds and just focus on writing the actual content.

Why Use a SERP API Instead of Scraping?

I tried building my own scraper for this years ago, and it was a never-ending maintenance nightmare:

Google keeps changing their HTML structure, so my selectors would break every few months
I kept getting blocked and had to mess around with rotating proxies
CAPTCHAs would randomly stop everything
When Google loads content dynamically with JavaScript, I needed a whole headless browser

Using a SERP API means someone else handles all that for you. You just make one API call and get clean structured JSON back.

What makes a SERP API good for this? You want an API that returns clean, structured JSON with titles, snippets, and URLs. No parser maintenance, no proxy management.

I'm using TalorData for this project because:

1,000 free requests on signup (no credit card required)
$0.90 per 1,000 successful requests — super affordable
Supports Google, Bing, Yandex, and Baidu through a single endpoint
Simple API that doesn't require a heavy SDK

Let's Code It!

Install dependencies:

pip install requests openai python-dotenv

Create a .env file:

TALORDATA_TOKEN=your_talordata_token_here
OPENAI_API_KEY=your_openai_key_here

Complete code:

import requests
import openai
import os
from dotenv import load_dotenv

load_dotenv()

TALORDATA_TOKEN = os.getenv("TALORDATA_TOKEN")
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
openai.api_key = OPENAI_API_KEY


def get_top_results(keyword: str, engine: str = "google", limit: int = 10):
    """Fetch top N search results from SERP API"""
    url = "https://serpapi.talordata.net/serp/v1/request"
    headers = {
        "Authorization": f"Bearer {TALORDATA_TOKEN}",
        "Content-Type": "application/x-www-form-urlencoded"
    }
    data = {
        "engine": engine,
        "q": keyword,
        "json": "2",
        "num": limit
    }

    response = requests.post(url, headers=headers, data=data)
    response.raise_for_status()
    results = response.json()

    return [
        {
            "position": r["position"],
            "title": r["title"],
            "snippet": r.get("snippet", "")
        }
        for r in results.get("organic_results", [])[:limit]
    ]


def format_results_for_prompt(results):
    """Format search results into something the LLM can read"""
    output = "Current top 10 search results:\n\n"
    for i, result in enumerate(results, 1):
        output += f"#{i} - Title: {result['title']}\n"
        output += f"Summary: {result['snippet']}\n\n"
    return output


def generate_content_brief(keyword: str, results):
    """Generate content brief using GPT"""
    context = format_results_for_prompt(results)

    prompt = f"""
You are an SEO expert. I want to write an article targeting: "{keyword}"

Based on the current top 10 search results for this keyword, analyze:

1. COMMON THEMES: What topics and angles are the top results covering?
2. GAPS: What questions are NOT being answered well?
3. TITLE SUGGESTIONS: 5 compelling title ideas that can outperform current results
4. OUTLINE: A detailed article outline with H2s and H3s that covers everything better
5. KEYWORD OPPORTUNITIES: Related keywords and questions to target

Current top search results:
{context}

Generate a comprehensive content brief that would help me create content that ranks.
"""

    response = openai.ChatCompletion.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7
    )

    return response.choices[0].message.content


if __name__ == "__main__":
    keyword = "best SERP API for developers 2026"

    print(f"🔍 Fetching top results for: {keyword}")
    results = get_top_results(keyword)

    print(f"📊 Found {len(results)} results")
    print("Generating content brief...\n")

    brief = generate_content_brief(keyword, results)

    print("=" * 60)
    print(f"📝 CONTENT BRIEF: {keyword}")
    print("=" * 60)
    print(brief)

You can also use curl to test the API directly:

curl -X POST 'https://serpapi.talordata.net/serp/v1/request' \
  -H 'Authorization: Bearer your_token' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'engine=google' \
  -d 'q=best SERP API' \
  -d 'device=desktop' \
  -d 'num=10' \
  -d 'json=2'

The API returns clean, structured JSON with organic_results containing position, title, link, and snippet for each result.

What About Cost?

TalorData offers 1,000 free requests on signup with no credit card required. That's enough for extensive development and testing.

Beyond the free tier, pricing starts at $0.90 per 1,000 successful requests and drops to $0.25/1K at higher volumes. Compare that to incumbents at ~$10/1K. For full pricing details, visit talordata.com.

Next Steps

Once you have the basic generator running, consider:

Add competitor analysis: Include domain authority data for each result
Question extraction: Pull "People Also Ask" questions from SERP
SERP feature detection: Check if AI Overviews or featured snippets appear
Batch processing: Generate briefs for multiple keywords at once

Final Thoughts

Building an SEO content brief generator used to require managing scrapers, proxies, and parser maintenance. In 2026, a SERP API handles all that infrastructure so you can focus on what matters: creating great content.

The code above is production-ready — you can run it today with a free TalorData API key and start generating briefs in seconds instead of hours.

Have you built any SEO automation tools? What other workflows would you automate with a SERP API? Drop a comment below!

How to Build a Real-Time AI Agent with LangChain + TalorData

Elowen — Fri, 26 Jun 2026 06:30:13 +0000

If you're building AI agents in 2026, one thing becomes obvious pretty quickly: without real-time search, your agent is living in the past. LLMs have a knowledge cutoff, and no amount of prompt engineering can fix that.

The solution? Give your agent a search tool. In this tutorial, I'll walk you through building a LangChain agent that can search Google, Bing, or other search engines in real time using the TalorData SERP API. We'll keep it practical — complete code, explanations, and everything you need to run it yourself.

What We're Building

We'll create a ReAct-style agent that:

Takes a user question in natural language
Decides when it needs to search the web
Calls the TalorData SERP API to fetch live search results
Synthesizes the results into a coherent answer

The agent will use LangChain with TalorSerpTool from the langchain-talor-serp package.

Why TalorData?

Before we dive into code, let's talk about why I chose TalorData for this tutorial. According to a 1,009-query benchmark published in 2026, TalorData scored 79.19 overall, ranking first among six major SERP API providers including SerpApi, Bright Data, and Serper. It returned the largest number of organic rows (8,931) and had a zero-result rate of just 2.08%.

On the pricing side, TalorData offers 1,000 free requests upon sign-up with no credit card required. Paid plans start at $0.25 per 1,000 requests — significantly lower than incumbents like SerpApi at roughly $10 per 1,000 requests.

The langchain-talor-serp package supports 33 search engines in one tool, including Google, Bing, DuckDuckGo, Google News, Google Images, and more.

Prerequisites

Before we start, make sure you have:

Python 3.9+
An OpenAI API key (or any other LLM provider supported by LangChain)
A TalorData API key (sign up at talordata.com — you get 1,000 free requests)

Step 1: Install Dependencies

Create a new project and install the required packages:

pip install langchain langchain-openai langchain-talor-serp python-dotenv

The langchain-talor-serp package provides LangChain tools for TalorData's SERP APIs, enabling your AI agents to search with geo-targeting and language customization.

Step 2: Set Up Environment Variables

Create a .env file in your project root:

OPENAI_API_KEY=your-openai-api-key
TALOR_API_KEY=your-talordata-api-key

Then load them in your Python script:

import os
from dotenv import load_dotenv

load_dotenv()

os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
os.environ["TALOR_API_KEY"] = os.getenv("TALOR_API_KEY")

Step 3: Create the Search Tool (Modern Approach)

The langchain-talor-serp package gives us a clean wrapper around the TalorData API. Here's the recommended way to set it up using LangChain's modern tool-calling approach:

from langchain_talor_serp import TalorSerpTool
from langchain_openai import ChatOpenAI

# Initialize the LLM
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

# Create the search tool from environment variables
tool = TalorSerpTool.from_env()

# Bind the tool to the LLM for tool calling
model_with_tools = llm.bind_tools([tool])

# Now the model can call the search tool
response = model_with_tools.invoke("Search for the latest LangChain news")
print(response)

This is the modern LangChain approach — using bind_tools() instead of the older create_react_agent pattern.

Step 4: Using the Search Tool Directly

You can also invoke the search tool directly with specific parameters:

from langchain_talor_serp import TalorSerpTool

search_tool = TalorSerpTool.from_env()

result = search_tool.invoke({
    "query": "LangChain tutorial 2026",
    "engine": "google",
    "params": {
        "gl": "us",        # country
        "hl": "en",        # language
        "device": "desktop",
    },
})

print(result)

The search parameters include:

query: required — the search query text
engine: optional — engine key like google_news, google_images, bing, duckduckgo
params: optional — engine-specific parameter object

Common params fields include gl (country), hl (language), device, location, and no_cache.

You can also pass params as a JSON string when the parameter object is generated by a model tool call:

result = search_tool.invoke({
    "query": "LangChain tutorial",
    "engine": "google",
    "params": "{\"hl\": \"zh-CN\", \"gl\": \"cn\"}",
})

Step 5: Build a Complete Agent with ReAct

If you prefer the classic ReAct agent pattern, here's how to set it up:

import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from langchain.agents import create_react_agent, AgentExecutor
from langchain_talor_serp import TalorSerpTool
from langchain_core.prompts import PromptTemplate

load_dotenv()

os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
os.environ["TALOR_API_KEY"] = os.getenv("TALOR_API_KEY")

def create_search_agent():
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    search_tool = TalorSerpTool.from_env()

    prompt = PromptTemplate.from_template("""
You are a helpful AI assistant with access to a search engine.

You have access to the following tool:
- search: use this tool to search the web for real-time information

To use a tool, respond with:

Thought: I need to search for [your query]
Action: search
Action Input: {{"query": "your search query", "engine": "google"}}


When you have the final answer, respond with:

Thought: I now have the information I need
Final Answer: [your complete answer]


Question: {input}
{agent_scratchpad}
""")

    agent = create_react_agent(llm=llm, tools=[search_tool], prompt=prompt)
    return AgentExecutor(agent=agent, tools=[search_tool], verbose=True, handle_parsing_errors=True)

if __name__ == "__main__":
    agent = create_search_agent()

    questions = [
        "What's the current state of the SERP API market in 2026?",
        "Who are the main competitors to TalorData in the SERP API space?"
    ]

    for q in questions:
        print(f"\n{'='*50}")
        print(f"Q: {q}")
        print(f"{'='*50}")
        response = agent.invoke({"input": q})
        print(f"A: {response['output']}")

Step 6: Query History (Bonus)

The package also provides a history tool to fetch SERP request history with filters:

from langchain_talor_serp import TalorSerpTool

history_tool = TalorSerpTool.history_from_env()

result = history_tool.invoke({
    "page": 1,
    "page_size": 20,
    "search_query": "langchain",
    "search_engine": "google",
    "status": "success",
    "timezone": "Asia/Shanghai",
})

print(result)

History parameters include:

page: page number, defaults to 1
page_size: page size, defaults to 20
search_query: optional keyword filter
search_engine: optional engine filter such as bing
status: all, success, or error
start_time: optional Unix timestamp in seconds
end_time: optional Unix timestamp in seconds
timezone: optional timezone header such as Asia/Shanghai or +08:00

Step 7: Usage Statistics (Bonus)

To monitor your API usage, use the statistics tool:

from langchain_talor_serp import TalorSerpTool

statistics_tool = TalorSerpTool.statistics_from_env()

result = statistics_tool.invoke({
    "start_date": "2026-06-01",
    "end_date": "2026-06-05",
    "engines": "google,bing",
})

print(result)

What About Cost?

One of the nice things about this setup is cost transparency. TalorData uses a 1:1 billing model — one request, one credit. No hidden multipliers. And if your credits expire, buying any new plan automatically restores the expired credits, so your prior spend isn't wasted.

For most development and testing, the 1,000 free requests will take you quite far. Each search in our agent counts as one request.

Common Pitfalls

Missing API keys: Make sure both OPENAI_API_KEY and TALOR_API_KEY are set in your environment.
Engine parameter: If you don't specify an engine, the tool uses a default. For production, explicitly set engine to avoid surprises.
Rate limits: TalorData rejects requests beyond your quota rather than billing you for overage. This is actually a good thing — no surprise charges.
JSON string for params: When params comes from a model tool call, it may be a JSON string. The SDK accepts both dict and string formats.

Next Steps

Once you have the basic agent running, consider:

Adding multiple search tools (Google, Bing, DuckDuckGo) and letting the agent choose
Implementing a caching layer to reduce repeated searches
Adding the history tool to let the agent reflect on past searches
Deploying the agent with LangGraph for more complex workflows

Final Thoughts

Building an AI agent with real-time search capabilities doesn't have to be expensive or complicated. With LangChain's bind_tools() approach and TalorData's SERP API, you can have a working agent in minimal code. The key is giving your agent the right tools — and a search tool is arguably the most important one you can add.

Have you built an AI agent with web search capabilities? What other tools have you integrated? Drop a comment below — I'd love to hear about your experience.

Question for the community: What's the most creative use case you've built (or want to build) for a search-enabled AI agent? Share your ideas in the comments!

Web Scraping as We Know It Is Dead: 3 SERP API Trends for 2026

Elowen — Thu, 25 Jun 2026 07:51:26 +0000

If you are still running headless Chrome instances with Puppeteer or Playwright to scrape Google search results in 2026, you are likely familiar with the maintenance burden. Between Google mutating its DOM structure, Cloudflare deploying JavaScript challenges, and IP bans disrupting workflows, maintaining an in-house scraper has become a high-cost engineering effort.

The way developers extract search data is changing. The consumer of this data is increasingly an AI—not a human marketer reading an SEO report. For those building LLM applications, RAG pipelines, or autonomous agents, three trends are reshaping the SERP API landscape.

Trend 1: Raw HTML Is a Liability. Structured JSON Is the Standard.

Two years ago, scraping meant downloading HTML and parsing it with BeautifulSoup. Today, feeding a raw DOM tree—filled with inline CSS, tracking scripts, and extraneous markup—into an LLM wastes tokens and increases the risk of hallucinations. Modern data pipelines require clean, noise-free JSON out of the box. SERP APIs have evolved to abstract away parsing entirely. Developers want an array of organic_results and knowledge_graph data they can pass directly into LLM prompts without writing regex or maintaining selectors.

Trend 2: The End of "Paying for Failures."

Historically, legacy API providers charged based on bandwidth or raw request attempts. If a request hit a CAPTCHA, timed out, or returned a 403 error, the charge still applied. The industry standard in 2026 is shifting toward "Pay-Per-Success": if the API does not return valid, structured search data, the developer should not lose a credit.

Trend 3: Cost Efficiency for AI-Scale Workloads.

As RAG becomes standard, applications are making hundreds of search queries per minute to ground AI models with real-time data. Legacy SERP APIs charging $2.00–$5.00+ per 1,000 requests can significantly impact margins for SaaS products and independent developers. Infrastructure costs must decrease to sustain AI growth.

Example: SERP API Call in Python

import requests
def get_serp_data(query):
    url = "https://api.talordata.com/v1/serp"
    headers = {
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json"
    }
    payload = {
        "engine": "google",
        "q": query,
        "location": "United States",
        "hl": "en"
    }
    response = requests.post(url, json=payload, headers=headers)
    return response.json()

The API returns structured JSON containing organic results, AI Overviews, and Knowledge Graph data in a single response—no parsing, no CAPTCHA handling, no proxy management.

The Shift Is Already Underway

These three trends are not predictions—they are observable shifts already shaping how developers build data pipelines. Google's Custom Search JSON API, long a common entry point for programmatic search access, will have its "Search entire web" mode discontinued on January 1, 2027. Google updated the Custom Search API documentation on January 9, 2026 to announce this change. Thousands of integrations—search bots, academic crawlers, price-comparison engines, and internal knowledge tools—will cease functioning on that date.

Meanwhile, AI Overviews now appear on approximately 48% of tracked Google searches as of February 2026, up from roughly 31% a year earlier, according to BrightEdge data. The overlap between pages cited in AI Overviews and those ranking in the organic top 10 has dropped from approximately 76% to 38% in seven months, according to an updated Ahrefs analysis of 863,000 keywords and 4 million AI Overview URLs. Ranking and citation are no longer the same signal.

For developers building the next generation of AI-powered applications, the choice of SERP infrastructure is no longer just about uptime and latency. It is about whether the data pipeline can sustain AI-scale workloads without incurring prohibitive costs.

Evaluate your API provider on three criteria: structured JSON output, pay-per-success pricing, and transparent per-request cost at volume. The era of maintaining in-house scrapers is ending. The question is not whether to adopt a SERP API—it is which one.

Add Live Search Results to Dify Apps with TalorData SERP API

Elowen — Wed, 24 Jun 2026 07:10:28 +0000

LLM apps are powerful, but they often need one thing they do not have by default: fresh search data.

If you are building with Dify, you can now connect your AI workflows to live search engine results through TalorData SERP API.

With the TalorData integration for Dify, your apps can retrieve structured SERP data from search engines and search verticals such as Google, Bing, Yandex, News, Maps, Images, Local, Jobs, Hotels, Flights, and Finance.

Why this matters

Many AI apps need current external information:

A research assistant needs fresh search results.
An SEO workflow needs ranking data.
A brand monitoring bot needs news and search visibility.
A local business tool needs map and local search data.
A competitor intelligence agent needs public search signals.

Without a search data source, these workflows either rely on outdated model knowledge or require custom scraping infrastructure.

TalorData SERP API gives Dify builders a simpler way to bring live search data into AI applications.

What you can build with TalorData and Dify

Here are a few practical examples.

1. AI research assistant

Let your Dify app search Google, Bing, or Yandex before answering.

Useful for:

market research
company research
topic discovery
source collection
competitive research

2. SEO rank tracking workflow

Create a Dify workflow that checks where a domain appears for a target keyword.

Useful for:

keyword rank monitoring
competitor SERP comparison
search visibility reports
desktop vs mobile SERP checks

3. Brand and news monitoring bot

Use search and news results to monitor brand mentions, product mentions, or industry topics.

Useful for:

reputation monitoring
PR tracking
trend discovery
media intelligence

4. Local business discovery tool

Use Maps or Local search data to find businesses by category and location.

Useful for:

local SEO
lead discovery
store research
location intelligence

5. Ecommerce and market monitoring

Use search, shopping, images, and related SERP data to monitor products, categories, and competitors.

Useful for:

ecommerce research
pricing intelligence
product visibility checks
marketplace monitoring

How the integration works

The integration allows Dify apps and workflows to call TalorData SERP API as an external search data tool.

A typical setup looks like this:

Get a TalorData API token.
Add TalorData to your Dify workspace.
Configure the API token.
Use the TalorData tool inside your Dify app or workflow.
Send a query and receive structured SERP data.

The response can include fields such as organic results, titles, links, snippets, rankings, pagination, request parameters, and search metadata, depending on the search engine or vertical you request.

Example use case

Imagine you are building a Dify workflow for competitor monitoring.

The workflow could:

Receive a keyword such as serp api.
Search Google and Bing through TalorData.
Extract the top organic results.
Check whether your domain or competitors appear.
Summarize the SERP landscape with an LLM.
Send the result to a dashboard, email, or internal report.

This turns live search results into structured input for an AI workflow.

Why use TalorData SERP API

TalorData is designed for developers who need structured search engine data without maintaining scraping infrastructure.

With TalorData SERP API, you can work with:

Google Search
Bing Search
Yandex Search
Google News
Bing News
Google Maps
Bing Maps
Google Images
Bing Images
Google Local
Google Jobs
Google Hotels
Google Flights
Google Finance
Google Finance Markets

Instead of building and maintaining custom parsers, your Dify workflow can call an API and work with structured response data.

Get started

You can try the Dify integration here:

TalorData SERP API for Dify

Read the setup guide:

How to integrate TalorData with Dify

If you are building AI agents, SEO workflows, research assistants, or monitoring tools in Dify, TalorData SERP API can give your app access to live search results as structured data.

Best Google SERP APIs in 2026: What Developers Should Compare Before Choosing One

Elowen — Tue, 23 Jun 2026 06:32:18 +0000

If you're building SEO tools, rank tracking dashboards, market intelligence workflows, or AI agents that need fresh Google search results, choosing the right SERP API matters.

This guide compares key factors developers should look at in 2026: reliability, location and language coverage, structured results, response speed, pricing, and how easy the API is to integrate into real production systems.

📒Read the full comparison here: Best Google SERP API 2026

TalorData SERP API now supports LangChain

Elowen — Thu, 18 Jun 2026 07:35:39 +0000

We’ve added LangChain integration for TalorData SERP API.

Now LangChain apps and agents can use fresh SERP data for search-related workflows like:

SEO research
competitor analysis
content planning
market research
query intent analysis
RAG with live search context

This helps agents understand what currently ranks, which pages are visible, and what the search landscape looks like for a given query.

Integration page

Would love to hear how builders are using search data inside LangChain agents.

Best Google SERP APIs in 2026: A Practical Comparison If you're building SEO tools, rank trackers, market research workflows, choosing the right Google SERP API matters. Guide: https://www.talordata.com/blog/best-google-serp-api-2026

Elowen — Tue, 16 Jun 2026 08:33:19 +0000

6 Best Google SERP APIs in 2026

Looking for a Google SERP API? We benchmarked 7 leading providers in 2026 on pricing, free tier, and developer experience. Find the right fit.

talordata.com

Using SERP Data in RAG and AI Agent Workflows

Elowen — Mon, 15 Jun 2026 06:37:30 +0000

RAG and AI agents are much more useful when they can access fresh external data.
For search-related workflows, SERP data can provide useful context:

what pages currently rank
which competitors appear
what search intent looks like
what questions users are asking
which sources are visible for a topic

This is useful for AI agents that handle SEO research, content planning, market monitoring, or competitive analysis.

A simple workflow could look like this:
User query
→ AI agent
→ SERP data tool
→ extract ranking pages and SERP features
→ summarize search intent
→ generate recommendations

Instead of relying only on static knowledge, the agent can reason over current search results.

We are exploring this direction with TalorData SERP API and MCP workflows.
Docs:
https://docs.talordata.com/cn-tw/serp-api/mcp-server/quick-start

Connect SERP Data to AI Tools with a MCP Server

Elowen — Fri, 12 Jun 2026 07:17:13 +0000

Stop manually collecting search results or fighting broken scrapers.

The TalorData MCP Server gives AI workflows a clean, structured way to access live SERP data — without the usual headaches.

AI tools become far more useful when they can pull real‑time search context, not just static training data.

This is useful if you're building:

AI agents that need live search context
SEO research assistants
Competitor monitoring workflows
Keyword research tools
Content research automation
Search intelligence dashboards

Get started in minutes:
👉 Quick Start Guide

If you're experimenting with MCP, adding search data is one of the most practical moves you can make. It gives your model access to real‑world search context — not just what it already knows.

Only 2 of 6 SERP APIs reliably return AI Overviews: • SerpApi — 69.38% • TalorData — 64.52% • Bright Data — 3.17% • Oxylabs /Serper — 0% If you're building for GEO, this matters. Full benchmark https://www.talordata.com/products/serp-comparison

Elowen — Thu, 11 Jun 2026 07:32:37 +0000

SERP 6-Platform Data Quality Benchmark | TalorData

Cross-platform comparison of TalorData against 5 leading SERP API providers across 1,009 queries. See the data quality, accuracy, and field coverage advantages of TalorData.

talordata.com