DEV Community

LEO o
LEO o

Posted on

Stop Building SERP Scrapers From Scratch — Build a Keyword Rank Tracker in 10 Minutes

Let's be real: if you've ever tried to build a keyword rank tracker, you've probably gone through this cycle:
"How hard can it be? Just send a request to Google and parse the HTML"
Six hours later: you're dealing with IP blocks, CAPTCHAs, and selectors that broke because Google changed their class names yesterday
One week later: you're spending more time maintaining the scraper than building your actual product
Sound familiar? I've been there too. Today I want to show you a better way — using a SERP API to build a working keyword rank tracker in 10 minutes with Python.

Why Use a SERP API Instead of Building Your Own?
Let's get this out of the way: building your own SERP scraper makes sense only if you have extremely specialized requirements. For 99% of developers building SEO tools, competitive analysis dashboards, or marketing automation, a SERP API saves you time and money.
You don't have to:
Maintain a rotating proxy pool
Fix broken parsers every time Google updates their UI
Solve CAPTCHAs
Deal with rate limiting and blocks
A good SERP API gives you clean structured JSON with rankings, titles, links, and snippets — ready to drop straight into your database.
Quick Comparison: Top SERP APIs in 2025

Provider Pricing Model Free Tier Price per 1K (entry) Charges for failures?

SerpAPI Monthly subscription 100/mo $15.0 No
Serper.dev Pay-as-you-go 2,500 $1.0 No
DataForSEO Credits 5,000 credits $0.6 (standard) / $2.0 (real-time) Yes
Bright Data Pay-as-you-go Requires approval $2.1 No
TalorData Pay-as-you-go 1,000, no CC $1.0 No

For this tutorial, I'm usingTalorData because it's dead simple to integrate, has transparent pricing, and you can start testing without giving your credit card.

Let's Build the Rank Tracker!
Step 1: Get your API key
Head over to talordata.com and sign up — you get 1,000 free requests instantly. Copy your API token from the dashboard.
Step 2: The code

import requests
import sqlite3
from datetime import datetime
from typing import List, Dict

class KeywordRankTracker:
    def __init__(self, api_token: str):
        self.api_token = api_token
        self.endpoint = "https://serpapi.talordata.net/serp/v1/request"
        self.conn = sqlite3.connect('rank_tracker.db')
        self._create_table()

    def _create_table(self):
        cursor = self.conn.cursor()
        cursor.execute('''
            CREATE TABLE IF NOT EXISTS rankings (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                keyword TEXT,
                domain TEXT,
                rank INTEGER,
                checked_at TIMESTAMP
            )
        ''')
        self.conn.commit()

    def check_rank(self, keyword: str, target_domain: str, 
                   engine: str = "google", location: str = None) -> Dict:
        headers = {
            "Authorization": f"Bearer {self.api_token}",
            "Content-Type": "application/x-www-form-urlencoded"
        }
        data = {"engine": engine, "q": keyword, "json": "2"}
        if location:
            data["location"] = location

        response = requests.post(self.endpoint, headers=headers, data=data)
        response.raise_for_status()
        results = response.json()

        for result in results.get("organic_results", []):
            link = result.get("link", "")
            if target_domain in link:
                return {"keyword": keyword, "rank": result.get("position"), "found": True}
        return {"keyword": keyword, "rank": None, "found": False}

    def save_result(self, keyword: str, domain: str, rank: int):
        cursor = self.conn.cursor()
        cursor.execute('INSERT INTO rankings (keyword, domain, rank, checked_at) VALUES (?, ?, ?, ?)',
                      (keyword, domain, rank, datetime.now()))
        self.conn.commit()

    def batch_check(self, keywords: List[str], target_domain: str, location: str = None) -> List[Dict]:
        results = []
        for kw in keywords:
            print(f"Checking: {kw}...")
            res = self.check_rank(kw, target_domain, location=location)
            if res["found"]:
                self.save_result(kw, target_domain, res["rank"])
            results.append(res)
        return results


if __name__ == "__main__":
    API_TOKEN = "YOUR_API_TOKEN_HERE"
    tracker = KeywordRankTracker(API_TOKEN)

    TARGET_DOMAIN = "your-domain.com"
    KEYWORDS = ["best python seo tools", "how to build a rank tracker", "serp api for seo"]

    results = tracker.batch_check(KEYWORDS, TARGET_DOMAIN, location="United States")

    print("\n=== Ranking Check Summary ===")
    for res in results:
        if res["found"]:
            print(f"{res['keyword']}: Rank #{res['rank']}")
        else:
            print(f"{res['keyword']}: Not found in top results")

Enter fullscreen mode Exit fullscreen mode

How it works
When you run this script, it:
Loops through your list of keywords
Fetches fresh search results from TalorData
Checks if your domain is in the results
Saves the ranking to a SQLite database for historical tracking
Prints out a clean summary
Less than 100 lines of code. No proxies to manage, no parsing to debug.

Who Should Use What?
Side projects / startups: TalorData — 1k free requests without CC, simple API
Large-scale Google-only scraping: Serper.dev — unbeatable at 500k+/month
Batch processing with no latency requirements: DataForSEO — cheap standard queue
Enterprise with strict SLA: Bright Data — most mature infrastructure
Deep SERP feature extraction: SerpAPI — great SDK support, just pricey

Final Thoughts
Building internal tools doesn't mean you have to build everything from scratch. Offloading SERP scraping to a specialized API saves you hours of work, and for most projects, it's actually cheaper than maintaining your own infrastructure.
If you're starting a new SEO project or fed up with maintaining your own scraper, give TalorData a try:
👉 Sign up for free at talordata.com — 1,000 free requests, no credit card required.

Top comments (0)