If you run a site, you want to know where it ranks for the keywords that matter. Scraping search pages yourself breaks often, so this tutorial uses a SERP API that returns results as JSON.
Disclosure: this post is from the BestSERP team, and the examples use our API. Any SERP API with a similar response works the same way.
1. Get an API key
Sign up at bestserp.com (3,000 free searches, no card needed) and create an API key in the dashboard. Keep it in an environment variable:
export BESTSERP_API_KEY="your_key_here"
2. Run one search
import os
import requests
API_KEY = os.environ["BESTSERP_API_KEY"]
def search(q, gl="us", hl="en"):
r = requests.post(
"https://api.bestserp.com/search",
headers={"X-API-KEY": API_KEY, "Content-Type": "application/json"},
json={"q": q, "gl": gl, "hl": hl},
timeout=30,
)
r.raise_for_status()
return r.json()
Each item in the organic list has position, title, link, snippet and domain.
3. Find your domain's position
def rank_of(domain, q, gl="us"):
for item in search(q, gl=gl).get("organic", []):
if domain in item["link"]:
return item["position"], item["link"]
return None, None
keywords = ["serp api", "google search api", "serper alternative"]
for kw in keywords:
pos, url = rank_of("example.com", kw)
print(f"{kw!r}: {pos or 'not in top 10'} {url or ''}")
4. Save the results every day
import csv
import datetime
def track(domain, keywords, path="ranks.csv"):
today = datetime.date.today().isoformat()
with open(path, "a", newline="") as f:
w = csv.writer(f)
for kw in keywords:
pos, url = rank_of(domain, kw)
w.writerow([today, kw, pos or "", url or ""])
track("example.com", keywords)
Then schedule the script with cron, for example every morning at 6:00:
0 6 * * * /usr/bin/python3 /path/to/track.py
Each keyword costs one search, so tracking 50 keywords a day is about 1,500 searches a month.
Going further
- Change
glandhlto track rankings in other countries and languages. - Load
ranks.csvinto a spreadsheet or pandas to chart how each keyword moves over time. - The same request shape works for news, images, videos, shopping, places and maps results. See the BestSERP docs for the full parameter list.
Questions or ideas for the next tutorial? Leave a comment below.
Top comments (0)