Keyword research in Python: search volume, difficulty and AI Overviews in bulk (2026)

#python #seo #api #ai

In 2026 the first question about a keyword isn't only "how many searches?" but also "does Google answer it with an AI Overview?". Those keywords often send fewer clicks, and they're exactly the ones to target if you want to be cited in AI answers (AEO).

Google Keyword Planner is free, but it shows volume ranges unless you run active ads, and it has no keyword difficulty or AI Overview data. Ahrefs and Semrush have all of it, starting at $129/month and $139/month.

Here's how to get search volume, keyword difficulty, intent, CPC and the AI Overview flag for a whole list of keywords, paid per keyword, in a few lines of Python.

1. Setup

pip install "apify-client>=3"
export APIFY_TOKEN=...   # free account at console.apify.com

I'm using the Keyword Research Tool on Apify (disclosure: I built it). Search volume, CPC and the 12-month trend come from Google Ads data; difficulty, intent and SERP features come from a commercial SEO database. Keywords without data aren't charged.

2. A keyword list → CSV

import csv, os
from apify_client import ApifyClient

KEYWORDS = ["protein powder", "cold brew coffee", "keyword research tool", "standing desk", "how to make cold brew"]
COLUMNS = ["keyword", "searchVolume", "keywordDifficulty", "intents", "hasAiOverview"]

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jesting_grass/keyword-research-tool").call(run_input={"keywords": KEYWORDS, "country": "us"})

with open("keywords.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=COLUMNS, extrasaction="ignore")
    w.writeheader()
    for row in client.dataset(run.default_dataset_id).iterate_items():
        if row.get("searchVolume"):
            w.writerow({**row, "intents": ", ".join(row.get("intents") or [])})
            print(f'{row["keyword"]:<24} vol {row["searchVolume"]:>8,}  KD {row["keywordDifficulty"]:<4} AI Overview: {row["hasAiOverview"]}')

Real output (US, September 2026):

protein powder           vol  368,000  KD 32   AI Overview: True
cold brew coffee         vol   40,500  KD 23   AI Overview: True
keyword research tool    vol    4,400  KD 81   AI Overview: True
standing desk            vol  135,000  KD 38   AI Overview: False
how to make cold brew    vol   12,100  KD 4    AI Overview: True

Each row also has cpc (with a low/high bid range), adsCompetition, serpFeatures (featured snippet, People Also Ask, local pack…) and monthlySearches for the last 12 months, so you can spot seasonality.

3. Find easy wins from one seed keyword

Give it a seed and it returns related keyword ideas with the same metrics. Then filter for decent volume and low difficulty:

run = client.actor("jesting_grass/keyword-research-tool").call(run_input={
    "seedKeywords": ["cold brew coffee"],
    "ideasPerSeed": 100,
    "country": "us",
})
ideas = [r for r in client.dataset(run.default_dataset_id).iterate_items() if r.get("source") == "idea"]

easy, groups = [], set()
for r in sorted(ideas, key=lambda r: -(r.get("searchVolume") or 0)):
    # Google reports close variants with identical numbers: keep one keyword per group
    group = (r.get("searchVolume"), r.get("cpc"), r.get("keywordDifficulty"))
    if (r.get("searchVolume") or 0) >= 500 and (r.get("keywordDifficulty") or 100) <= 40 and group not in groups:
        groups.add(group)
        easy.append(r)
for r in easy[:10]:
    print(f'{r["keyword"]:<40} vol {r["searchVolume"]:>7,}  KD {r["keywordDifficulty"]}')

aio = [r for r in ideas if r.get("hasAiOverview")]
print(f"{len(aio)} of {len(ideas)} ideas trigger a Google AI Overview")

Real output:

cold brew coffee maker                   vol  49,500  KD 1
coffee for cold brew                     vol  40,500  KD 23
stōk cold brew coffee                    vol  40,500  KD 4
stok cold brew coffee                    vol  40,500  KD 2
cold brew coffee stok                    vol  40,500  KD 1
cold brew coffee from starbucks          vol  33,100  KD 13
starbucks cold brew coffee               vol  33,100  KD 7
how make cold brew coffee                vol  27,100  KD 11
how to brew cold coffee                  vol  27,100  KD 23
nitro cold brew coffee starbucks         vol  27,100  KD 20
33 of 100 ideas trigger a Google AI Overview

Two things stand out:

A third of this niche gets an AI Overview. For a coffee blog, those pages need to be written to be cited in the AI answer, not just to rank.
Google groups close variants. Keywords like "cold brew coffee maker" and "coffee cold brew maker" come back with identical numbers, which is why the code keeps one per group. Treat each group as one topic and write one page for it; the brand queries (Stōk, Starbucks) are better left to the brands.

4. No Python? One HTTP call

curl -X POST "https://api.apify.com/v2/acts/jesting_grass~keyword-research-tool/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords":["protein powder","cold brew coffee"],"country":"us"}'

It works for 17 countries (US, UK, Canada, Australia, the big EU markets, the Nordics, Brazil, Mexico, India), runs from n8n, Make and Zapier, and AI agents can call it through the Apify MCP server.

A note on the numbers

Every SEO tool estimates volume and difficulty its own way, so Ahrefs, Semrush and this will not match exactly. Volumes here are Google Ads 12-month averages. Compare keywords within one tool rather than across tools.

All examples (bulk CSV, easy-win finder, Node.js, cURL): github.com/emiohr/keyword-research-api. Questions or feature requests? Drop a comment.

This article was written by an AI agent under my direction, and all code was tested against the live API. Not affiliated with Ahrefs, Semrush or Google.

DEV Community