Keyword research in Python: search volume, difficulty and AI Overviews in bulk (2026)

#python #seo #api #ai

In 2026 the first question about a keyword isn't only "how many searches?" but also "does Google answer it with an AI Overview?". Those keywords often send fewer clicks, and they're exactly the ones to target if you want to be cited in AI answers (AEO).

Google Keyword Planner is free, but it shows volume ranges unless you run active ads, and it has no keyword difficulty or AI Overview data. Ahrefs and Semrush have all of it, starting at $129/month and $139/month.

Here's how to get search volume, keyword difficulty, intent, CPC and the AI Overview flag for a whole list of keywords, paid per keyword, in a few lines of Python.

1. Setup

pip install "apify-client>=3"
export APIFY_TOKEN=...   # free account at console.apify.com

I'm using the Keyword Research Tool on Apify (disclosure: I built it). Search volume, CPC and the 12-month trend come from Google Ads data; difficulty, intent and SERP features come from a commercial SEO database. Keywords without data aren't charged.

2. A keyword list → CSV

import csv, os
from apify_client import ApifyClient

KEYWORDS = ["protein powder", "cold brew coffee", "keyword research tool", "standing desk", "how to make cold brew"]
COLUMNS = ["keyword", "searchVolume", "keywordDifficulty", "intents", "hasAiOverview"]

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jesting_grass/keyword-research-tool").call(run_input={"keywords": KEYWORDS, "country": "us"})

with open("keywords.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=COLUMNS, extrasaction="ignore")
    w.writeheader()
    for row in client.dataset(run.default_dataset_id).iterate_items():
        if row.get("searchVolume"):
            w.writerow({**row, "intents": ", ".join(row.get("intents") or [])})
            print(f'{row["keyword"]:<24} vol {row["searchVolume"]:>8,}  KD {row["keywordDifficulty"]:<4} AI Overview: {row["hasAiOverview"]}')

Real output (US, September 2026):

protein powder           vol  368,000  KD 72   AI Overview: True
cold brew coffee         vol   40,500  KD 50   AI Overview: True
keyword research tool    vol    4,400  KD 66   AI Overview: False
standing desk            vol  135,000  KD 74   AI Overview: False
how to make cold brew    vol   12,100  KD 48   AI Overview: True

Each row also has cpc (with a low/high bid range), adsCompetition, serpFeatures (featured snippet, People Also Ask, local pack…) and monthlySearches for the last 12 months, so you can spot seasonality.

3. Find easy wins from one seed keyword

Give it a seed and it returns related keyword ideas with the same metrics. Then filter for decent volume and low difficulty:

run = client.actor("jesting_grass/keyword-research-tool").call(run_input={
    "seedKeywords": ["cold brew coffee"],
    "ideasPerSeed": 100,
    "country": "us",
})
ideas = [r for r in client.dataset(run.default_dataset_id).iterate_items() if r.get("source") == "idea"]

easy = sorted((r for r in ideas if (r.get("searchVolume") or 0) >= 500 and (r.get("keywordDifficulty") or 100) <= 40),
              key=lambda r: -r["searchVolume"])
for r in easy[:10]:
    print(f'{r["keyword"]:<40} vol {r["searchVolume"]:>7,}  KD {r["keywordDifficulty"]}')

aio = [r for r in ideas if r.get("hasAiOverview")]
print(f"{len(aio)} of {len(ideas)} ideas trigger a Google AI Overview")

Real output:

coffee and cold brew                     vol  40,500  KD 32
brew cold brew coffee                    vol  40,500  KD 1
cold brew using french press             vol  22,200  KD 35
coarse ground coffee cold brew           vol  22,200  KD 15
grind for cold brewed coffee             vol  12,100  KD 33
cold brew coffee grind                   vol  12,100  KD 18
beans for cold brew                      vol   8,100  KD 26
coffee gourmet                           vol   6,600  KD 24
iced coffee from coffee maker            vol   6,600  KD 18
what is a cold brew coffee               vol   6,600  KD 37
43 of 64 ideas trigger a Google AI Overview

Two things stand out:

Two thirds of this niche gets an AI Overview. For a coffee blog, that changes the plan: those pages need to be written to be cited, not just to rank.
Google groups close variants. "coffee and cold brew" and "brew cold brew coffee" share the 40,500 of "cold brew coffee", so that's one page, not three. The grind keywords ("coarse ground coffee cold brew", "cold brew coffee grind") are a separate, lower-difficulty topic worth its own article.

4. No Python? One HTTP call

curl -X POST "https://api.apify.com/v2/acts/jesting_grass~keyword-research-tool/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords":["protein powder","cold brew coffee"],"country":"us"}'

It works for 17 countries (US, UK, Canada, Australia, the big EU markets, the Nordics, Brazil, Mexico, India), runs from n8n, Make and Zapier, and AI agents can call it through the Apify MCP server.

A note on the numbers

Every SEO tool estimates volume and difficulty its own way, so Ahrefs, Semrush and this will not match exactly. Volumes here are Google Ads 12-month averages. Compare keywords within one tool rather than across tools.

All examples (bulk CSV, easy-win finder, Node.js, cURL): github.com/emiohr/keyword-research-api. Questions or feature requests? Drop a comment.

This article was written by an AI agent under my direction, and all code was tested against the live API. Not affiliated with Ahrefs, Semrush or Google.

DEV Community