Google autocomplete for keyword research in Python: A-Z, question and YouTube suggestions in bulk (2026)

#python #seo #keywordresearch #tutorial

Autocomplete is the closest thing to free search demand data you can get. Every suggestion Google, YouTube or Bing shows is something real people type often enough for the engine to offer it, and the endpoints that power those dropdowns return plain JSON with no API key. Tools like AnswerThePublic and keyword.io are mostly this idea plus a nice UI.

This post covers the three endpoints, the expansion trick that turns 10 suggestions into hundreds, and a small script that merges everything into one ranked CSV.

The endpoints

Google (and YouTube and Google Shopping, same host):

https://suggestqueries.google.com/complete/search?client=chrome&hl=en&gl=us&q=standing+desk

client=chrome returns JSON shaped like ["query", ["suggestion 1", ...], [descriptions], [], {"google:suggesttype": ["QUERY", "NAVIGATION", ...], "google:suggestrelevance": [...]}]. The type list lets you drop navigational results (URLs) and keep real queries.
ds=yt switches to YouTube suggestions, ds=sh to Google Shopping. YouTube ideas skew toward "how to", reviews and comparisons, which is exactly what you want for video or tutorial content.
hl is the language and gl the country. Suggestions differ a lot by market, so set both.
Decode the body as UTF-8 yourself. With client=chrome the response can come back without a charset, and non-English suggestions get mangled if you trust the default.

Bing:

https://api.bing.com/osjson.aspx?query=standing+desk&mkt=en-US

This is the OpenSearch suggestion format: ["query", ["suggestion 1", ...]]. Bing's list is noticeably different from Google's, so it's worth a second source.

Turning 10 suggestions into 1,000

Each request only returns up to about 10 suggestions. The standard trick is to query variations of the seed and merge the results:

A-Z: standing desk a, standing desk b, ... standing desk z. Each letter surfaces a different slice of long-tail phrases.
Question prefixes: how standing desk, is standing desk, why standing desk. These give you the "is a standing desk worth it" style queries that make good FAQ sections and blog posts.
Modifier suffixes: vs, for, without, near me surface comparisons, audiences and local intent.

Then dedupe (lowercase and collapse whitespace) and keep track of which engines suggested each idea. A phrase that Google, YouTube and Bing all suggest is a stronger signal than one that shows up once.

The script

Only dependency: pip install requests.

import csv, json, string, sys, time
import requests

UA = {"User-Agent": "Mozilla/5.0 (autocomplete-tutorial/1.0)"}
PREFIXES = ["how", "what", "why", "can", "is", "best"]  # question-style ideas
SUFFIXES = ["vs", "for", "without", "near me"]

def google(q, hl="en", gl="us", ds=None):
    p = {"client": "chrome", "ie": "utf-8", "oe": "utf-8", "hl": hl, "gl": gl, "q": q}
    if ds:
        p["ds"] = ds  # "yt" = YouTube, "sh" = Google Shopping
    r = requests.get("https://suggestqueries.google.com/complete/search", params=p, headers=UA, timeout=20)
    r.raise_for_status()
    data = json.loads(r.content.decode("utf-8"))
    types = (data[4] if len(data) > 4 else {}).get("google:suggesttype", [])
    return [s for i, s in enumerate(data[1]) if i >= len(types) or types[i] == "QUERY"]

def bing(q, mkt="en-US"):
    r = requests.get("https://api.bing.com/osjson.aspx", params={"query": q, "mkt": mkt}, headers=UA, timeout=20)
    r.raise_for_status()
    return r.json()[1]

ENGINES = {"google": google, "youtube": lambda q: google(q, ds="yt"), "bing": bing}

def expand(seed):
    yield seed
    for c in string.ascii_lowercase:
        yield f"{seed} {c}"
    for w in PREFIXES:
        yield f"{w} {seed}"
    for w in SUFFIXES:
        yield f"{seed} {w}"

def ideas(seed, engines=("google", "youtube", "bing"), pause=0.3):
    seen = {}
    for engine in engines:
        for q in expand(seed):
            try:
                results = ENGINES[engine](q)
            except requests.RequestException as e:
                print(f"  {engine} {q!r}: {e}", file=sys.stderr)
                time.sleep(5)
                continue
            for pos, s in enumerate(results, 1):
                k = " ".join(s.lower().split())
                row = seen.setdefault(k, {"keyword": k, "engines": set(), "best_position": pos, "hits": 0})
                row["engines"].add(engine)
                row["best_position"] = min(row["best_position"], pos)
                row["hits"] += 1
            time.sleep(pause)
    return sorted(seen.values(), key=lambda r: (-len(r["engines"]), -r["hits"], r["best_position"]))

if __name__ == "__main__":
    seed = " ".join(sys.argv[1:]) or "standing desk"
    t = time.time()
    rows = ideas(seed)
    with open("keywords.csv", "w", newline="") as f:
        w = csv.writer(f)
        w.writerow(["keyword", "engines", "hits", "best_position"])
        for r in rows:
            w.writerow([r["keyword"], "+".join(sorted(r["engines"])), r["hits"], r["best_position"]])
    print(f"{len(rows)} unique ideas for {seed!r} in {time.time() - t:.1f} s -> keywords.csv")
    for r in rows[:10]:
        print(f'{r["keyword"]:<45} {"+".join(sorted(r["engines"])):<20} hits={r["hits"]}')

Results

I ran python3 autocomplete_ideas.py standing desk on October 20, 2026. That's 37 queries per engine, 111 requests in total with a 0.3 s pause between them:

1,013 unique keyword ideas in 57 seconds.
73 of them were suggested by all three engines, for example "standing desk with drawers", "standing desk topper", "standing desk treadmill" and "standing desk benefits".
Each engine contributed a lot of its own: 286 ideas only from Google, 275 only from YouTube and 212 only from Bing. Using just one engine would have missed roughly two thirds of the list.
The question prefixes produced usable content ideas straight away: "is standing desk worth it", "is standing desk better than sitting", "is standing desk hsa eligible", "what standing desk to buy".
One Bing request failed with a transient SSL error. The script logs it, waits 5 seconds and moves on, which is why the error handling is there.

Things that will bite you at scale

Rate limits: a few hundred requests from one IP is fine. Thousands in a row from a single IP will start getting HTTP 429 or a captcha page instead of JSON (the response stops parsing as JSON, which is the easiest way to detect it). Slow down, add retries with backoff, or spread requests across IPs.
Personalization: suggestions vary with language, country and sometimes location. Pin hl, gl and mkt so runs are comparable over time.
No volume numbers: autocomplete tells you a query is popular enough to suggest, not how many searches it gets. Use the engine count, the number of expansions an idea appears in, and its position as a relative signal, then check the top candidates in a volume tool if you need numbers.
Duplicates across expansions: "standing desk with drawers" comes back from the seed, from "w" and from "with". Merge before you count, otherwise your favourite ideas look more important than they are.

Hosted version

Disclosure: I'm with Datawright Labs, and we packaged this (Google, YouTube, Google Shopping and Bing, A-Z, 0-9 and question expansions, any language and country, recursive depth, must-contain and exclude-word filters, retries and proxy rotation for big seed lists) as a hosted tool on Apify: Autocomplete Keyword Scraper. It costs $0.20 per 1,000 unique suggestions, with duplicates merged before you pay, so Apify's free monthly credit covers most small jobs. You can run it on a schedule or call it by API. The script above is the whole idea, though, so use whichever fits.

DEV Community