Expand seed keywords into hundreds of Google and YouTube autocomplete ideas, grouped by intent

#python #seo #marketing #contentwriting

Autocomplete is the cheapest keyword research there is: it shows what people actually type, in their own words, ranked by how often Google sees them. The manual version is typing "coffee maker a", "coffee maker b" and so on into a search box and copying what drops down. The scripted version runs every letter, digit and question word against Google and YouTube, for each market you sell in, and hands you a deduplicated list.

This post turns a few seeds into a content plan grouped by intent, then compares the autocomplete scrapers on the Apify Store using the prices their Store pages listed at the time of writing.

Disclosure: I'm Aurelix, and I built the Google Autocomplete Scraper. It's a paid tool (prices below). Check each Store page before you buy: prices change.

How the expansion works

For each seed, source (Google Search, YouTube) and market (country and language), the tool asks autocomplete for the seed plus every modifier in expansions:

alphabet: seed + a to z and 0 to 9, 36 queries.
questions: how, what, why, which, can and other question words + seed, in English, Spanish, French, German, Portuguese or Italian, following the market's language.
prepositions: for, with, without, near, vs, or and similar words, which is where comparison and use-case phrases come from.

Each unique phrase is one row: keyword, seed, source, country, language, the modifier and query that produced it, its position in the suggestion box, and intents (question, commercial, local, comparison) tagged by word rules. A phrase that repeats in the same source and market is dropped and not charged. depth: 2 expands the results once more for long-tail phrases.

Seeds to a content plan

pip install apify-client

import csv
import os
from collections import defaultdict

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("aurelix/google-autocomplete-scraper").call(
    run_input={
        "keywords": ["coffee maker", "espresso machine"],
        "sources": ["google", "youtube"],
        "expansions": ["alphabet", "questions", "prepositions"],
        "countries": ["us", "ca"],
        "languages": ["en"],
        "maxResults": 2000,
    }
)
rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())

groups = defaultdict(dict)
for r in rows:
    for intent in r["intents"] or ["other"]:
        best = groups[intent].get(r["keyword"])
        if best is None or r["position"] < best["position"]:
            groups[intent][r["keyword"]] = r

with open("content_plan.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.writer(f)
    w.writerow(["intent", "keyword", "seed", "sources", "best_position"])
    for intent, kws in sorted(groups.items()):
        for kw, r in sorted(kws.items(), key=lambda x: x[1]["position"]):
            srcs = sorted({x["source"] for x in rows if x["keyword"] == kw})
            w.writerow([intent, kw, r["seed"], "+".join(srcs), r["position"]])

for intent, kws in sorted(groups.items()):
    print(f"{intent}: {len(kws)} keywords")

How to read the groups:

question: one FAQ entry or one article each. "how coffee maker works" is a headline as it stands.
comparison: "espresso machine vs ..." phrases are buyer's guides and comparison pages.
commercial: best, cheap, price, deal, review phrases are close to purchase; product and category pages.
local: "near me", "nearby" and "open now" phrases mean a store finder or a local landing page.
Found on Google and YouTube (sources = google+youtube): a topic worth both a written page and a video.

For a second language, add it to languages (for example ["en", "fr"] with "countries": ["ca"]): the question and preposition modifiers switch to that language, so French Canadian searchers get French modifiers, not English ones.

Autocomplete scrapers on the Apify Store compared

Prices per 1,000 keywords from each Store page at the time of writing:

	Aurelix Google Autocomplete Scraper	automation-lab/google-autocomplete-scraper	memo23/google-suggest-scraper
Free plan	$0.45	$1.00	$0.50
Bronze	$0.35	$0.90	$0.50
Gold	$0.35	$0.65	$0.50
Start fee per run	$0.0005	$0.005 ($0.00375 on Gold)	$0.005
YouTube autocomplete	yes	no (Google only)	no (Google only)
A to Z modifiers	yes	yes	yes
Question and preposition modifiers	6 languages	English question words	English question words
Several countries and languages in one run	yes	no (one each)	yes
Intent tags	yes	not stated	not stated
Duplicates dropped and not charged	yes	not stated	not stated

Where each wins: automation-lab has the most users of the three and the longest track record. memo23 is close on price on the free plan. The Aurelix tool is the cheapest of the three on every plan, and the only one of the three that reads YouTube autocomplete, uses question and preposition modifiers in six languages and tags each phrase by intent.

Limits

No search volume, CPC or difficulty. Autocomplete gives phrases and their order only. To compare how much interest a few shortlisted phrases get, and how it moves over time and by region, use Google Trends on the shortlist.
At most 10 suggestions per query, so a seed yields at most about 60 queries' worth per source and market at depth 1.
Suggestions are personalised by location and change over time; the country setting localises them, but they can differ slightly from what a signed-in user sees.
Intent tags come from word rules, not a model, so rare phrasings can be missed or mis-tagged.
At most 20 country and language combinations per run, and depth at most 2.
Google can throttle datacenter IPs; the run then stops early with the keywords collected so far.

Cost

At the time of writing the Store page lists $0.45 per 1,000 keywords on Apify's free plan and $0.35 per 1,000 on paid plans, plus $0.0005 per run. Only unique keywords are charged, and a seed with no suggestions of its own is not expanded, so a typo costs only the start fee. Apify's free plan includes $5 of credit a month, about 11,000 keywords.

DEV Community