DEV Community

Cover image for Long-tail keyword research from Google Autocomplete, hundreds of seeds at once
Joshua Smith
Joshua Smith

Posted on

Long-tail keyword research from Google Autocomplete, hundreds of seeds at once

#seo #marketing #api #data

Type best crm for into Google and the drop-down shows ten to fifteen phrases real people search for. Keyword tools charge a subscription to collect those at scale. The endpoint behind the box is public and free; the work is in the expansion and de-duplication.

How the suggest endpoint works

https://suggestqueries.google.com/complete/search?client=chrome&q=best+crm+for&gl=us&hl=en returns JSON with the suggestions, their type and a relevance score. Tools get more out of it by expanding each seed:

  • Alphabet: best crm for a, best crm for b, ... best crm for z
  • Questions: how best crm for, what best crm for, why ...
  • Prepositions: best crm for vs, best crm for with, ... without
  • Numbers: best crm for 2, ... 10
  • Depth 2: re-query every suggestion found at depth 1

One seed with alphabet, question and preposition expansions at depth 2 is roughly 190 requests and can yield around 1,000 unique phrases. Send those too fast from one IP and you get 429s, so you need small delays, low concurrency and retries, plus per-seed de-duplication.

The two-minute version

Google Autocomplete Scraper does the expansion, throttling and de-duplication for any country and language.

  1. Enter your seed keywords in Seed keywords, one per line. Partial phrases work best.
  2. Set Country and Language for the market.
  3. Tick the expansions you want and optionally set Depth to 2. Click Start, then export CSV/Excel or send to Google Sheets.

From code 

curl -X POST "https://api.apify.com/v2/acts/josh99smith~google-autocomplete-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "keywords": ["best crm for", "how to start a podcast"], "country": "us", "language": "en", "expandQuestions": true, "maxSuggestionsPerKeyword": 50, "depth": 1 }'
Enter fullscreen mode Exit fullscreen mode

What you get back

One record per unique suggestion:

{
    "seedKeyword": "best crm for",
    "query": "what best crm for",
    "suggestion": "what's the best crm for small business",
    "position": 1,
    "relevance": 601,
    "suggestType": "QUERY",
    "modifierType": "question",
    "modifier": "what",
    "country": "us",
    "language": "en"
}
Enter fullscreen mode Exit fullscreen mode

relevance is Google's own score, useful for ranking suggestions within a seed. modifierType tells you which expansion produced the phrase, so question-style queries are one filter away.

Cost and limits

$0.003 per seed keyword, flat, whatever expansions and depth you enable and however many suggestions come back; 15 or 1,000 cost the same. A keyword whose requests fail on rate limiting or a network error is not charged and comes back with an errorType. Per-suggestion pricing elsewhere runs $0.0005 to $0.001 each, which for a 1,000-suggestion seed is $0.50 to $1.00 instead of $0.003.

Suggestions change by country, language and time, so two runs are not identical, and rare phrases may return nothing. No search result pages are scraped and no personal data is collected.

Use it from an AI agent

Add https://mcp.apify.com?tools=josh99smith/google-autocomplete-scraper to your MCP client and ask for suggestions around any phrase; the agent can group them into topics for you.

Disclosure: I built this Actor. Source: github.com/josh99smith/google-autocomplete-scraper.

Top comments (0)