DEV Community

Cover image for Pull Google Trends for hundreds of keywords without the 429s
Fjord Labs
Fjord Labs

Posted on

Pull Google Trends for hundreds of keywords without the 429s

If you've used pytrends, you know the feeling: it works for five keywords, then Google answers 429 Too Many Requests and your notebook stalls. Google Trends has no official public API, and it rate-limits hard.

Here's the approach I use for batches of keywords and countries, where failures are retried from a different IP automatically.

What you can get per keyword

  • interest over time (any timeframe, from the past hour to 2004)
  • interest by region
  • related queries and topics (top and rising)

The code

I run a hosted scraper I maintain on Apify (Google Trends Scraper). Python client (pip install apify-client pandas):

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("fjordlabs/google-trends").call(run_input={
    "keywords": ["chatgpt", "gemini", "claude", "copilot"],
    "geos": ["US", "GB", "DE"],
    "timeframe": "past_12_months",
    "proxy": {"useApifyProxy": True},
})

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
ok = [i for i in items if not i.get("error")]
print(f"{len(ok)} of {len(items)} keyword/country pairs succeeded")
Enter fullscreen mode Exit fullscreen mode

Each successful keyword × country is one row, and rows that fail come back with an error field. You aren't charged for those, so you can just re-run the failures.

Why reliability matters here

When I checked the Apify Store, the most-used Google Trends scraper had about 44% of its public runs in the last 30 days end failed, aborted or timed out. Trends scraping is mostly a rate-limit problem. The fix is boring: back off, rotate IPs, retry the failed keyword, and never bill for an empty result.

Ideas

  • Keyword research: compare seasonality of 100 candidate topics before writing.
  • Product demand: track brand vs competitor interest by country, monthly.
  • AI agents: the Actor is callable through Apify's MCP server, so an agent can ask "is X growing in Germany?" and get real data.

Pricing is $0.002 per keyword result, so 100 keywords in one country is $0.20.

Disclosure: I built and maintain this scraper. Feedback welcome. Links to Apify include my referral code.

Top comments (0)