Gio Rich

Posted on Oct 3 Fully Autonomous

Getting Google Trends data in Python without fighting 429 errors

#webscraping #python #javascript #api

I built this actor; it's a paid tool on Apify with a free trial credit.

If you've used pytrends, you've probably seen 429 Too Many Requests. Google Trends throttles anything that looks automated, and a script that worked yesterday can fail today. There's no official public API for it either.

I'm an 18-year-old engineering student, and I built Google Trends Scraper mostly around that one problem. This post covers calling it from Python and Node, what the output looks like, and a small weekly keyword tracker.

How it deals with rate limits

Everything runs over plain HTTP, no headless browser. Each session first gets its own Google cookie, then makes one small flow per search (explore, then the data widgets). When Google returns a 429, that session is retired and the search retries on a fresh session and proxy IP with backoff and jitter. Results are saved per search as soon as they're ready, so one stubborn search never blocks the rest. A RUN_SUMMARY record lists anything that failed and why, and failed searches aren't charged.

In my tests, 9 searches plus 2 trending-now countries (29 results) took about 20 seconds.

Python example

pip install apify-client

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("rel8ble/google-trends-scraper").call(run_input={
    "searchTerms": ["air fryer", "chatgpt, gemini, claude"],
    "geo": "US",
    "timeRange": "today 12-m",
})

for r in client.dataset(run["defaultDatasetId"]).iterate_items():
    for s in r.get("keywordStats", []):
        print(s["keyword"], s["average"], s["changePercent"])

One line in searchTerms is one search. Put up to 5 comma-separated terms on a line to compare them on the same 0-100 scale.

Node example

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: "<YOUR_APIFY_TOKEN>" });

const run = await client.actor("rel8ble/google-trends-scraper").call({
  trendingNowGeos: ["US", "GB"],
  includeInterestOverTime: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const t of items.filter((i) => i.type === "trending")) {
  console.log(t.geo, t.rank, t.title, t.approxTraffic);
}

What comes back

One result per search, with every data type nested inside. A real result (US, past 12 months), arrays shortened:

{
  "type": "search",
  "searchTerm": "chatgpt, gemini, claude",
  "keywords": ["chatgpt", "gemini", "claude"],
  "geo": "US",
  "hasData": true,
  "keywordStats": [
    { "keyword": "chatgpt", "average": 77.6, "peakValue": 100, "peakDate": "2025-10-19T00:00:00.000Z", "latestValue": 76, "changePercent": -20.2 },
    { "keyword": "gemini", "average": 26.9, "peakValue": 35, "latestValue": 33, "changePercent": 20.9 },
    { "keyword": "claude", "average": 17.9, "peakValue": 34, "latestValue": 19, "changePercent": 263.6 }
  ],
  "interestOverTime": [
    { "date": "2026-09-13T00:00:00.000Z", "formattedTime": "Sep 13 – 19, 2026", "values": { "chatgpt": 76, "gemini": 33, "claude": 19 }, "isPartial": false }
  ],
  "interestByRegion": [
    { "geoCode": "US-CA", "geoName": "California", "values": { "chatgpt": 57, "gemini": 26, "claude": 17 } }
  ],
  "relatedQueriesRising": [
    { "query": "how to use chatgpt effectively", "keyword": "chatgpt", "formattedValue": "+2,050%", "isBreakout": false }
  ]
}

changePercent compares the average of the last quarter of the period to the first quarter, with partial (still-running) weeks left out. Trending-now rows look like this:

{ "type": "trending", "geo": "US", "rank": 1, "title": "dodgers schedule", "approxTraffic": "1000+", "news": [{ "title": "The teams no one wants to play in October", "source": "MLB.com" }] }

Use case: a weekly keyword tracker for content planning

Say you write about kitchen gear and want to know every week which products are gaining interest and which rising queries you should write about.

import csv
import datetime
import os
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("rel8ble/google-trends-scraper").call(run_input={
    "searchTerms": ["air fryer, instant pot, slow cooker", "espresso machine"],
    "geo": "US",
    "timeRange": "today 3-m",
    "includeInterestByRegion": False,
    "includeRelatedTopics": False,
})

today = datetime.date.today().isoformat()
new_file = not os.path.exists("trends-log.csv")
with open("trends-log.csv", "a", newline="", encoding="utf-8") as f:
    w = csv.writer(f)
    if new_file:
        w.writerow(["date", "search", "keyword", "average", "latest", "changePercent"])
    for r in client.dataset(run["defaultDatasetId"]).iterate_items():
        for s in r.get("keywordStats", []):
            w.writerow([today, r["searchTerm"], s["keyword"], s["average"], s["latestValue"], s["changePercent"]])
        for q in r.get("relatedQueriesRising", [])[:5]:
            print(f'rising for {q["keyword"]}: {q["query"]} ({q["formattedValue"]})')

Run it weekly. The CSV builds a history you can chart, and the printed rising queries are a ready-made list of article ideas. Remember that values are only comparable inside one search, which is why the three cookers share one line.

What it costs

$2.50 per 1,000 results. One result is one search (a comparison of up to 5 keywords counts as one, with timeline, regions and related queries included) or one trending-now search. Failed searches aren't charged.

The weekly tracker above: 2 results a week ≈ $0.02/month
1,000 searches = $2.50
Apify's free plan gives $5 of monthly credit, about 2,000 results

Limits

Up to 5 keywords per comparison (Google's own limit).
Values are Google's relative 0-100 index, not search volumes. 100 is the peak for that search.
Numbers can move a few points between requests, even in the browser. Google Trends works from a sample.
Related queries and topics only appear with enough volume. Related topics have become rare in Google's responses lately; you get an empty array, not an error. hasData: false means too little volume.
Trending now returns what Google's feed lists, typically about 10 searches per country.

If a run shows lots of "rate-limited, rotating session" warnings, that's handled; if searches still fail, switch the proxy to RESIDENTIAL or lower maxConcurrency. Anything else, use the Issues tab on the actor page.

Google Trends Scraper on Apify

This article was drafted with AI and published by me.

DEV Community