Pulling Google Trends data in Python without constantly hitting 429s

If you've ever used pytrends, you know the arc. The first three requests work. You get excited, wrap it in a loop over 40 keywords, and somewhere around keyword seven Google starts answering with HTTP 429. You add time.sleep(5). Then time.sleep(30). Then you start reading Stack Overflow threads from 2019 about rotating user agents.

pytrends itself is archived now (the GitHub repo is read-only, last push in August 2024), so the duct tape is all yours. I ended up building my own Google Trends actor on Apify, and most of the work went into exactly this: not babysitting retries. This post is what I learned about why the 429s happen, and how I pull interest over time, regional data and trending searches from Python now. The actor is mine (Google Trends Scraper & API), so read this with that in mind.

Why Google Trends rate-limits you so fast

Google has no public Trends API. What everyone uses (pytrends, my actor, every other scraper) are the JSON endpoints the Trends website calls in your browser. And one keyword isn't one request. You first hit an "explore" endpoint that hands back widget tokens, then make a separate request per widget: one for the timeline, one for regions, one for related queries. So 40 keywords is well over 100 requests, all from the same IP, in a short burst. That's exactly the pattern a rate limiter is built to stop.

In my cloud tests the numbers looked like this. One keyword ("air fryer", US) on a datacenter proxy took 12 HTTP requests, and 4 of those were 429s. A 5-keyword batch through residential proxies hit 8 rate limits before finishing. Both runs succeeded, but only because every 429 got a new IP and a backoff instead of a crash.

What the actor does on a 429 is boring, which is the point. It rotates to a fresh proxy session, waits with exponential backoff and some random jitter (starting around 2 seconds, capped at a minute), and retries, up to maxRetries times (8 by default). Datacenter proxies go first because they're fast and cheap. After two blocks in a row, it switches the rest of the run to residential proxies (residentialFallback). If a query still fails, you get a row with status: "failed", an error and a hint, and you're not charged for it. I care a lot about that last part. A silent gap in a time series is much worse than a loud error.

The part nobody tells you: USER_TYPE_SCRAPER

This is the most interesting thing I found while building it, and also the most annoying.

Google tags each Trends session with a user type. Automated sessions get USER_TYPE_SCRAPER. I expected datacenter IPs to get flagged. I did not expect residential IPs to get flagged too, and I really didn't expect a real headless Chrome to get flagged. All three did, in October 2026.

The flag doesn't seem to touch the main data. Timelines, regional values, top related queries and trending searches matched what a normal browser shows. But two things change. Related topics get withheld completely. And "rising" related queries get decoys mixed in. For "air fryer" I got rising queries like "hotel booking", "coffee grinder" and "laptop stand". For "iphone" I got "iphone 6s to buy" and "iphone 3gs to buy" as Breakout terms, which would be a fun market insight if it were true.

So the actor tells you when this happened (every keyword row has googleUserType and a dataQualityWarning) and runs a decoy filter on rising queries (risingFilter, strict by default). Suspicious entries are moved into suspectedDecoys with a reason, not deleted, so you can check what was removed. In one test with three keywords, it kept 47 rising queries and moved 28. I'd still call rising queries experimental. Top queries, timelines and regions are the data I'd actually build on.

Setup

pip install "apify-client>=3" pandas
export APIFY_TOKEN=your_token_here   # Apify Console > Settings > API & Integrations

Pricing: $0.01 per run plus $1.50 per 1,000 results. A result is one keyword with its timeline, regions and related queries, or one trending search. So 100 keywords is about $0.16. Proxies are included; you don't pay for them separately.

Interest over time

Start with one keyword and look at what comes back:

import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
ACTOR = "ivora/google-trends-api"

run = client.actor(ACTOR).call(run_input={
    "searchTerms": ["air fryer"],
    "geo": "US",
    "timeRange": "today 3-m",
})
row = next(client.dataset(run.default_dataset_id).iterate_items())

print(row["status"], row["googleUserType"])
print(row["interestOverTimeSummary"]["air fryer"])
print([q["query"] for q in row["relatedQueries"]["air fryer"]["top"][:4]])

From my test run:

ok USER_TYPE_SCRAPER
{'average': 54.92, 'max': 100, 'min': 37, 'latest': 42, 'peakDate': '2026-08-12', 'changePercent': -9.3, 'direction': 'stable'}
['air fryer chicken', 'chicken in air fryer', 'air fryer oven', 'ninja air fryer']

Yes, Americans search "air fryer chicken" and "chicken in air fryer" as two separate things. No, I can't explain it.

The summary block (average, peak date, latest value, % change, direction) is computed by the actor so you don't have to redo it for every term. The raw timeline is in interestOverTime, a list of {"date": ..., "values": {term: value}} points.

For a batch, here's how I turn it into one DataFrame:

import pandas as pd

terms = ["air fryer", "standing desk", "protein powder"]
run = client.actor(ACTOR).call(run_input={
    "searchTerms": terms,
    "geo": "US",
    "timeRange": "today 12-m",
    "maxRetries": 8,
})

series = {}
for it in client.dataset(run.default_dataset_id).iterate_items():
    if it.get("status") != "ok":
        print("failed:", it.get("searchTerm"), it.get("error"), it.get("hint"))
        continue
    points = it["interestOverTime"]
    for term in it["interestOverTimeSummary"]:
        series[term] = pd.Series(
            {p["date"]: p["values"][term] for p in points}, name=term
        )

df = pd.DataFrame(series)
df.index = pd.to_datetime(df.index)
print(df.tail())
print(df.idxmax())  # peak week per term

One thing that trips people up: in the default comparisonMode: "separate", each term is its own query and Google scales each one to 0-100 on its own. "Protein powder at 80" and "air fryer at 80" do not mean the same search volume. If you want them on one scale, like the compare box on the Trends website, use "comparisonMode": "compareAll". That compares up to 5 terms per query.

The results can be humbling. I compared "iphone", "samsung galaxy" and "google pixel" in the US over 12 months. iPhone averaged 62.6. Samsung Galaxy averaged 4.32. Google Pixel averaged 1.36, with a peak of 2.

run = client.actor(ACTOR).call(run_input={
    "searchTerms": ["iphone", "samsung galaxy", "google pixel"],
    "comparisonMode": "compareAll",
    "geo": "US",
    "timeRange": "today 12-m",
})
row = next(client.dataset(run.default_dataset_id).iterate_items())
summary = pd.DataFrame(row["interestOverTimeSummary"]).T
print(summary[["average", "max", "latest", "direction"]])

Interest by region

interestByRegion comes in the same row. Worldwide queries give you countries. With a country in geo, you get states or regions by default, and you can ask for cities or US metros with regionResolution (REGION, CITY, DMA). Google doesn't always have city data for smaller terms, so the actor falls back to the next level and tells you which one it used in interestByRegionResolution. That fallback exists because an empty city list looks exactly like a bug in your own code, and it usually isn't.

run = client.actor(ACTOR).call(run_input={
    "searchTerms": ["air fryer"],
    "geo": "US",
    "timeRange": "today 3-m",
    "regionResolution": "REGION",
})
row = next(client.dataset(run.default_dataset_id).iterate_items())

regions = pd.DataFrame(
    [{"code": r["geoCode"], "name": r["geoName"], "value": r["values"]["air fryer"]}
     for r in row["interestByRegion"]]
).sort_values("value", ascending=False)

print(row["interestByRegionResolution"])
print(regions.head(5).to_string(index=False))

In my run, Oklahoma came out at 100, Kentucky at 91 and Louisiana at 89. Same caveat as before: these values are relative. 100 means "highest share of searches for this term among all states", not "most searches". Small states win this a lot.

If you need low-volume regions too (Google returns them as 0), set includeLowVolumeRegions. And if you want YouTube or Google Shopping search instead of web search, there's property (youtube, froogle, and so on). Yes, Google Shopping is still called "froogle" in the API. Some things never get renamed.

Trending searches

"Trending now" is a different mode and a much lighter request. My test across the US, Ukraine and Germany returned 60 trends in 4.3 seconds on a datacenter IP, with no 429s and no scraper weirdness at all.

run = client.actor(ACTOR).call(run_input={
    "mode": "trending",
    "trendingGeos": ["US", "GB", "DE"],
    "trendingHours": "24",
    "trendingActiveOnly": False,
    "trendingMaxItems": 20,
})

trends = pd.DataFrame(client.dataset(run.default_dataset_id).iterate_items())
cols = ["geo", "rank", "term", "searchVolumeFormatted", "growthPercent", "isActive", "categories"]
print(trends[cols].sort_values(["geo", "rank"]).groupby("geo").head(3).to_string(index=False))

Each row has the volume bucket ("100K+"), growth %, start time, whether it's still active, related queries, categories and a link to explore it. You can also filter by category with trendingCategories.

Germany's number one trend in my run was "wetter morgen" (tomorrow's weather), 100K+ searches. Which is either a deep cultural insight or just October. In the US it was a wall of baseball and NFL names, with "did messi retire" somewhere in the top five.

This mode is the one I schedule hourly or daily. The explore mode doesn't have an "only changes" option, since every run returns the full current data, which is usually what you want for a trend report anyway.

A few gotchas I hit

The timezoneOffsetMinutes input maps to Google's tz parameter, and Google's sign is inverted. UTC+3 (Kyiv, where I live) is -180. It's the kind of detail that quietly shifts every hourly chart by three hours and nobody notices for a week.

Custom date ranges use customTimeRange with the format "2024-01-01 2024-12-31". Google picks the granularity (daily, weekly, monthly) based on the length of the range, not you. Long ranges give you monthly points whether you like it or not.

If you already have a Trends URL from the browser, paste it into startUrls. The actor takes its terms, geo, date, category and search type exactly as they are, which is handy when someone on the team says "can you pull this one" and sends a link.

And set a budget. maxItems stops the run after N results, and the Apify client lets you cap spend per run:

from decimal import Decimal

run = client.actor(ACTOR).call(
    run_input={"searchTerms": terms, "geo": "US"},
    max_total_charge_usd=Decimal("0.50"),
)

If you just want to keep using pytrends for a handful of keywords, that's fair. Keep requests slow, cache everything, and accept that some days Google will simply not talk to your IP. Once I needed a few hundred keywords on a schedule, maintaining my own retry and proxy logic stopped being fun, which is how this actor happened. It's on Apify Store if you want to try it on your own keywords.