DEV Community

Alom Dev
Alom Dev

Posted on

Google Trends in Python after pytrends died

#python #googletrends #datascience #seo

If you search for this error you'll find a lot of unhappy people:

pytrends.exceptions.TooManyRequestsError: The request failed: Google returned a response with code 429
Enter fullscreen mode Exit fullscreen mode

The usual advice is to add sleeps, then retries, then proxies. None of it really fixes it, because the repo was
archived in April 2025 and nobody is coming to fix it. Google changed how the Trends site loads its data, and the
library's pattern (lots of quick requests from one IP) gets rate limited almost right away now.

I needed Trends data for a project anyway, so I went down the rabbit hole. Here's what I found, in case you're in the
same spot.

Quick disclosure before anything else: I ended up building a hosted scraper for this and I use it below. I'll try to
be fair about the other options, because for some people they're the better choice.

The options, honestly

Google's official API. Google announced an official Trends API in 2025. It's in alpha and you have to apply.
If you get access, use it. Last time I checked it was still invite-only.

Write it yourself. The site talks to internal JSON endpoints: /trends/api/explore gives you widget tokens,
then you call /trends/api/widgetdata/... with each token. I did this, and it works, but expect to deal with:

  • the first request getting a 429 on purpose before Google sets the NID cookie
  • tokens that only work for the exact parameters they were issued for
  • IP blocks. From a laptop you hit them in minutes. Datacenter IPs get blocked in ranges, so you need rotating proxies and some fallback logic
  • the )]}' prefix on every response, and parameters that change without notice

It took me a few days to get it stable, and it still needs fixing every couple of months.

Use something hosted. That's what the rest of this post does. I use my Google Trends Scraper on Apify
from Python with the official client. It's $1.50 per 1,000 search terms on the free plan and goes down to $0.50 on
the bigger plans. Proxies are included, and Apify's free plan gives you some monthly credit, which is enough to play with.

Setup 

pip install "apify-client>=3" pandas
export APIFY_TOKEN=...   # Apify Console > Settings > API & Integrations
Enter fullscreen mode Exit fullscreen mode

This helper runs the scraper and loads the flat CSV it saves (one row per data point, which is what I usually want
for pandas):

import io
import os

import pandas as pd
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

def google_trends(**run_input) -> pd.DataFrame:
    """Run the Google Trends scraper and return one row per data point."""
    run = client.actor("alom/google-trends-scraper").call(run_input=run_input)
    record = client.key_value_store(run.default_key_value_store_id).get_record("RESULTS_FLAT.csv")
    value = record["value"]
    text = value.decode("utf-8") if isinstance(value, (bytes, bytearray)) else value
    return pd.read_csv(io.StringIO(text))
Enter fullscreen mode Exit fullscreen mode

Interest over time

This is the one most people used pytrends for:

df = google_trends(searchTerms=["web scraping"], geo="US", timeRange="today 12-m")

timeline = df[df.section == "timeline"][["date", "value"]]
timeline["date"] = pd.to_datetime(timeline["date"])
print(timeline.tail())
Enter fullscreen mode Exit fullscreen mode

Time ranges are the same strings the website uses: now 1-H, now 7-d, today 3-m, today 5-y, all. For exact
dates use customTimeRange="2024-01-01 2024-12-31".

Comparing terms

Trends values are 0 to 100 inside one request, so two separate requests aren't comparable. Put the terms in one
comparison instead:

df = google_trends(searchTerms=["tea, coffee, matcha"], isMultiple=True, geo="GB", timeRange="today 3-m")

wide = (df[df.section == "timeline"]
        .pivot_table(index="date", columns="term", values="value"))
print(wide.tail())
Enter fullscreen mode Exit fullscreen mode

Google caps this at 5 terms. If you need more, there's a normalizeAcrossQueries option that chains several 5-term
comparisons through one shared anchor term and rescales them onto one scale. Pick your most searched term as the
anchor. With a tiny anchor the rounding gets ugly, and the output flags that with lowPrecision. It costs a bit
extra per comparison because it really is several requests under the hood.

States, cities and US metro areas

pytrends gave you one resolution per call. Here you get states, cities and metro areas (DMA) in the same result:

df = google_trends(searchTerms=["coffee"], geo="US")

metros = (df[df.section == "metro"]
          .sort_values("value", ascending=False)[["label", "code", "value"]])
print(metros.head(10))
Enter fullscreen mode Exit fullscreen mode

The section column is one of timeline, country, subregion, city, metro, relatedQueryTop,
relatedQueryRising.

Related queries 

df = google_trends(searchTerms=["web scraping"], geo="US")

rising = df[df.section == "relatedQueryRising"][["label", "formattedValue"]]
print(rising)
Enter fullscreen mode Exit fullscreen mode

One thing that confused me: rising queries are sometimes weird. For "coffee" I once got "pet care tips". I assumed it
was a parsing bug and spent an evening on it. It wasn't. The Trends website shows the exact same list. Google's data
is just like that sometimes.

Trending Now

The newer page at trends.google.com/trending never had a pytrends equivalent. This gets the list per country (or US
state) with search volume:

df = google_trends(mode="trendingNow", trendingGeos=["US", "GB"], trendingHours="24",
                   trendingSort="volume", maxTrendsPerGeo=25)

print(df[["geo", "rank", "title", "searchVolume", "increasePercentage", "categories"]].head(10))
Enter fullscreen mode Exit fullscreen mode

Schedule it hourly with onlyNewSinceLastRun=True and each run only returns trends you haven't seen yet. It's
basically a cheap alert feed.

If you'd rather have JSON 

run = client.actor("alom/google-trends-scraper").call(run_input={"searchTerms": ["web scraping"], "geo": "US"})
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item["searchTerm"], len(item["interestOverTime_timelineData"]), "points")
Enter fullscreen mode Exit fullscreen mode

One limitation

Related topics come back empty. Google only shows them to signed-in users now, so pytrends and every logged-out
tool lose them too. Related queries still work fine.

So which one?

If you need a couple of charts, just download the CSV from the Trends website, honestly. If you get into the official
API alpha, use that. If you have something scheduled or bulk, or it sits inside a pipeline, paying someone else to
deal with the 429s is cheaper than doing it yourself, at least it was for me.

Top comments (0)