DEV Community

Ocius Data
Ocius Data

Posted on Fully Autonomous

Google Trends API in Python without pytrends 429 errors (and with more than 5 terms on one scale)

Disclosure: I build and maintain the Apify Actor used in this tutorial (ocius-data/google-trends-reliable). The method for comparing more than five terms works with any tool; the code below uses ours.

Google has never shipped a public Google Trends API. Most Python projects use pytrends, an unofficial library that calls the same endpoints as the website. It works well in a notebook and then breaks in production for two reasons:

  1. TooManyRequestsError (HTTP 429). Google rate-limits by IP address. A scheduled job on a cloud server hits the limit after a few requests, and the usual fix (time.sleep plus retries) makes every run slow and still unreliable.
  2. The five-term limit. Google Trends compares at most five terms per request, and every request is scaled to its own 0-100 range. If you request terms in two batches, "60" in batch A and "60" in batch B do not mean the same thing.

This tutorial shows how to get interest over time, interest by region and related queries from Python with a single function call, and how to compare 10, 50 or 500 terms on one common scale.

What you need

  • Python 3.11 or later.
  • A free Apify account and its API token (Console, Settings, API and Integrations). The free plan includes monthly platform credit, enough to follow this tutorial.
  • The official client (version 3.x) and pandas: pip install "apify-client>=3" pandas. In version 3.x, call() returns a Run object, so the dataset ID is run.default_dataset_id.

Step 1: interest over time for one keyword

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("ocius-data/google-trends-reliable").call(run_input={
    "searchTerms": ["air conditioner"],
    "geo": "US",
    "timeRange": "today 12-m",
})

rows = list(client.dataset(run.default_dataset_id).iterate_items())
df = pd.DataFrame(rows)
print(df["type"].value_counts())
Enter fullscreen mode Exit fullscreen mode

One run returns every data type in the same dataset, distinguished by the type field:

  • interestOverTime: one row per week (or day, or hour, depending on the range) with date, value and an isPartial flag for the current period.
  • interestByRegion: one row per state or country.
  • relatedQuery: top and rising related queries.

Retries, session rotation and the switch from datacenter to residential proxies happen inside the run, so your script never sees a 429. If a term returns no data, it is reported in the run summary with the status NO_DATA and is not charged.

Step 2: build a clean time series

iot = df[df["type"] == "interestOverTime"][["date", "value", "isPartial"]]
iot = iot[~iot["isPartial"]].set_index("date")
print(iot.tail())
Enter fullscreen mode Exit fullscreen mode

Dropping the partial period avoids the classic mistake of reading the current, incomplete week as a sudden drop.

Step 3: compare more than five terms on one scale

This is where most projects get stuck. The standard workaround is an anchor term: a term that appears in every batch of five. You measure how the anchor scores in each batch and rescale the other four terms by the same factor. The Actor does this for you when compare is on:

terms = ["coffee", "tea", "beer", "wine", "water", "juice", "soda",
         "kombucha", "energy drink", "sparkling water"]

run = client.actor("ocius-data/google-trends-reliable").call(run_input={
    "searchTerms": terms,
    "compare": True,
    "anchorTerm": "coffee",
    "geo": "US",
    "timeRange": "today 12-m",
    "includeInterestByRegion": False,
    "includeRelatedQueries": False,
})

rows = client.dataset(run.default_dataset_id).iterate_items()
df = pd.DataFrame([r for r in rows if r["type"] == "interestOverTime"])

wide = df.pivot_table(index="date", columns="searchTerm", values="valueComparable")
print(wide.mean().sort_values(ascending=False).round(1))
Enter fullscreen mode Exit fullscreen mode

valueComparable is a single 0-100 scale across all ten terms, where 100 is the highest interest of any term in the period. If the anchor term had very low interest in any batch, every row of the run is marked lowPrecision: true and the run log shows a warning; in that case, choose a more popular anchor instead of trusting a noisy number.

Choose a popular anchor. If your terms are niche ("heat pump water heater"), use a mid-volume term from the same topic rather than a giant term such as "weather", which would compress everything else toward zero.

Step 4: the searches trending right now

The same Actor reads the Google Trends "Trending now" page:

run = client.actor("ocius-data/google-trends-reliable").call(run_input={
    "mode": "trendingNow",
    "geo": "US",
    "trendingHours": "24",
    "maxTrendingSearches": 25,
})
for r in client.dataset(run.default_dataset_id).iterate_items():
    print(r["rank"], r["searchTerm"], r["searchVolumeLabel"], r["increasePercent"])
Enter fullscreen mode Exit fullscreen mode

Step 5: run it on a schedule

In Apify Console, save your input as a task and add a schedule (for example, every Monday at 07:00). Each run stores its dataset, and you can read the newest one from Python, push it to Google Sheets, or receive a webhook when the run finishes.

Cost

The Actor charges per search term that returns data, with all data types included (USD 0.01 per term at the time of writing; check the Actor page for current pricing). A weekly check of ten keywords costs about USD 0.10. Terms without data and failed terms are free.

When pytrends is still the right choice

If you run a handful of queries by hand from your own computer, pytrends is free and fine. Move to a managed run when you need scheduled jobs on a server, more than five comparable terms, or results you can rely on without babysitting retries.

Try it: Google Trends API on Apify. Questions or feature requests are welcome in the Actor's Issues tab.


Runnable versions of every example in this article: apify-data-apis-examples on GitHub.

This article was drafted with AI assistance and reviewed, with every code sample tested against the live API.

Top comments (0)