Google Trends has two limits that come up the moment you use it for real work:
- You can compare at most 5 terms at once.
- For any range longer than about 9 months, you get weekly or monthly points, not daily ones.
Both limits have the same cause, and the same fix. Every response from Google is scaled 0-100 against its own peak. A "50" in one request and a "50" in another mean nothing relative to each other. You can still join requests, though, as long as they share something you can measure in both. Then you rescale one to match the other.
I spent a while getting this right for a scraper I built. Here's what works and what doesn't.
Comparing more than 5 keywords
The usual advice is to put one anchor keyword in every batch of 5 and rescale each batch by the anchor's total. It works until your keywords have very different sizes.
Google rounds every value to a whole number. Compare "coffee" with "kombucha" and kombucha comes back as 0 or 1 on most days, so most of its signal is lost. If coffee is your anchor, every small keyword gets crushed. If kombucha is your anchor, coffee overflows and you can't rescale it at all.
The fix is to run it in two passes:
- Rough pass. Batch everything against one anchor, only to rank the keywords by popularity.
-
Precise pass. Sort by that ranking and cut the list into batches where each batch shares one keyword with the previous one:
[1,2,3,4,5],[5,6,7,8,9],[9,10,...]. Neighbours in popularity end up together, so nothing gets rounded to zero, and the shared keyword links each batch to the last.
import time
from pytrends.request import TrendReq
def compare_many(keywords, timeframe="today 12-m", geo=""):
pytrends = TrendReq(hl="en-US", tz=0)
def fetch(batch):
pytrends.build_payload(batch, timeframe=timeframe, geo=geo)
time.sleep(2)
return pytrends.interest_over_time().drop(columns="isPartial")
# Rough pass: rank against one anchor.
anchor = keywords[0] # pick something mid-popular, it must not score 0
rough = {}
for i in range(1, len(keywords), 4):
df = fetch([anchor] + keywords[i:i + 4])
rough.update((df.sum() / df[anchor].sum()).to_dict())
ordered = sorted(rough, key=rough.get, reverse=True)
# Precise pass: similar sizes together, one shared keyword per link.
merged = None
for start in range(0, max(len(ordered) - 1, 1), 4):
batch = ordered[start:start + 5]
df = fetch(batch)
if merged is None:
merged = df
continue
link = batch[0]
df = df * merged[link].sum() / df[link].sum()
merged = merged.join(df.drop(columns=link))
return (merged * 100 / merged.max().max()).round(2)
How close does it get? I checked 7 coffee drinks against direct 5-term comparisons on the Google Trends site. Relative to chai:
| Keyword | Chained | Google, direct |
|---|---|---|
| matcha | 2.07 | 2.09 |
| espresso | 3.20 | 3.22 |
| latte | 1.76 | 1.75 |
| kombucha | 0.36 | 0.36 |
With a single anchor, kombucha came out at roughly 1 or 2 against coffee's 100, mostly rounding noise. The ladder fixed that.
Daily data for multi-year ranges
Google returns daily points for ranges up to about 269 days. For anything longer, request overlapping daily windows and rescale each window to match the previous one on the shared days:
from datetime import timedelta
import pandas as pd
WINDOW_DAYS = 260
OVERLAP_DAYS = 60
def daily_interest(keywords, start, end, geo=""):
pytrends = TrendReq(hl="en-US", tz=0)
merged = None
window_start = start
while True:
window_end = min(window_start + timedelta(days=WINDOW_DAYS - 1), end)
pytrends.build_payload(keywords, timeframe=f"{window_start} {window_end}", geo=geo)
df = pytrends.interest_over_time().drop(columns="isPartial")
time.sleep(2)
if merged is None:
merged = df
else:
overlap = merged.index.intersection(df.index)
scale = merged.loc[overlap].values.sum() / df.loc[overlap].values.sum()
merged = pd.concat([merged, df.drop(overlap) * scale])
if window_end >= end:
return (merged * 100 / merged.max().max()).round(2)
window_start = window_end - timedelta(days=OVERLAP_DAYS - 1)
The overlap length matters more than you'd think. With a few days of overlap, one spike inside it throws off the scale of everything after it. Sixty days averages that out. For bitcoin and ethereum over 2023-2025, the stitched daily series correlates 0.997 with Google's own weekly series for the same period.
To get daily data for more than 5 keywords, combine the two: stitch each batch over time first, then chain the batches.
The part that will actually stop you: 429 errors
All of this takes many requests. Seven keywords in two passes is 4 requests. Seven keywords over 5 years of daily data is about 36. Google rate-limits Trends per IP, and you'll start seeing 429 Too Many Requests well before that.
pytrends won't save you here. It was archived in 2025, and its retries= option crashes on current urllib3 with unexpected keyword argument 'method_whitelist'. Google's official Trends API is still invite-only. What works is retrying with a growing, randomised delay, starting a fresh session after each 429, and rotating proxies once you're past a few dozen requests.
Or skip all of it
I packaged everything above, including proxy rotation and retries, into an Apify Actor. You give it a list of keywords of any length, turn on daily data if you want it, and get one dataset on a single 0-100 scale. It also returns regions, related queries and Trending Now. In a test of 30 keywords with 5 years of daily data, it returned 54,810 data points in under 2 minutes with no errors. It costs $0.002 per keyword, and keywords that fail are free.
from apify_client import ApifyClient
client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("rishupishu/google-trends-scraper").call(run_input={
"keywords": ["coffee", "tea", "matcha", "espresso", "latte", "chai", "kombucha"],
"timeRange": "today 5-y",
"dailyData": True,
})
rows = client.dataset(run.default_dataset_id).list_items().items
Either way, the method is the same. Rescale through something the two requests share, and keep the things you compare at similar sizes.
Top comments (0)