Track what's trending on Google in 20 countries with 15 lines of Python

#python #datascience #seo #api

Google Trends has a page called Trending Now: the searches that are spiking right now in a given country, with a rough search volume and the news stories behind them. It's great for content ideas, newsjacking and market signals. But it's one country at a time, and there's no official API.

Here's how I pull it for many countries at once and get clean JSON back.

The data

For each trending search you get:

rank and search term
approximate searches ("10K+") plus a number you can sort on (10000)
when the trend started (UTC)
related news: headline, source, link
a ready-made Google Trends explore link

The code

I use a small hosted scraper I maintain on Apify (Google Trends Trending Now). With the official Python client (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("fjordlabs/google-trending-now").call(run_input={
    "geos": ["US", "GB", "DE", "FR", "IN", "BR", "JP"],
    "includeNews": True,
    "minTraffic": 1000,        # skip the small ones
})

for t in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f'{t["geo"]} #{t["rank"]:>2} {t["title"]} ({t["approxTraffic"]})')

Output looks like:

US # 1 tesla stock (10K+)
GB # 1 strictly come dancing (20K+)
...

Cross-country trends in pandas

The fun part is seeing what trends in several countries on the same day:

import pandas as pd

df = pd.DataFrame(client.dataset(run["defaultDatasetId"]).iterate_items())
spread = (df.groupby(df["title"].str.lower())["geo"]
            .agg(["nunique", lambda g: ", ".join(sorted(g))])
            .sort_values("nunique", ascending=False))
print(spread.head(10))

Make it a monitor

Schedule the run hourly in Apify and connect the Slack or webhook integration, and you have a "what's spiking right now" alert for every market you care about. Each trending search costs $0.001, so 10 countries every hour is about $0.10 an hour at most, and failed rows are never charged.

Free history

I also keep a daily snapshot for 20 countries and publish the trend facts as a free Kaggle dataset: Daily Google Trending Searches, 20 Countries (updated weekly). It's handy for seasonality or "how do trends spread between countries" questions.

Disclosure: I built and maintain the scraper. Questions and feature requests welcome in the comments.

DEV Community