Scrape Google Trends with Python (simple API, pytrends alternative)

We are surrounded by tons of data. But what's the point if we can't understand it all? That's where Google Trends comes in. Google Trends offers valuable insights into what people search for on the internet. Let's dive in and see how we can programmatically scrape Google Trends data using Python (Psst.. no PyTrends needed!).

Step-by-step scraping Google Trends data with Python

Without further ado, let's start and collect data from Google Trends.

Step 1: Tools

We'll use the new official Python library by SerpApi: serpapi-python .

That's the only tool that we need!

As a side note: You can use this library to scrape search results from other search engines, not just Google.

Usually, you'll write your DIY solution using PyTrends, BeautifulSoup, Selenium, Scrapy, Requests, etc., to scrape Google Trends results. You can relax now since we perform all these heavy tasks for you. So, you don't need to worry about all the problems you might've encountered while implementing your web scraping solution.

Step 2: Setup and preparation

• Sign up for free at SerpApi. You can get 100 free searches per month.
• Get your SerpApi Api Key from this page.
• Create a new .env file, and assign a new env variable with value from API_KEY above. SERPAPI_KEY=$YOUR_SERPAPI_KEY_HERE
• Install python-dotenv to read the .env file with pip install python-dotenv
• Install SerpApi's Python library with pip install serpapi
• Create new main.py file for the main program.

Your folder structure will look like this:

|_ .env
|_ main.py

Step 3: Write the code for scraping Google Trends

Let's say I want to get this result:

- Keyword: 'standing desk'

- Region: 'worldwide'

- Time: 'Past 12 months'

Here is the Python code

import os
import serpapi
from dotenv import load_dotenv

load_dotenv()
api_key = os.getenv('SERPAPI_KEY')

client = serpapi.Client(api_key=api_key)
search =  client.search(
    engine="google_trends",
    q="standing desk",
    api_key=api_key,
  )

print(search)

By default, the Google Trends API will set the location for worldwide and duration of the past 12 months . We'll take a look at how to change these values later. For now, take a look at our perfect result:

Please note, that we only return the response in JSON format. You'll need to build the graph yourself if you need one.

Step 4: Adjust region and timeline

Let's change the settings. I set the region to United States and timeline to Past 30 days .

Let's see how it is represented in our code.

search =  client.search(
    engine="google_trends",
    q="standing desk",
    api_key=api_key,
    geo="US", // Two-code-letter country code
    date="today 1-m" // Timeline for last 1 month
)

The result is perfect as well:

More information about available parameters: https://serpapi.com/google-trends-api

SerpApi Google Trends API features

Using SerpApi, it's possible to grab other parts from Google Trends, like the breakdown by region result, interest by region, related queries, related topics, and what's trending now.

Searching for multiple terms

It's possible to search for multiple terms. You can separate the terms by a comma. Here are the limitations:

- Max 5 terms per search

- Max 100 characters per search

search =  client.search(
    engine="google_trends",
    q="standing desk, ergonomic chair",
    api_key=api_key,
)

Interest by Region

We can grab the data for interest by region. We need to change the data_type to GEO_MAP_0

search = client.search(
    engine="google_trends",
    q="coffee latte",
    date="today 12-m",
    tz="420",
    data_type="GEO_MAP_0", // Important part
    api_key=api_key,
)

Here is the list of possible values for data_type

For example, if you're interested in the related topics, you need to adjust your code to:

search = client.search(
    engine="google_trends",
    q="coffee latte",
    date="today 12-m",
    tz="420",
    data_type="RELATED_TOPICS", // Adjust this accordingly
    api_key=api_key,
)

print(search)

References to our other related API:

• Trends Autocomplete API
• Trends Breakdown by region API
• Trends by Region API
• Interest overtime API
• Trends related queries API
• Trends related topics API
• Trending Now API

Pytrends Tutorial to scrape Google Trends data

There is also another unofficial API for Google Trends for Python. It's called Pytrends. We'll see a basic tutorial on how to implement this.

Please note that the repository has not been very active these last few months. There are also some unadressed issues. Therefore, we recommend using Google Trends API instead.

Install Pytrends

pip install pytrends

Initialize the object. We also initialize pandas so it's easier to read the data.

from pytrends.request import TrendReq
import pandas as pd

pytrend = TrendReq()
pytrend.build_payload(kw_list=['Standing Desk'])

Inside the build_payload method, we can put the keywords list in an array.

Interest Over Time

# Interest over time
pytrend = TrendReq()
pytrend.build_payload(kw_list=['Standing Desk'])
df = pytrend.interest_over_time()
print(df)

Here is the result

There are some other parameters for build_payload method. Unfortunately, I sometimes get error results. Here are the parameters you can play with: year_start, month_start, day_start, hour_start, year_end, month_end, day_end, hour_end.

Interest by Region

Here is how to split the trends by country and sort the top 10 results.

pytrend = TrendReq()
pytrend.build_payload(kw_list=['Standing Desk'])
df = pytrend.interest_by_region(resolution='COUNTRY')
print(df.sort_values(by='Standing Desk', ascending=False).head(10))

Related topics

Here is how to show related topics using Pytrends:

df = pytrend.related_topics()
print(df)

Related queries

Here is how to show related topics using Pytrends:

df = pytrend.related_queries()
print(df)

Trending searches

Here is how to display trending searches by Region from Google Trends

#Trending Searches
df = pytrend.trending_searches(pn='united_states') # trending searches in real time for United States
print(df.head)

The following API methods are available for PyTrends:

• Interest Over Time: returns historical, indexed data for when the keyword was searched most as shown on Google Trends' Interest Over Time section.
• Multirange Interest Over Time: returns historical, indexed data similar to interest over time but across multiple time date ranges.
• Historical Hourly Interest: returns historical, indexed, hourly data for when the keyword was searched most, as shown on Google Trends' Interest Over Time section. It sends multiple requests to Google, each retrieving one week of hourly data. It seems like this would be the only way to get historical, hourly data.
• Interest by Region: This returns data for where the keyword is most searched, as shown in Google Trends' Interest by Region section.
• Related Topics: returns data for the related keywords to a provided keyword shown on Google Trends' Related Topics section.
• Related Queries: returns data for the related keywords to a provided keyword shown on Google Trends' Related Queries section.
• Trending Searches: returns data for the latest trending searches shown on Google Trends' Trending Searches section.
• Top Charts: returns the data for a given topic shown in Google Trends' Top Charts section.
• Suggestions: Return a list of additional suggested keywords that can be used to refine a trend search.

---

Thank you for reading this blog post! I hope it can help you to gather data from the Google Trends site.

We have a Google Trends playground where you can play with our API: https://serpapi.com/playground?engine=google_trends

See you in the next post!