DEV Community: Rafi-siddiq

A pytrends alternative that handles Google Trends rate limits

Rafi-siddiq — Sat, 31 Oct 2026 17:32:09 +0000

For years, the default way to get Google Trends data in code was pytrends. Today it's a frustrating choice: the repository is archived, it no longer gets fixes, and many people now see 429 Too Many Requests on the very first call.

This post covers why that happens, what your options are, and how to get Trends data without managing it yourself. Disclosure: the hosted option at the end is a tool I built.

Why Google Trends returns 429s

Google Trends has no public API that anyone can use (more on the official one below). Libraries like pytrends call the same internal endpoints the Trends website uses. Google limits how often those endpoints can be called, and it doesn't publish the limits. When you go over, you get a 429 and have to wait.

The limits are stricter for traffic from cloud servers than from home connections, which is why a script that works on your laptop often fails on a server or in a notebook environment.

Your options

1. Keep a library and slow it down. Space requests out by several seconds, retry with exponential backoff (wait 2 s, then 4 s, then 8 s…), and cache results so you don't ask twice. This works for small, occasional jobs. It gets painful when you need hundreds of keywords.

2. Apply for the official Google Trends API. Google opened an alpha of an official Trends API in 2025. It returns consistently scaled data, which the website doesn't. Access is by application, it's limited to approved testers, and there's no general release date yet. If Trends data matters to your product, apply now.

3. Use a hosted scraper. Someone else deals with retries, pacing, and changes to the site, and you call an API.

Using a hosted Google Trends scraper

I built the Google Trends Scraper on Apify for my own keyword research. For each search term it returns:

Interest over time: the 0–100 timeline for any period, from the past hour to 2004 onward, or exact dates.
Interest by region: countries, states, cities, or US metro areas (DMA).
Related searches: top and rising queries, with a flag for "Breakout" (growth over 5,000%).
Comparisons: up to 5 terms on the same scale.
Trending Now: today's trending searches per country, with traffic estimates and news links.

It paces itself, backs off when Google rate-limits it, and only charges for terms that return data: failed lookups and terms with no data are free.

From the web UI

Enter your terms, a location such as US or US-CA, and a time range, then click Start. You can download the results as JSON, CSV or Excel.

From code

Apify can run it and return the results in one HTTP call. Get your API token from Settings → API & Integrations in Apify Console, and keep it out of your code and repositories (use an environment variable).

With curl:

curl -X POST "https://api.apify.com/v2/acts/rsiddiq95~google-trends-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchTerms": ["air fryer", "standing desk"], "geo": "US", "timeRange": "today 12-m"}'

With Python:

import os
import requests

resp = requests.post(
    "https://api.apify.com/v2/acts/rsiddiq95~google-trends-scraper/run-sync-get-dataset-items",
    headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
    json={"searchTerms": ["air fryer", "standing desk"], "geo": "US", "timeRange": "today 12-m"},
    timeout=300,
)
for row in resp.json():
    if row["resultType"] == "keyword":
        print(row["searchTerm"], row["averageInterest"])

Each keyword comes back as one record:

{
    "resultType": "keyword",
    "hasData": true,
    "searchTerm": "air fryer",
    "geo": "US",
    "timeRange": "today 12-m",
    "averageInterest": 65,
    "interestOverTime": [{ "date": "2025-10-26T00:00:00.000Z", "value": 52, "isPartial": false }],
    "interestByRegion": [{ "geoCode": "US-WY", "geoName": "Wyoming", "value": 100 }],
    "relatedQueries": {
        "top": [{ "query": "air fryer chicken", "value": 100 }],
        "rising": [{ "query": "best air fryer 2026", "formattedValue": "Breakout", "isBreakout": true }]
    },
    "trendsUrl": "https://trends.google.com/trends/explore?q=air+fryer&date=today+12-m&geo=US"
}

It costs $2.50 per 1,000 keywords and $1 per 1,000 trending searches, with no platform fees on top. Apify's free plan includes monthly credit.

Things to know about Trends data (whatever tool you use)

Values are relative. 100 is the peak within your query, not a search volume. If you want two keywords on the same scale, compare them in one query.
Results can shift slightly between runs, because Google re-scales each request. Use exact date ranges for reports you want to reproduce.
Related topics aren't available from my scraper, because Google doesn't provide them to automated tools.
Classic Explore is going away. Google shows a notice on the Trends site that the "classic Explore" page is expiring soon, with no date announced. The new version requires signing in. Every unofficial Trends tool reads the classic version, so expect disruption when it shuts down. That's another reason to apply for the official API if you depend on this data.

If you have questions or need a feature, open an issue on the Actor page.

How to track competitors' Google ads in a Google Sheet automatically

Rafi-siddiq — Sat, 31 Oct 2026 17:31:44 +0000

If you run paid search, you've probably opened the Google Ads Transparency Center to see what a competitor is running. It's useful, but it shows one ad at a time, and there's no export. If you want to notice when a competitor changes their messaging, you have to remember to check.

This tutorial sets up a small pipeline that does it for you:

Every morning, collect the latest Google ads for a list of competitors, including the ad copy.
Write them to a Google Sheet, updating ads you already have instead of duplicating them.

It takes about 15 minutes and runs on Apify. Disclosure: I built both Actors used here.

What you'll end up with

A sheet with one row per ad, kept up to date:

Column	Example
`advertiserName`	Nike, Inc.
`format`	TEXT
`adText`	Nike, nike.com, Nike® Official Site, Shop exclusive new styles…
`firstShownAt`	2023-11-17T00:26:00Z
`lastShownAt`	2026-10-30T00:29:17Z
`daysShown`	911
`landingDomain`	nike.com
`adUrl`	link to the ad in the Transparency Center

Because lastShownAt updates on every run, sorting by it shows which ads are still live, and new rows at the bottom are ads that appeared since yesterday.

Step 1: Collect the ads

Open the Google Ads Transparency Scraper and click Try for free.

You can find a competitor's ads three ways:

Website: competitor.com. This also finds resellers and agencies running ads that point to the same site.
Verified advertiser name: for example Nike, Inc.
Advertiser ID or URL: paste a link from the Transparency Center.

Here's an input that tracks two competitors' US search ads:

{
    "advertiserDomains": ["competitor-one.com", "competitor-two.com"],
    "region": "US",
    "platform": "SEARCH",
    "maxAdsPerTarget": 50,
    "extractAdCopy": true
}

Ads come back newest first, so 50 per competitor is plenty for a daily check: anything new shows up at the top. For the very first run you might raise it to a few hundred to get the full history, then lower it again.

Click Start. 100 ads take about a minute.

A note on ad copy: Google stores a text preview for many ads, but some only as images. When there's no text preview, adText is empty and you still get the image URL.

When the run finishes, click Save as a new task and name it something like competitor-ads-daily. A task is just a saved input you can schedule.

Step 2: Create a Google service account (one time)

To write to your sheet, the sync Actor uses a Google service account: a separate login that can only open sheets you share with it.

In Google Cloud Console, create a project.
Go to APIs & Services → Library and enable Google Sheets API.
Go to APIs & Services → Credentials → Create credentials → Service account, give it a name, and click Done.
Open the service account, go to Keys → Add key → Create new key → JSON. A key file downloads.
Create a Google Sheet, click Share, and add the service account's email (the client_email in the key file) as an Editor.

Step 3: Send the ads to the sheet automatically

Open your competitor-ads-daily task, go to Integrations → Add integration → Apify Actor, and pick Google Sheets Sync. Use this input:

{
    "mode": "upsert",
    "spreadsheet": "https://docs.google.com/spreadsheets/d/your-sheet-id/edit",
    "sheetName": "Competitor ads",
    "datasetId": "{{resource.defaultDatasetId}}",
    "keyColumns": ["creativeId"],
    "omitFields": ["previewScriptUrl", "resultType"],
    "serviceAccountKey": "paste the contents of your key file here"
}

What each part does:

upsert with creativeId as the key means an ad already in the sheet gets its row updated (new lastShownAt, new daysShown) instead of being added again. New ads are appended.
{{resource.defaultDatasetId}} is filled in by Apify with the results of each run.
Columns you add yourself, such as a "Notes" column, are left alone.
Values are written as plain text, so ad text that happens to start with = can't run as a formula in your sheet.

The key is stored encrypted in Apify.

Step 4: Schedule it

Go to Schedules → Create new, set it to run daily (for example at 7 AM), and add the competitor-ads-daily task. That's it: the scraper runs, and when it finishes, the sync writes the results to your sheet.

What it costs

Both Actors charge per result, and you don't pay Apify platform fees on top:

Ads: $3 per 1,000 ads. Two competitors at 50 ads each, every day, is about 3,000 ads a month, or about $9.
Sheets sync: $0.05 per 1,000 rows, so about 15 cents a month for the same data.

Checking weekly instead of daily cuts that to about $2 a month.

Lookups that find nothing aren't charged, and Apify's free plan includes monthly credit to try it.

Ideas to extend it

Spot new messaging: if firstShownAt is in column F, add a column with =IF(DATEVALUE(LEFT(F2,10))>TODAY()-7,"NEW","") to flag ads first shown in the last week.
Track YouTube creative: change platform to YOUTUBE and adFormat to VIDEO.
Watch your own brand: search your domain to see who else runs ads pointing to your site.

If you try it and something's missing, open an issue on the Actor page. I read every one.