DEV Community: FeatherScrape

See every Google ad your competitors run, with Python (Google Ads Transparency Center)

FeatherScrape — Thu, 22 Oct 2026 22:03:28 +0000

Since 2023, Google publishes every ad it shows in the Ads Transparency Center: Search ads, YouTube videos, display banners, Shopping, Maps and Play ads, for every advertiser, with the countries and dates they ran.

That's a goldmine for competitive research. But the website only lets you browse ads one by one, there's no export, and most Search ads are shown as screenshots, so you can't even copy their text.

In this post I'll pull every ad of two competitors (Asana and monday.com) into pandas with a few lines of Python, then answer a few questions you'd actually ask about a competitor's advertising.

Disclosure: I built the Google Ads Transparency Scraper used below. It runs on Apify, which has a free plan with monthly credits, so you can follow along without paying anything.

What we'll get

One row per ad, with:

advertiserName, format (text, image or video), firstShown, lastShown
headline, description, displayUrl for Search ads, read from the screenshot when Google doesn't publish the text
countries where the ad ran, with per-country dates and impression ranges (published for EU and political ads)
paidBy: the entity that paid for the ad, which sometimes reveals the agency behind a brand
imageUrl, youtubeUrl for image and video ads

Step 1: install the client

Create a free Apify account, copy your API token from Settings → API & Integrations, and install:

pip install apify-client pandas

Step 2: fetch the ads of two competitors

Searching by domain is the most complete option: Google returns the ads of every account that sends traffic to that domain, including agencies and local entities.

import os
import pandas as pd
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("featherscrape/google-ads-transparency-scraper").call(run_input={
    "searchTerms": ["asana.com", "monday.com"],  # domains, brand names or advertiser IDs
    "region": "US",                                # where the ads were shown; empty = everywhere
    "maxAdsPerSearch": 100,                        # most recently shown first
})

ads = pd.DataFrame(client.dataset(run["defaultDatasetId"]).iterate_items())
ads["firstShown"] = pd.to_datetime(ads["firstShown"])
ads["lastShown"] = pd.to_datetime(ads["lastShown"])
print(len(ads), "ads")

You can also filter by platform (search, youtube, shopping, maps, play), format (text, image, video) and a dateFrom / dateTo range.

Step 3: what are they running?

print(ads["format"].value_counts())
print(ads["advertiserName"].value_counts())

On my run (US, October 2026), 200 ads came back: 158 Search ads, 36 videos and 6 image ads. The advertiser list already holds a surprise:

monday.com LTD      99
Asana, Inc.         94
MightyHive, INC.     6

MightyHive is a media agency: it runs the display banners pointing to asana.com, in up to 23 countries each. That's the kind of thing you won't see by searching Google yourself.

Step 4: read their ad copy

text_ads = ads[ads["headline"].notna()]
for _, ad in text_ads[text_ads["advertiserName"] == "monday.com LTD"].head(10).iterrows():
    print(f'{ad["headline"]}\n  {ad["description"]}\n  {ad["displayUrl"]}\n')

A few of them:

monday CRM is So Much Better
  Manage contacts, sales, and pipelines in one place with tools your team will enjoy.
  monday.com

What Makes monday.com Better?
  Choose monday.com to Build Workflows Perfectly Customized to Your Team's Needs.
  monday.com

Project Management Templates
  Plan, Track, Manage and Collaborate Using the Top Templates Teams Love. Try for Free.
  monday.com

Comparative headlines, a CRM push and templates as a lead magnet: you get monday.com's messaging strategy in one screen.

Step 5: find the ads that work

Advertisers don't keep paying for ads that don't convert. An ad that has been running for years is a strong signal:

ads["daysRunning"] = (ads["lastShown"] - ads["firstShown"]).dt.days
print(ads.sort_values("daysRunning", ascending=False)[["advertiserName", "headline", "daysRunning"]].head())

"What Makes monday.com Better?" has been running since October 2021: five years. If you compete with monday.com, that's the message to answer.

Step 6: where do they advertise?

print(ads.explode("countries")["countries"].value_counts().head(10))

Each ad also has a regions list with first and last shown dates per country, and impression ranges where Google publishes them (ads shown in the EU and political ads).

Step 7: get alerted on new ads

Turn on onlyNewAds and schedule the run (for example every morning in the Apify Console, or from cron with the code above). Each run then returns only the ads that weren't in previous runs, so you can push them to Slack, a Google Sheet or an email digest:

run = client.actor("featherscrape/google-ads-transparency-scraper").call(run_input={
    "searchTerms": ["asana.com", "monday.com"],
    "onlyNewAds": True,
})
new_ads = list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(f"{len(new_ads)} new ads since the last run")

A few things to know

Impressions are rare. Google publishes impression ranges only for ads shown in the EU (Digital Services Act) and for political ads. There's no spend or click data for commercial ads.
Screenshot text is read automatically (textSource: "ocr"). It's accurate on standard ads, with the occasional OCR slip (for example "Al" instead of "AI"). ocrText keeps the raw text.
Domains beat names. A name like "Asana" is matched to the closest advertiser accounts; a domain catches every account and agency advertising for it.

Wrapping up

The Ads Transparency Center is one of the best free sources of competitive intelligence, once you can get the data out of it. With a few lines of Python you can see what your competitors say, where and since when, and get notified when they launch something new.

The scraper costs $1.50 per 1,000 ads. If you need a field or hit a bug, open an issue on the scraper's page: I usually answer within 48 hours. And I'm curious: which competitor would you look at first?

Get Google Trends data in Python without pytrends (and without 429 errors)

FeatherScrape — Thu, 22 Oct 2026 17:38:47 +0000

If you have ever pulled Google Trends data from Python, you have probably used pytrends. And you have probably met this:

pytrends.exceptions.TooManyRequestsError: The request failed: Google returned a response with code 429

Two things changed in the last couple of years:

pytrends is archived. The repository was archived by its maintainers and hasn't received a commit since August 2024, so breakages on Google's side don't get fixed anymore.
Google Trends rate-limits hard. A few requests in a row from the same IP are enough to get HTTP 429 errors, which is painful as soon as you track more than a handful of keywords, or run a script on a schedule from a server.

In this post I'll show how to get the same data (interest over time, interest by region, related queries and topics) into pandas with a few lines of Python, using a hosted scraper that handles the rate limits for you, and how to schedule it with no code at all.

Disclosure: I built the Google Trends Scraper used below. It runs on Apify, which has a free plan with monthly credits, so you can follow this tutorial without paying anything.

What we'll get

For each keyword, one JSON item with:

interestOverTime: the full time series (0–100 index, like on trends.google.com)
interestByRegion: countries, regions/states, cities or US metros
relatedQueries: top and rising searches (+400%, Breakout)
ready-made metrics: averageInterest, latestInterest, peakInterestDate
googleTrendsUrl: a link to the same chart on trends.google.com, handy to double-check

Step 1: get an API token

Create a free account on Apify, then copy your API token from Settings → API & Integrations. Install the Python client and pandas:

pip install apify-client pandas

Step 2: run the scraper from Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("featherscrape/google-trends-scraper").call(run_input={
    "keywords": ["bitcoin", "ethereum", "solana"],
    "compareKeywords": True,   # same 0-100 scale for all keywords, like on trends.google.com
    "geo": "US",               # empty string = worldwide; also "US-CA", "FR-IDF"...
    "timeRange": "today 3-m",  # or "today 12-m", "today 5-y", "now 7-d"... or "2025-01-01 2025-06-30"
    "searchType": "web",       # or "youtube", "news", "shopping", "images"
})

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(len(items), "keywords")

When Google answers with a 429, the scraper retries with a fresh session and a new IP, so you don't need any sleep() or retry logic on your side. You're charged per keyword returned; keywords that fail after all retries aren't charged.

Step 3: interest over time in a DataFrame

import pandas as pd

frames = []
for item in items:
    df = pd.DataFrame(item["interestOverTime"])[["date", "value"]]
    df["keyword"] = item["keyword"]
    frames.append(df)

trends = (pd.concat(frames)
            .assign(date=lambda d: pd.to_datetime(d["date"]))
            .pivot(index="date", columns="keyword", values="value"))

print(trends.tail())
trends.plot(title="Google Trends, US, last 90 days")

Because compareKeywords is on, the three columns share the same scale, exactly like the comparison chart on trends.google.com. Turn it off and each keyword gets its own 0–100 scale instead, which is better when you track many unrelated keywords.

Step 4: where is the interest, and what's rising?

for item in items:
    top_regions = item["interestByRegion"][:5]
    rising = item["relatedQueries"]["rising"][:5]
    print(f'\n{item["keyword"]}  (peak on {item["peakInterestDate"][:10]})')
    print("  top regions:", ", ".join(f'{r["name"]} {r["value"]}' for r in top_regions))
    print("  rising:", ", ".join(f'{q["query"]} ({q["formattedValue"]})' for q in rising))

Rising queries are the most useful part for SEO and content ideas: a Breakout query is one whose search volume grew by more than 5,000% over the period.

No-code option: a weekly report in Google Sheets

If you don't want to run Python at all:

Open the scraper's page, click Try for free, enter your keywords and run it once.
In the run's Storage tab, export the results as CSV or Excel, or open the Integrations tab to send each run's data to Google Sheets or Google Drive.
Create a Schedule (for example every Monday at 8 a.m.) so the data refreshes on its own.

A few Google Trends gotchas

Values are relative, not search volumes. 100 is the peak popularity for the selected keywords, region and period. Comparing two separate requests only makes sense if they used the same comparison group.
Kansas is often #1 in the US. Searches that Google can only locate to "United States" are assigned to the country's geographic center, which is in Kansas.
Numbers move a little between runs. Google Trends is computed on a sample of searches, so expect small variations from one request to the next.
Related topics are sometimes empty, especially when comparing several keywords. Related queries are usually there.

Wrapping up

pytrends was a great library, but with the project archived and Google's rate limits getting stricter, a hosted scraper is the least painful way I know to get Google Trends data reliably, from a notebook or on a schedule.

If you need a field that isn't there, or hit a bug, open an issue on the scraper's page: I usually answer within 48 hours. And I'd love to hear in the comments what you use Google Trends data for.