DEV Community: Kyungmin Lee

I pulled 50 Google ads from monday.com and Asana. Here is what Google's ad library reveals

Kyungmin Lee — Sat, 10 Oct 2026 13:49:43 +0000

Google keeps a public ad library: the Ads Transparency Center. Type a company name and you can scroll through the ads it runs on Search, YouTube, Display and Shopping. It is a goldmine for competitor research, with one catch: it is an infinite wall of cards. There is no export, Search ads are shown only as screenshots, and comparing two companies means a lot of squinting.

So I exported it instead. Below: what 50 of the newest ads from monday.com and Asana (25 each) say about how two direct competitors are fighting for the same buyers, and how to run the same analysis on your own competitors in a couple of minutes.

Disclosure: I built the scraper used here (link at the end). The numbers are from a run on 10 October 2026; ads change daily.

What the export looks like

Each ad becomes one row: advertiser, format (text, image, video), the ad copy (Search ads are read from the screenshot with OCR), the YouTube video ID for video ads, the countries it ran in, and the first shown / last shown dates plus the number of days it has been running. That last field turns out to be the most interesting one.

The whole pull — 50 ads, 29 with extracted copy, two advertiser profiles — took under two minutes and cost about $0.19.

Finding 1: they are not even playing the same format

	monday.com	Asana
Newest 25 ads	16 video, 9 search text	25 search text
Median days running	263	122
Oldest ad still running	first shown Oct 2021 (1,812 days)	first shown Jun 2023 (989 days)

Among their 25 newest creatives, monday.com spends most of its effort on YouTube video; Asana is almost entirely in Search. If you sell in this category and only watch competitors' search ads, you would miss most of what monday.com is doing.

Finding 2: the longest-running ads are the ones to study

A useful rule of thumb in ad research: advertisers rarely keep paying for an ad that does not perform, so ads that have been running for months or years are probably working.

monday.com's oldest still-active search ad was first shown in October 2021:

What Makes monday.com Better? Choose monday.com to Build Workflows Perfectly Customized to Your Team's Needs.

Its next-longest runners lean on the same idea, "100% customizable", plus a scheduling angle ("The #1 Online Schedule Platform"). Asana's longest runner (since June 2023) is a free-tool hook: "Free Calendar Tool for Teams – See Asana in Action". Five years of one message is a strong signal of what converts for monday.com; Asana keeps coming back to free templates and tools as the entry point.

Finding 3: both are pivoting to AI and agencies

The newest creatives tell you where each company is heading:

Asana, September 2026: "Agentic Work Management — drafts context-aware briefs, summaries, communications and slides from live Asana data" and "Asana Client Management — a branded client portal".
monday.com, July 2026: "CRM for Marketing Agencies — Win more clients for your marketing agency. Focus on deals, let AI do the admin", plus new Spanish-language calendar ads ("Calendario Compartible Equipos").

Both are now selling AI that does the busywork, and both are going after agencies. monday.com is also localising (Spanish), while the Asana ads in this sample that list countries were shown most often in Germany and France.

How to do this for your own competitors

You can reproduce the exact run from a ready-made example — monday.com vs Asana — and change the names. The input is just:

{
  "searchQueries": ["monday.com", "Asana"],
  "maxAdsPerAdvertiser": 25,
  "includeAdText": true,
  "ocrAdText": true
}

You can also search by domain ("domains": ["yourcompetitor.com"]) to catch every advertiser linking to a site — handy for spotting resellers or people bidding on your brand.

Then rank the export the way I did above:

import pandas as pd

df = pd.read_json("dataset.json")
ads = df[df["type"] == "ad"]

# Format mix per competitor
print(ads.groupby(["advertiserName", "format"]).size().unstack(fill_value=0))

# Longest-running ads = the messages they keep paying for
cols = ["advertiserName", "format", "daysShown", "firstShown", "adText"]
print(ads.sort_values("daysShown", ascending=False)[cols].head(10).to_string())

# What changed recently
print(ads.sort_values("firstShown", ascending=False)[cols].head(10).to_string())

For ongoing monitoring, schedule the run weekly with startDate set to last week (only ads shown in that window) and compare the creativeIds with the previous export: anything new is a creative launched since your last check — the closest thing to a "competitor just started a new campaign" alert.

Caveats

No spend or impressions. The Transparency Center shows what ran, where and when, not budgets or results. "Days running" is a proxy, not proof.
OCR is imperfect. Search ads are published as images; the text is read with English OCR. Expect small artifacts ("Al" for "AI", stray "Sponsored" labels), and non-Latin ads come out garbled.
Regions are partial. Google lists countries for some ads only.
Fresh data, moving target. These are the 25 newest creatives per advertiser on the day of the run, not their full history.

The scraper is open source (GitHub) and runs hosted on Apify, pay per ad: Google Ads Transparency Center Scraper. If you want the technical side — the RPC endpoints behind the Transparency Center and the OCR pipeline — I wrote that up in an earlier post.

pytrends is dead. Here is how Google Trends actually loads data in 2026 (including Trending Now)

Kyungmin Lee — Tue, 15 Sep 2026 04:41:15 +0000

If you have tried pytrends recently you know the story: TooManyRequestsError on the first call, dailytrends returning 404, and an unmaintained repo. Google redesigned Trends in 2024, moved the "Trending Now" feed to a different backend, and started returning 429 to anything that looks like a script.

The good news is that the site is still a plain JSON API underneath. Here is the map I put together while building a browserless client with nothing but httpx.

The two-step token dance (explore → widgetdata)

Every Trends chart is loaded in two requests:

GET /trends/api/explore?hl=en-US&tz=-540&req=<JSON> — you describe what you want (keywords, geo, timeframe, property) and Google answers with a list of widgets, each carrying a token and a request object.
GET /trends/api/widgetdata/<kind>?token=<token>&req=<JSON(widget.request)> — you send the widget's own request back with its token and get the data.

The req for explore looks like this:

import json

def explore_req(keywords, geo="", timeframe="today 12-m", category=0, prop=""):
    return json.dumps({
        "comparisonItem": [{"keyword": k, "geo": geo, "time": timeframe} for k in keywords],
        "category": category,
        "property": prop,          # "" web, "images", "news", "youtube", "froogle"
    }, separators=(",", ":"))

Widget IDs tell you which endpoint to call:

Widget id	Endpoint	What you get
`TIMESERIES`	`/trends/api/widgetdata/multiline`	`timelineData[]` with `time`, `formattedTime`, `value[]`, `isPartial`
`GEO_MAP` (or `GEO_MAP_0` … per keyword)	`/trends/api/widgetdata/comparedgeo`	`geoMapData[]` with `geoCode`, `geoName`, `value[]`
`RELATED_QUERIES_0`	`/trends/api/widgetdata/relatedsearches`	two `rankedList` entries: TOP then RISING
`RELATED_TOPICS_0`	same endpoint, `keywordType: ENTITY`	topics — empty for logged-out sessions (see below)

Every response starts with the anti-JSON-hijacking prefix )]}'\n — strip it before json.loads.

Two details that cost me an evening:

The token is bound to the exact request JSON you send back. Changing userConfig → 401. The only fields you may edit are resolution (COUNTRY/REGION/CITY/DMA) and includeLowVolumeGeos on the geo widget.
Explore stamps userConfig.userType. Logged-out HTTP sessions get USER_TYPE_SCRAPER; a real Google login gets USER_TYPE_LEGIT_USER. Time series, geo and related queries work fine for scrapers, but related topics return {"rankedList": []}. If you need topics you need your own cookies — I don't ship that.

Where Trending Now went

/trends/api/dailytrends and /trends/api/realtimetrends are gone (404). The new Trending Now page is a Google "batchexecute" app:

POST https://trends.google.com/_/TrendsUi/data/batchexecute?rpcids=i0OFE&source-path=/trending&hl=en-US&rt=c
Content-Type: application/x-www-form-urlencoded
f.req=[[["i0OFE","[null,null,\"US\",0,\"en-US\",24]",null,"generic"]]]

The inner argument list is [null, null, geo, 0, hl, hours]. The response is chunked: )]}'\n\n<length>\n[["wrb.fr","i0OFE","<json string>",...]] — read the length prefix, parse the chunk, then parse the JSON string inside it. Each trend item is a positional array:

[title, null, geo, [start_ts], [end_ts] | null, null, search_volume, null,
 pct_increase, [breakdown_keywords], [category_ids], [[article_id, lang, geo], ...],
 normalized_title]

I keep the mapping in one parse_trending() function and treat it as "will break someday". Note that the category argument in the request does not filter server-side (I got 566 vs 567 items with and without it), so filter on category_ids client-side.

There is also /trending/rss?geo=US, a plain RSS feed with ht:approx_traffic and ht:news_item, which makes a nice fallback when the batchexecute format changes.

Staying under the 429 radar

Google Trends rate-limits per IP and, more subtly, per session:

Warm up first: GET https://trends.google.com/ (302 → /trends/) sets the NID cookie. Endpoints work without it on a fresh IP, but sessions without it hit 429 far sooner.
relatedsearches is the touchiest endpoint — it 429s on the first call in a burst but is fine with ~5 s spacing.
On 429 (an HTML "Error 429" page, no Retry-After): back off 5 → 10 → 20 s with jitter, drop the cookie jar and re-warm, and if you have a proxy pool, build a new client on a new IP.
Cache explore results per (keywords, geo, timeframe): the tokens stay valid for a while, so three widgets cost one explore call, not three.

In production I keep ≥ 2 s between calls and rotate the proxy session every handful of requests. A keyword with 12 months of history, regions and related queries takes about 15 seconds end to end.

What you get

A keyword becomes rows like these:

{"type": "interestOverTime", "keyword": "ai agents", "geo": "US", "date": "2026-09-07", "value": 83, "isPartial": true}
{"type": "interestByRegion",  "keyword": "ai agents", "geo": "US", "geoCode": "US-CA", "geoName": "California", "value": 100}
{"type": "relatedQuery",      "keyword": "ai agents", "kind": "rising", "query": "ai agents framework", "value": "Breakout"}
{"type": "trending", "geo": "US", "rank": 1, "title": "…", "searchVolume": 500000, "startedAt": "2026-09-14T02:10:00Z", "relatedQueries": ["…"], "newsArticles": [...]}

That is enough to rebuild everything pytrends used to give you, plus the Trending Now feed it never had.

Source: https://github.com/m2kyungmin/apify-actors/tree/main/google-trends-scraper (the endpoint table at the top of trends_client.py is the living version of this post). Hosted, pay-per-row version: https://apify.com/kyungminlee/google-trends-scraper.

Scraping the Google Ads Transparency Center without a browser (and reading image ads with OCR)

Kyungmin Lee — Tue, 15 Sep 2026 04:39:19 +0000

The Google Ads Transparency Center shows every ad an advertiser has run on Search, YouTube, Display, Maps and Shopping. Marketers love it and hate it in equal measure: the data is all there, but the UI is an infinite scroll of cards, there is no export, and the text of image ads is baked into pixels.

This post walks through how the site actually loads its data, how to call those endpoints from Python with httpx, and how to get the copy out of image creatives with Tesseract. No Selenium, no Playwright.

1. The page is a thin shell over an RPC API

Open DevTools on an advertiser page and filter for rpc. Everything interesting is a POST to

https://adstransparency.google.com/anji/_/rpc/<Service>/<Method>?authuser=

with a form body of exactly one field, f.req, that contains JSON. Three methods do all the work:

Method	Purpose
`SearchService/SearchSuggestions`	advertiser search by name (returns advertiser IDs, regions, ad-count ranges, and the domains they advertise)
`SearchService/SearchCreatives`	the ad list for an advertiser, with filters and a cursor
`LookupService/GetCreativeById`	one ad's detail: regions, first/last shown, formats, video ID

The JSON is protobuf-as-JSON: keys are field numbers, not names. That is the only really annoying part, because you have to work out what "3": {"4": 2, "8": ["US"]} means by changing filters in the UI and diffing the requests. After an hour of that you end up with a payload builder like this:

FORMAT_BY_NAME = {"TEXT": 1, "IMAGE": 2, "VIDEO": 3}

def creatives_payload(advertiser_id, *, page_size=40, fmt=None, regions=None,
                      start=None, end=None, cursor=None, topic_id=1, viewer_geo="US"):
    f = {"1": advertiser_id}
    if fmt:      f["4"] = FORMAT_BY_NAME[fmt.upper()]
    if regions:  f["8"] = regions          # ISO alpha-2 codes
    if start:    f["6"] = start            # {"1": year, "2": month, "3": day}
    if end:      f["7"] = end
    payload = {"2": page_size, "3": f, "7": {"1": topic_id, "2": 0, "3": viewer_geo}}
    if cursor:
        payload["4"] = cursor              # opaque, returned as "2" in the previous response
    return payload

Calling it is a one-liner:

import json, httpx

RPC = "https://adstransparency.google.com/anji/_/rpc/"

async def rpc(client: httpx.AsyncClient, method: str, payload: dict) -> dict:
    r = await client.post(RPC + method + "?authuser=",
                          data={"f.req": json.dumps(payload, separators=(",", ":"))})
    r.raise_for_status()
    return r.json()

The response for SearchCreatives has the creatives in "1", the next cursor in "2", and the total count range in "4"/"5". Each creative carries its ID, format, a preview URL (image or a YouTube video ID) and the advertiser ID. Regions and run dates come from the per-creative lookup, so a full record costs two requests.

2. Blocking: it is the IP, not the headers

The endpoints do not need cookies or special headers beyond a normal User-Agent. What they do have is a per-IP budget. From a single datacenter IP you get somewhere between a few dozen and a few hundred RPC calls before every response becomes a 302 to google.com/sorry/…. There is no Retry-After; you just get bounced.

Two things fix it:

Rotate IPs every N calls. On Apify I create a proxy configuration once and derive a new session (= new IP) every 8 calls by changing the session-… part of the proxy URL. Do the derivation synchronously if your HTTP calls run in a worker thread — awaiting new_url() from a thread deadlocks.
Back off on 302/429/503 with exponential delay (2, 4, 8, 16 s) and rotate before retrying. Four attempts is enough; if all four are blocked, stop the run instead of burning money.

blocked = r.status_code in (301, 302) and "google.com/sorry" in r.headers.get("location", "")
if blocked or r.status_code in (429, 403, 503):
    await self._rotate()
    await asyncio.sleep(delay); delay *= 2
    continue

One honest caveat: when I ran seven advertiser exports in parallel from the same proxy pool, Google started bouncing every request for about an hour, including ones that had worked minutes earlier, while the container's own egress IP was still fine. Space your runs, and if your proxy pool is shared, keep a direct-connection fallback for the moments the whole pool is bounced.

3. Image ads: the copy is in the pixels

Text ads come back as text. Image ads come back as a preview PNG/JPEG, and for most brands that is 60–80 % of the creatives. The headline you want is inside the image.

Tesseract handles ad creatives surprisingly well because ad copy is large, high-contrast and horizontal. The whole OCR step:

from io import BytesIO
from PIL import Image, ImageOps
import pytesseract

async def ocr_image(client, url: str) -> str | None:
    r = await client.get(url, timeout=20)
    if r.status_code != 200:
        return None
    img = Image.open(BytesIO(r.content)).convert("L")     # greyscale
    if img.width < 600:                                    # upscale small creatives
        img = img.resize((img.width * 2, img.height * 2))
    img = ImageOps.autocontrast(img)
    text = pytesseract.image_to_string(img, lang="eng", config="--psm 6")
    text = " ".join(text.split())
    return text if len(text) >= 3 else None

--psm 6 ("assume a single uniform block of text") beats the default for ads; autocontrast and 2× upscaling recover most of the small legal lines. Install tesseract-ocr and tesseract-ocr-eng in your Dockerfile; on the apify/actor-python image that is one apt-get line.

Expect OCR to add roughly a second per image ad. In the hosted Actor I mark OCR-derived copy with adTextSource: "ocr" so downstream users can treat it as noisy, and I charge the OCR event only when Tesseract actually returned text.

4. Putting it together

The final record per ad looks like this:

{
  "adId": "CR1234…",
  "advertiserId": "AR0987…",
  "advertiserName": "Shopify",
  "format": "IMAGE",
  "adText": "Start selling online today. Free trial.",
  "adTextSource": "ocr",
  "imageUrl": "https://tpc.googlesyndication.com/…",
  "regions": ["US", "CA", "GB"],
  "firstShown": "2026-05-02",
  "lastShown": "2026-09-13",
  "detailUrl": "https://adstransparency.google.com/advertiser/AR0987…/creative/CR1234…"
}

From here it is ordinary data work: diff two weekly exports to see which creatives a competitor killed, group by regions to see where they are expanding, or feed adText to an LLM for messaging analysis.

Takeaways

The Transparency Center is an RPC API with protobuf-style JSON; three methods cover advertiser search, ad listing and ad detail.
The only real defence is per-IP rate limiting — rotate sessions every few calls and back off on 302 → /sorry.
Tesseract with --psm 6, greyscale and 2× upscaling turns image ads into searchable copy.

Source: https://github.com/m2kyungmin/apify-actors/tree/main/ads-transparency-scraper. Hosted, pay-per-ad version: https://apify.com/kyungminlee/ads-transparency-scraper.