DEV Community: Aniruddha Garje

Google Ads Transparency Center API: export a competitor's Google ads

Aniruddha Garje — Sun, 11 Oct 2026 04:03:53 +0000

Google's Ads Transparency Center shows the ads that advertisers run with Google, and lets you filter them by Search, YouTube, Shopping, Maps and Play. You can browse it by advertiser or website. If you want those ads as data, to watch competitors or build a swipe file, here is what I learned exporting them, and the request I use.

How long competitors keep an ad running

I exported 10 US ads each for three CRM companies: hubspot.com, salesforce.com and zoho.com (run saWgrSwy1OvGlOGjZ). The median days shown was 319.5 for HubSpot, 195 for Salesforce and 757.5 for Zoho. When you study a competitor, read the longest-running ads first.

Across a larger run of 4,578 ads, the median days shown was 164 for text ads, 52 for image ads and 21 for video ads (run CmLk2DJUK1igwGucM).

One website can have several advertisers

The 10 hubspot.com ads came from 3 advertiser accounts. So search by website domain when you want everything a brand runs, and by advertiser ID when you want exactly one account.

You can also search by name. A search for Shopify found Shopify Inc.; its 30 ads were 13 text, 13 video and 4 image, the earliest first shown on 2022-08-13 (run VKplR3pJxcOgnEisJ).

Most text ads come back as images

This surprised me the most. In the CRM export, 27 of the 30 ads were text ads, and 26 of those 27 came back as images. In the larger run, 2,866 of 2,958 text ads (96.9%) were images. The Transparency Center publishes most text ads as one rendered picture, so there is often no headline to copy as text. My export marks each ad's content status, so you know which ads hold text and which hold only an image URL. It does not read text out of images.

Video ads are easier. 30 Sony Music Entertainment video ads all came with a YouTube link and first and last shown dates (run qT42FaC6ihuBOgQVE).

Dates you can rely on

firstShown, lastShown and daysShown came back for every ad in the regions I measured, the US and "anywhere" (runs KX1cJFN14b0BG3Kxk and MrvckmGUntglLNYnJ). One limit to know: the Transparency Center shows per-region first-shown dates and impression ranges only for EU countries. Outside the EU you get a per-region last-shown date.

The request I use

I built an Apify Actor for this, so my export is one HTTP request. You need an Apify account and your API token in the APIFY_TOKEN environment variable.

import os

import requests

url = "https://api.apify.com/v2/acts/garje~google-ads-transparency-scraper/run-sync-get-dataset-items"
run_input = {
    "domains": ["hubspot.com", "salesforce.com", "zoho.com"],
    "region": "US",
    "maxAdsPerDomain": 10,
    "includeCreativeContent": True,
    "proxyConfiguration": {"useApifyProxy": True},
}
resp = requests.post(
    url,
    params={"format": "csv"},
    json=run_input,
    headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
    timeout=300,
)
resp.raise_for_status()
with open("ads.csv", "wb") as f:
    f.write(resp.content)

I ran the same request with 2 ads for one domain, and it returned a CSV with 2 ads (run hxovNrfVkZSYua84X). The price is $1.50 per 1,000 ads, and you are charged only for rows saved.

Watch for new ads

Add "onlyNewSince": "2026-10-01" to the input and you get only ads first shown after that date. Run it weekly and move the date forward, and you see each new creative as it launches.

If you would rather click than code, the competitor comparison is saved as a public example you can run: Google ads of three competing brands in one table.

Sources

What	Where it comes from
Median days shown for the three CRM brands, 3 advertisers, 26 of 27 text ads as images	run saWgrSwy1OvGlOGjZ
Median days shown by format, 96.9% of text ads as images	run CmLk2DJUK1igwGucM
Shopify by name	run VKplR3pJxcOgnEisJ
Sony Music video ads with YouTube links	run qT42FaC6ihuBOgQVE
Dates on every ad, US and anywhere	runs KX1cJFN14b0BG3Kxk and MrvckmGUntglLNYnJ
The request above, 2 ads	run hxovNrfVkZSYua84X

I built the Google Ads Transparency scraper used above, and every week I check it against the live site. I drafted this article with AI help; every number comes from the runs in the table.

Google Hotels API: hotel prices by date as JSON

Aniruddha Garje — Sun, 11 Oct 2026 03:55:55 +0000

Google Hotels shows prices from many booking sites for any city and night. If you want those prices as data, for a price calendar, a daily tracker or a rate check, here is what I learned collecting them, and the request I use.

Prices move with the check-in date

I priced the first 10 hotels Google listed in Barcelona for check-in 21 and 22 days ahead, on 1 and 2 November 2026, for 2 nights and 2 adults, in EUR (run WpExyDflthAbuKfDG).

The median price per night was EUR 122.51 for 1 Nov and EUR 138.38 for 2 Nov.
One hotel listed on both days went from EUR 190.66 to EUR 230.92 per night, 21.1% more, one check-in day later.

The hotel list moves too

The two lists of 10 hotels had only 4 hotels in common. Each date had its own list. So never compare "the first 10 hotels" across dates. Compare the same hotelId across dates.

The list also changes between visits, and it can differ from what your browser shows. In my comparison test, two browser visits 12 minutes apart had 87% of their hotels in common. For the hotels a run did return, prices matched the browser within 10% in 99% of cases.

You choose the currency

Prices come back in the currency you ask for, not the one your location suggests. With currency set to AED, all 20 Dubai prices came back in AED (run Rja3KztI5aZO5DQ5o). For the same night, the median of the first 10 hotels was EUR 103.36 in Lisbon and EUR 113.92 in Porto (run hw8Ld0SQ8Xq9SVcay).

The request I use

I built an Apify Actor for this, so I get the prices with one HTTP request. You need an Apify account and your API token in the APIFY_TOKEN environment variable.

import os

import requests

url = "https://api.apify.com/v2/acts/garje~google-hotels-scraper/run-sync-get-dataset-items"
run_input = {
    "queries": ["hotels in Barcelona"],
    "checkInFromDaysAhead": 21,
    "checkInToDaysAhead": 27,
    "nights": 2,
    "adults": 2,
    "currency": "EUR",
    "maxHotelsPerQuery": 10,
    "proxyConfiguration": {"useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"]},
}
resp = requests.post(
    url,
    json=run_input,
    headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
    timeout=300,
)
resp.raise_for_status()
for row in resp.json():
    print(row["checkInDate"], row["name"], row["pricePerNight"], row["currency"])

Each row carries the check-in date, the hotel's name and ID, the price per night, the currency, and the stay length and occupancy it was priced for. I ran the same request with 2 dates and 2 hotels each, and it returned 4 priced rows in EUR (run igBdVfjBIZWsCAS8m). The price is $1.99 per 1,000 hotel prices, and you are charged only for rows saved.

Make it a daily tracker

checkInFromDaysAhead and checkInToDaysAhead count from the day of the run. Schedule the request daily and the check-in dates move forward by themselves, with no dates to update.

If you would rather click than code, the same setup is saved as a public example you can run: Google Hotels prices by date for 10 hotels.

Sources

What	Where it comes from
Barcelona medians, the hotel that rose, 4 hotels in common	run WpExyDflthAbuKfDG
Dubai prices in AED	run Rja3KztI5aZO5DQ5o
Lisbon and Porto medians	run hw8Ld0SQ8Xq9SVcay
The request above, 4 rows	run igBdVfjBIZWsCAS8m
Prices within 10% of the browser in 99% of cases	run HZKxhoaE3zdIU5HDE
Two browser visits 12 minutes apart, 87% in common	browser baseline, 8 Oct 2026

I built the Google Hotels scraper used above, and every week I check it against the live site. I drafted this article with AI help; every number comes from the runs in the table.

How to export Google Play reviews to CSV for any app and country

Aniruddha Garje — Sun, 11 Oct 2026 03:55:52 +0000

If you need reviews for an app you do not own, or for many apps and countries at once, you have to export them yourself. Here is what a Google Play review export looks like, what is missing from it, and the one request I use to get a CSV.

What one review row holds

Each row I export has these fields:

rating, text and date, in ISO 8601 and UTC
appVersion, the version the reviewer used
developerReply and developerReplyDate
thumbsUpCount
country and language
reviewId and reviewUrl, a link that opens the review on Google Play

Reviewer names are off by default. Most analysis does not need them, and leaving them out keeps the file free of personal data.

What 400 reviews showed me

I exported the 400 newest US reviews of one large app, Spotify (run DkHhBhzqyaoiUPzj3).

They spanned under 16 hours, from 2026-10-09 11:31 to 2026-10-10 03:22 UTC. For a popular app, the review list never stands still. Export on a schedule and keep only what is new.
67 of the 400 (16.8%) had no app version. Plan for empty values before you group by version.
61 of the 400 (15.3%) carried a developer reply.
243 (60.8%) were 5 stars and 69 (17.3%) were 1 star.

Reviews are collected per country

I collect each country separately, and every row carries the country it came from. Ask for two countries and you get both in one table, with the country on each row.

If you also need App Store reviews

Apple's public feed stops at 10 pages of 50 reviews (500) per country and sort order. Page 11 answers HTTP 400. So collect each country separately, and collect often, so new reviews are captured before they drop out of that window.

The two stores do not hold the same fields. In one run with 50 App Store and 50 Google Play reviews of the same app (run uwff59RL3Sq0bfG6i), the App Store rows had no developer reply or language, and the Google Play rows had no title. I keep both stores in one schema: a field a store does not have is empty, never missing. That way both stores line up in one sheet.

The request I use

I built an Apify Actor for this, so my export is one HTTP request. You need an Apify account and your API token in the APIFY_TOKEN environment variable.

import os

import requests

url = "https://api.apify.com/v2/acts/garje~app-store-play-reviews-scraper/run-sync-get-dataset-items"
run_input = {
    "apps": ["com.spotify.music"],
    "stores": ["googlePlay"],
    "countries": ["us", "gb"],
    "sort": "newest",
    "maxReviewsPerApp": 100,
}
resp = requests.post(
    url,
    params={"format": "csv"},
    json=run_input,
    headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
    timeout=300,
)
resp.raise_for_status()
with open("reviews.csv", "wb") as f:
    f.write(resp.content)

maxReviewsPerApp applies to each country. I ran the same request with 5 reviews for each of 2 countries, and it returned a CSV with 10 reviews (run jyfg8laqCOZLLKxGq). The price is $0.12 per 1,000 reviews, and you are charged only for reviews saved.

Keep the export fresh

Add "since": "2026-10-05" to the input and you get only reviews posted on or after that date. Run it on a schedule and move the date forward each time.

If you would rather click than code, the same setup is saved as a public example you can run: Monitor new app reviews since a date.

Sources

What	Where it comes from
400 newest reviews: time span, versions, replies, ratings	run DkHhBhzqyaoiUPzj3
Fields each store does not have	run uwff59RL3Sq0bfG6i
The request above, 10 reviews	run jyfg8laqCOZLLKxGq
since returns only reviews on or after the date	run YWTUOzRZnqWD2Uaze
Apple's 500 limit, page 11 answers HTTP 400	checked for both sort orders on 8 Oct 2026

I built the App Store and Google Play reviews scraper used above, and every week I check it against the live stores. I drafted this article with AI help; every number comes from the runs in the table.

I searched Google Hotels for a city that does not exist. It found five hotels.

Aniruddha Garje — Fri, 09 Oct 2026 13:44:10 +0000

I typed "hotels in Qwzxvbnmplk" into my Google Hotels scraper to watch it fail. It did not fail. It came back with five hotels near San Antonio, Texas, each with a price, a rating and a map position.

This is the story of that answer and the fix. It is also how my fix broke a real capital city, and the small check I now run before I trust any search result.

A test that was supposed to fail

On 8 October I was testing inputs that should go wrong. One of them was a search for a city that does not exist. I expected an error, or an empty table.

Here is what the run returned (run pLMLM09FNCR1f0Tft):

Name	Price per night	Rating	Reviews	Latitude, longitude
Flat with pool & barbeque, Pet-friendly	$75.77	4.665	49	29.404, -98.626
Holiday Inn San Antonio Seaworld by IHG	$83.09	4.4	2,198	29.448, -98.680
Texas Sage One Room Cabin	$113.00	4.725	32	29.777, -98.950
Luxurious Canyon Lake glamping with hot tub + pool	$254.00	4.4	22	29.869, -98.302
Apartment with 1 bedroom for 2 persons	$64.79	4.805	64	29.475, -98.536

Every field was filled. Every price was a real price for a real place. My scraper charges per priced hotel, so each of those five rows would have been a paid result.

Why nothing looked wrong

The same run had two other searches. "hotels in Paris" returned hotels in Paris. "hotels in Springfield" returned five hotels around 39.75, -89.6, which is Springfield, Illinois. Google picked one of the many Springfields and did not say so.

That is the pattern. A search engine always tries to answer. When it cannot match your words to a place, it still shows you hotels somewhere. I do not know why it chose San Antonio.

My scraper checked the shape of the answer. Prices were numbers, ratings were between 1 and 5, every row had coordinates. All of that was true. None of it said whether the answer belonged to the question.

The signal was there all along

Google's results page carries the place it understood. For Paris, the page held a place ID and the name "Paris". For Springfield, a place ID and "Springfield".

For "hotels in Qwzxvbnmplk" it held nothing. The run's own statistics record the place for the nonsense search as null. The source had told me it did not understand. I was not reading that part.

The fix

I changed the order of work:

Resolve the place first. If Google returns no place, skip the search, charge nothing and name it in the run's status message.
Put the place Google understood on every row, in a field called resolvedPlace, so you can see which Springfield you got.
If the resolved place shares no word with the search, log a warning. Only a warning: it never drops results.

The next run said: "Finished with 10 priced hotels. Not recognised by Google, so skipped and not charged: "hotels in Qwzxvbnmplk"." (run 5jrjtwZ72Z9SFK7tY).

Then it skipped Prague

Later that day another run finished with this message: "Finished with 0 priced hotels. Not recognised by Google, so skipped and not charged: "hotels in Prague"." (run YTSa6YvgqYp6QrGc5).

Google had sent the page for Prague without its place data. My new check did exactly what I told it to. It treated a real capital city like a made-up one.

The second fix: when the place is missing, retry the page up to 3 times on a new IP before skipping the search. The next run resolved Prague and still skipped the nonsense search (run WiNWSzkUnEGFirBjx).

A strict check needs a way to be wrong safely. Mine had none until Prague showed me.

The check you can use tomorrow

This is not my scraper's code. It is the idea in a form you can paste into anything that asks a source a question and gets a confident answer back. Think of a geocoder, a search API, an entity lookup or a tool call from a language model.

import re

def words(text):
    return set(re.findall(r"\w+", text.lower()))

def check_answer(question, resolved):
    """resolved is whatever the source says it understood, for example a place name."""
    if not resolved:
        return "skip: the source did not say what it understood"
    if not words(question) & words(resolved):
        return "warn: the source may have answered a different question"
    return "ok"

print(check_answer("hotels in Qwzxvbnmplk", None))          # skip
print(check_answer("hotels in Springfield", "Springfield"))  # ok, but which Springfield?
print(check_answer("hotels in Paris", "Paris"))              # ok

Two rules hide in those lines. Before you count rows, check that the source echoed your question. And keep what it understood next to every row, so a person can see it later.

The same bug in different clothes

On 9 October, while setting up public examples, I ran one that asked for 2 hotels with each booking site's offer. The run delivered 2 hotels and 478 offers (run 3BgiBhR7DZifI3D2P).

The scraper had fetched offers for every hotel it kept on the page, not only for the two it delivered. Each offer is a paid event. Again every row was complete and plausible, and the total was wrong.

After the fix, the same kind of request gave 2 hotels and 49 offers (run BNCMJr19AnUWUQUt4). The check here is simple: compare what you bill against what you delivered, every run.

How do you test a source that disagrees with itself?

After this, I wanted a number for how often my hotel results match what a person sees. So I had a real browser read the live Google Hotels pages, ran the scraper straight after, and compared the two.

The first surprise was the browser. Two browser visits 12 minutes apart had only 87% of their hotels in common. Google reshuffles its list between visits.

A test that demands 95% agreement with a source that agrees with itself 87% of the time will fail forever. Worse, it teaches you to ignore it. So my pass mark for the hotel list is the browser's own agreement, 87.2%, minus 3 points: 84.2%.

Prices get a strict bar, because a price should not drift between two looks at the same hotel. For hotels that appeared in both, the price matched within 10% in 99% of cases.

Separate what the source keeps changing from what it should never get wrong, and test each one against its own bar.

The alarm that fired because the data got better

On 9 October my daily check on Paris hotels failed. It watches how often each field comes back empty, and alarms when that share moves more than 5 points from a saved baseline.

What moved was good news. Fewer hotels than usual were missing a star class: 15%, against 30% in the baseline (run J3MVimtp32Xg6WCHp). On 20 hotels, 5 points is a single hotel, and the list reshuffles between visits.

I changed the alarm to fire only when empty values rise, and only when 3 of the 20 hotels change. The rerun passed (run pql2nTclNeiU8o8sn). A check that cries wolf gets switched off, and then it protects nothing.

What I still cannot know

Google's hotel list changes between visits. In my test, two browser visits 12 minutes apart had 87% of their hotels in common. "Correct" for a list means "matches what Google showed at that moment", not a fixed truth.
My warning is a word check. A search for a landmark can resolve to a city whose name shares no word with it, and then the warning fires for nothing. That is why it only warns.
I can tell you which Springfield Google picked. I cannot tell you whether it is the one you meant. Adding the state to the search, as in "hotels in Springfield, Massachusetts", is what fixes that.

Over to you

Where in your stack does a source answer confidently when it did not understand the question? I would like to hear the strangest example you have found. Reply below.

These checks now live in the Google Hotels scraper I built on Apify: every row carries resolvedPlace, and a search Google does not recognise is skipped and not charged. It is here if you want to see them work: https://apify.com/garje/google-hotels-scraper?utm_source=devto&utm_medium=article&utm_campaign=story&utm_content=city-that-does-not-exist

Sources

Fact or number	Source
"hotels in Qwzxvbnmplk" returned 5 hotels near San Antonio, with the names, prices, ratings, review counts and coordinates in the table	Run pLMLM09FNCR1f0Tft (8 Oct 2026, 08:46 UTC, build 0.1.9), dataset rows; reports/fixes/FIXES.md item (b)
Each priced hotel is a paid result	claims.csv row 70 (pricing); reports/fixes/FIXES.md item (c)
"hotels in Springfield" returned hotels around 39.75, -89.6	Run pLMLM09FNCR1f0Tft dataset rows (coordinates of Springfield, Illinois)
Place recorded as null for the nonsense search; place IDs and names for Paris and Springfield	Run pLMLM09FNCR1f0Tft, RUN_STATS.places (reports/fixes/before.json)
resolvedPlace on every row; unrecognised search skipped, not charged, named in the status message; warning when no word is shared	claims.csv rows 64 and 66 (row 66 UNVERIFIED for the log message wording, so the article only says it logs a warning, as in the code); reports/fixes/FIXES.md item (b); google-hotels-scraper/src/main.py place_matches
Status message after the fix, 10 priced hotels	Run 5jrjtwZ72Z9SFK7tY (build 0.1.10)
Prague skipped, 0 priced hotels	Run YTSa6YvgqYp6QrGc5 (8 Oct 2026, 09:18 UTC, build 0.1.10)
Retry up to 3 times on a new IP; Prague then resolved	reports/fixes/FIXES.md item (b); run WiNWSzkUnEGFirBjx (build 0.1.11, 12 priced hotels, nonsense search skipped)
2 hotels and 478 offers	Run 3BgiBhR7DZifI3D2P (9 Oct 2026, build 0.1.13), chargedEventCounts hotel 2, offer 478
Each offer is a paid event	claims.csv row 70
2 hotels and 49 offers after the fix	Run BNCMJr19AnUWUQUt4 (build 0.1.14); reports/decisions.md row 47
Two browser visits 12 minutes apart had 87% of hotels in common	claims.csv row 67
Adding the state chooses the Springfield	claims.csv row 65 (runs klsxwgZlpaG94y8Bh, 5jrjtwZ72Z9SFK7tY)

I searched for Nike and my scraper never found Nike

Aniruddha Garje — Fri, 09 Oct 2026 13:44:07 +0000

I asked my Google Ads Transparency scraper for the ads of "nike". It returned ads from two advertisers called Nikena and nikey. It returned nothing from Nike, Inc., which Google counts at 9,000 to 10,000 ads.

Every row in that output was valid. Each ad had an ID, a format, dates and a link that opened. This is about two bugs like that, where nothing was missing and the answer was still wrong. It is also about the time the bug turned out to be in my own test.

The search that found the wrong company

The Google Ads Transparency Center lets anyone look up the ads an advertiser has run. You can search by advertiser name. When you type a name, Google suggests a list of advertisers, and you pick one.

My scraper picked for you. It took the first suggestions Google gave. For "nike", Nike, Inc. was eighth on that list. The run asked for 2 ads per advertiser, and the four ads it returned for "nike" came from Nikena and nikey (run w3qudNilMcBOHnIh4).

Here is what Google's suggestion list held, with the number of ads Google reports for each (run T2yQUXYCf9kV2Bl9A):

Advertiser	Country	Ads Google reports
Nike, Inc.	US	9,000 to 10,000
NIKE SRL	IT	8
Nike	KE	1
nikey	BG	10
Nikena	BG	27

A score instead of a guess

The fix was to stop guessing silently. Every suggested advertiser now gets a match score, and every candidate is saved, so you can see what was picked and what was skipped.

This is the actual scoring function from the scraper, shortened only in the list of legal forms:

import difflib, re

LEGAL = {"inc", "llc", "ltd", "srl", "gmbh", "ag", "bv", "sa", "co", "plc", "corp"}

def norm(text):
    return " ".join(w for w in re.findall(r"\w+", text.lower()) if w not in LEGAL)

def match_score(term, name):
    t, n = norm(term), norm(name)
    if not t or not n:
        return 0.0
    if t == n:
        return 1.0
    return round(difflib.SequenceMatcher(None, t, n).ratio(), 2)

for name in ["Nike, Inc.", "NIKE SRL", "nikey", "Nikena"]:
    print(name, match_score("nike", name))
# Nike, Inc. 1.0 / NIKE SRL 1.0 / nikey 0.89 / Nikena 0.8

By default only exact names, a score of 1, are scraped, the advertiser with the most ads first. The next run scraped Nike, Inc. with a score of 1 (run T2yQUXYCf9kV2Bl9A).

Here is the honest part. NIKE SRL in Italy and an advertiser named Nike in Kenya also score 1. The score tells you the names match. It does not tell you they are the brand you meant. That is why every candidate, picked or not, is saved with its score, and why searching by website domain is the safer route when you know it.

Eleven dates, each one day late

The second bug came from an accuracy check. A real browser read the live Transparency Center pages, the scraper ran straight after, and a script compared the two.

25 of 25 ads were found. But the last-shown date matched for only 14 of them (run HM4pQz7xlm09D9wRV). The other 11 were all exactly one day late. For one image ad, the site showed 6 October and my output said 7 October.

The pattern gave it away: every one of the 11 was last shown between midnight and about 8:00 UTC. The Transparency Center displays dates in US Pacific time. My scraper took the UTC timestamp and printed its date. Between midnight UTC and the end of the day in California, those are different dates.

Why only 11? The UTC date and the Pacific date differ only from midnight UTC until midnight in California. An ad last shown in the afternoon UTC gets the same date either way. That is why 14 dates were right and hid the bug: the error appeared only for ads that stopped running in those early UTC hours.

You can see it in three lines (the timestamp is an example, not from the data):

from datetime import datetime
from zoneinfo import ZoneInfo

ts = datetime.fromisoformat("2026-10-07T03:00:00+00:00")
print(ts.date())                                              # 2026-10-07
print(ts.astimezone(ZoneInfo("America/Los_Angeles")).date())  # 2026-10-06

The fix kept the UTC timestamps exactly as they were and added firstShownDate and lastShownDate as the site displays them. The next check matched 25 of 25 (run 6erVdpbUYI1O7psAY).

If your data comes from a page people read, store the raw timestamp and also the date the page shows. People will compare your output with the page, not with UTC.

The time the bug was mine

A later accuracy check found 135 of 146 ads (run DNTVboAQ1sdphxKl1). All 11 misses belonged to one advertiser. My audit had capped the scraper at 30 ads per advertiser, and that advertiser's browser ads sat hundreds of places deep.

With the cap at 1,000, the same audit found 146 of 146 (run G0zl0lGm3dQc33FVz). Nothing in the scraper changed. Before you fix the code, check that the test asked the same question as the browser.

One more thing I did not expect

If you plan to analyse ad copy from this source, know this first. In one run of 4,578 ads, 2,866 of 2,958 text ads, 96.9%, came back as a single image rather than as words (run CmLk2DJUK1igwGucM). You get an image URL, not the headline text.

That raised a quieter problem. A blank headline can mean "this ad has no headline" or "I could not read it". Those are different facts, and a null field hides the difference.

So every ad now carries a contentStatus: ok, imageOnly, externallyHosted, notAvailable, previewFailed or none. In that same run, 79.6% were imageOnly, 14.3% were ok and 6.1% were notAvailable. Image URLs came back for 83% of ads. When a field is empty, the status says why.

Three checks for your own pipeline

If you pull data by name from any source that suggests matches, these came out of the two bugs above:

Never take the first suggestion silently. Score every candidate against what was asked, keep the scores, and pick by rule. Save the candidates you skipped, so a person can see the choice.
Store two times, not one. Keep the raw timestamp for arithmetic, and the date exactly as the page shows it for people. They will compare your output with the page, not with UTC.
Test the test. When a check fails, first confirm it asked the same question as the browser: the same limits, the same time zone, the same page depth.

The safest name is often no name at all. If you know the brand's website, search the Transparency Center by domain. A domain such as nike.com has no spelling to guess.

What I still have not solved

An exact name is not proof of identity. The score narrows the guess; a person still decides.
My scraper does not read text out of images, so for most text ads you get the image, not the words.
Outside the EU, the Transparency Center shows only a last-shown date per region, not a first-shown date or impression ranges. I cannot return what the source does not show.

Over to you

Two habits came out of this: score your matches instead of guessing, and keep both the raw timestamp and the date your users will see. What is the worst silent match you have found in a data pipeline? Reply below, I read every one.

These fixes now live in the Google Ads Transparency scraper I built on Apify. Every row carries the search term and its match score, and dates come both as UTC timestamps and as the site shows them. It is here if you want to try it: https://apify.com/garje/google-ads-transparency-scraper?utm_source=devto&utm_medium=article&utm_campaign=story&utm_content=searching-for-nike

Sources

Fact or number	Source
Search "nike" returned ads from Nikena and nikey and none from Nike, Inc.; 2 ads per advertiser asked, 8 ads delivered across "nike" and "hubspot"	Run w3qudNilMcBOHnIh4 (build 0.1.10), sample rows and RUN_STATS (reports/fixes/before.json); reports/fixes/FIXES.md item (d)
Nike, Inc. was eighth in Google's suggestions	reports/fixes/FIXES.md item (d)
Advertisers, countries and ad counts in the table	Run T2yQUXYCf9kV2Bl9A, SEARCH_MATCHES record (reports/fixes/after.json)
Scoring function and the scores 1.0, 1.0, 0.89, 0.8	google-ads-transparency-scraper/src/parsers.py match_score (legal-form list shortened in the article); scores reproduced locally 9 Oct 2026 and stored in run T2yQUXYCf9kV2Bl9A
Only exact names scraped by default, biggest first; Nike, Inc. scraped with score 1	claims.csv row 27; reports/fixes/FIXES.md item (d); run T2yQUXYCf9kV2Bl9A
25 of 25 found, last-shown date right for 14 of 25, 11 one day late, all last shown between midnight and about 8:00 UTC	reports/ACCURACY_REPORT.md (run HM4pQz7xlm09D9wRV, build 0.1.6)
Example ad shown 6 October, output 7 October	reports/ACCURACY_REPORT.md mismatch table, CR11595459565080018945
Dates displayed in US Pacific time	claims.csv row 26
25 of 25 after the fix	Run 6erVdpbUYI1O7psAY (build 0.1.7)
135 of 146 with a cap of 30, all 11 misses one advertiser; 146 of 146 with a cap of 1,000	reports/launch/gate_sheet.md; reports/decisions.md row 25; runs DNTVboAQ1sdphxKl1 and G0zl0lGm3dQc33FVz
2,866 of 2,958 text ads (96.9%) came back as images, in a run of 4,578 ads	claims.csv rows 31 and 76 (run CmLk2DJUK1igwGucM)
The scraper does not read text out of images	claims.csv row 33
Per-region first-shown dates and impression ranges only for EU countries	claims.csv row 36
UTC and Pacific dates differ only from midnight UTC until midnight in California; 14 of 25 right	Time zone arithmetic; reports/ACCURACY_REPORT.md (all 11 misses last shown between midnight and about 8:00 UTC)
contentStatus values; imageOnly 79.6%, ok 14.3%, notAvailable 6.1%; image URLs for 83%	claims.csv rows 29 and 32 (run CmLk2DJUK1igwGucM)
Ads can be looked up by website domain	claims.csv row 23

Apple's review feed went quiet, and my scraper believed it

Aniruddha Garje — Fri, 09 Oct 2026 13:43:48 +0000

Apple's public App Store review feed sometimes answers with an empty page that still has reviews behind it. My scraper read the first empty page as the end of the data. For WhatsApp in four countries it collected 50 reviews, and after the fix the same request collected 1,050.

This is how I found it, why my first fix was not enough, and the one question to ask of any paged source before you trust an empty page.

How the feed works

Apple publishes App Store reviews through a public JSON feed. You ask for an app, a country and a sort order, and you page through the results 50 at a time.

The feed has a hard end. It gives 10 pages, 500 reviews, per country and sort order. Ask for page 11 and it answers HTTP 400.

So the natural rule is: walk the pages, and stop when a page comes back empty. That rule is in a lot of scrapers. It was in mine.

The page that stayed empty

On 8 October the feed started returning some pages empty. They were not the end. Reviews existed past them.

One page stayed empty on 45 tries over 105 seconds. My scraper never made 45 tries. It saw the first empty page and stopped.

For WhatsApp across four countries, that meant 50 App Store reviews (run 44E0AdNn8IGPY6454). Nothing failed. No error was raised. The run simply finished early and looked complete.

What an empty page looks like

I saved one of those empty pages from 8 October. It is a perfectly normal-looking answer. Here it is, shortened:

{
  "feed": {
    "author": {"name": {"label": "iTunes Store"}},
    "updated": {"label": "2026-10-08T01:54:23-07:00"},
    "title": {"label": "iTunes Store: Customer Reviews"},
    "id": {"label": "https://itunes.apple.com/us/rss/customerreviews/page=2/id=310633997/sortby=mostrecent/json"}
  }
}

The title, the author, the timestamp and the page address are all there. Only the entry list is missing, and entry is where the reviews live. The saved file is 895 bytes. A normal page for the same app, with its 50 reviews, is 37,971 bytes.

Nothing about that response says "error". If your code asks "did the request succeed?" the answer is yes. If it asks "is this the last page?" the honest answer is "I cannot tell from this".

Fix 1: do not stop at an empty page

The first fix changed three things. Retry an empty page. If it stays empty, carry on to the next page instead of stopping. And list every page that stayed empty in the run summary, so a person can see what may be missing.

The same request then collected 1,050 reviews (run fnfG0xKFneKM5RKp1). I thought that was the end of it.

The check that failed anyway

Before launch I ran an accuracy check. A real browser read the live review pages, the scraper ran straight after, and a script compared the two.

Recall was 61.2%: 101 of 165 reviews (run bf4Uu4WnKX4qwbqNv). All 64 misses were App Store reviews, and every one sat on a page Apple had served empty to the scraper.

The run's own message said: "Apple returned empty pages for 12 app and country pairs even after retries". It would have been easy to stop there. Apple was flaky, the scraper said so, nothing to fix.

But the browser had seen those reviews. The data existed. My retries came one straight after another, so they all saw the same empty page.

Fix 2: come back later

The second fix spreads the tries out in time. When a page stays empty, the scraper carries on with the rest of the walk and comes back to it: 3 more tries, 15 seconds apart.

The next check found 165 of 165 (run Sr0gowqUd4rynW5tH). That run still reported 3 app and country pairs where a page stayed empty after every try. Those are listed in the run summary, not hidden.

Here is the idea as a sketch you can adapt. It is not the scraper's code.

import time

def walk(fetch, last_page=10, tries=4, late_tries=3, wait=15):
    """fetch(page) returns a list of rows, or [] when the page comes back empty."""
    rows, stuck = [], []
    for page in range(1, last_page + 1):
        got = []
        for _ in range(tries):
            got = fetch(page)
            if got:
                break
        if got:
            rows += got
        else:
            stuck.append(page)        # an empty page is not the end: keep walking
    for page in list(stuck):          # come back later, when the page may have filled in
        for _ in range(late_tries):
            time.sleep(wait)
            got = fetch(page)
            if got:
                rows += got
                stuck.remove(page)
                break
    return rows, stuck                # report what stayed empty instead of hiding it

The question behind it: what is this source's real end signal? For Apple's feed it is page 10, and an HTTP 400 on page 11. An empty page is not that signal. If you only know "empty means done", you will stop early the day the source has a bad minute.

Two stores, two ways to say "the end"

The same scraper reads Google Play, and Google Play ends differently. Each page comes with a token for the next page. When there is no token, there are no more reviews.

Apple says it another way: 10 pages, then an HTTP 400 on page 11. Neither store uses an empty page to mean the end. I had invented that rule myself, because it is usually true.

How much the quiet pages cost

In the failed check, 64 of 165 reviews were missing. That is 38.8% of the reviews a person could see in the browser, and every one sat on a page the scraper was told was empty.

A scraper that stops early is harder to catch than one that crashes. A crash shows up in red. An early finish shows up as a smaller, tidy dataset that nobody questions.

How I watch for it now

Every run lists the pages that stayed empty after all tries, in its summary and in its status message. A small fixed run checks the scraper every morning. Once a week a real browser reads the live pages again and the scraper is compared against it.

None of that stops Apple from serving an empty page. It makes sure I hear about it the day it happens, not the day a user notices.

The time the bug was mine

The same kind of check, on Google Play, first showed 13 reviews with a date one day off. That one was not the scraper. Google Play displays the UTC date, and all 13 reviews were written after 18:30 UTC, which is midnight in India, where my check ran. My comparison used local time.

I changed the comparison to UTC. No expected value was edited to make the numbers pass. On that earlier build, the check then matched 165 of 165 on every compared field (run X6yvF3mZldZch9wZt).

What I could not fix

Apple's public feed stops at 500 reviews per country and sort order. Page 11 answers HTTP 400. The App Store website loads more through an access token issued to Apple's own web app, which is not a public method, so I do not use it. Adding countries is the honest way to get more.
Some pages stay empty even after every try. I cannot make Apple serve them. I can only tell you exactly which ones.

Over to you

Look at the paged sources in your own code. What does each one use to say "this is the end"? If the answer is "an empty page", I would like to know whether you have seen it lie. Reply below.

These retries now live in the App Store and Google Play reviews scraper I built on Apify, and every run lists any page that stayed empty. It is here if you want to try it: https://apify.com/garje/app-store-play-reviews-scraper?utm_source=devto&utm_medium=article&utm_campaign=story&utm_content=feed-that-went-quiet

Sources

Fact or number	Source
10 pages of 50 (500) per country and sort order; page 11 answers HTTP 400	claims.csv row 11; reports/fixes/FIXES.md item (f)
Apple's feed returned some pages empty although they held reviews	claims.csv row 12; reports/fixes/FIXES.md item (f) extra
One page stayed empty on 45 tries over 105 seconds	reports/fixes/FIXES.md item (f) extra
50 App Store reviews for WhatsApp in 4 countries before the fix	Run 44E0AdNn8IGPY6454 (8 Oct 2026, build 0.1.5); reports/fixes/FIXES.md
1,050 reviews after fix 1	Run fnfG0xKFneKM5RKp1 (build 0.1.6)
Recall 61.2%, 101 of 165; all 64 misses App Store reviews on pages served empty	reports/decisions.md row 4; reports/launch/gate_sheet.md (run bf4Uu4WnKX4qwbqNv, build 0.1.10)
Status message "Apple returned empty pages for 12 app and country pairs even after retries"	Run bf4Uu4WnKX4qwbqNv status message
Late retries: 3 more tries, 15 seconds apart, while the walk carries on	reports/decisions.md row 4; app-store-play-reviews-scraper/src/main.py APPLE_LATE_TRIES, APPLE_LATE_WAIT
165 of 165 after fix 2; 3 pairs still empty after every try	reports/decisions.md row 5; run Sr0gowqUd4rynW5tH (build 0.1.11) status message
13 Google Play dates one day off; Google Play displays the UTC date; all 13 written after 18:30 UTC; no expected value changed	reports/FINAL_ACCURACY.md section 1
The check ran on a browser in India	reports/ACCURACY_REPORT.md ("a real Chromium browser on this computer, in India")
165 of 165 on every compared field	claims.csv row 49; reports/FINAL_ACCURACY.md (run X6yvF3mZldZch9wZt)
More reviews need an access token issued to Apple's web app, not a public method	reports/fixes/FIXES.md item (f)
Google Play pages run until the store has no more reviews (a next-page token)	claims.csv row 10 (run bpwfcFd4HmPVSohLv; test_play_page_parses_reviews_token_and_replies)
64 of 165 missing is 38.8%	Computed from reports/decisions.md row 4 (64 / 165)
Empty pages listed in the run summary and status message	claims.csv row 12
A fixed check runs every morning; a browser comparison runs every week	Daily canary task canary-reviews (06:00 UTC, reports/decisions.md row 40 context); claims.csv row 50
The empty page: fields present, no `entry` list, 895 bytes; a 50-review page is 37,971 bytes	Saved responses app-store-play-reviews-scraper/tests/fixtures/apple_rss_us_310633997_p2_empty_flaky.json and apple_rss_us_310633997_p1.json (8 Oct 2026); the article shows the empty page shortened