DEV Community

frankxie
frankxie

Posted on

Google News Monitoring for a Brand or Topic: Daily Headlines into a Spreadsheet with Python

If you work in PR, marketing or investor relations, someone has probably asked you for "a list of everything written about us this week". Google News is the obvious place to look, because it pulls from thousands of publishers. But it gives you a page to read, not a table. Copying headlines, publishers, dates and links by hand works for one week. It doesn't work every day, or for five competitors.

Google doesn't offer an official Google News API. What it does have is an RSS feed for every search, and that is a reasonable start if you only want to read new headlines in a feed reader. For a spreadsheet it has three problems: one feed returns at most about 100 articles, the links point to news.google.com instead of the publisher, and you still have to write the code that parses, de-duplicates and saves it.

This tutorial uses an Apify Actor that does those parts and exports the results as JSON, CSV or Excel, then shows a short pandas script for a daily monitoring sheet.

Disclosure: I built and maintain this Actor. It is a paid tool ($1.50 per 1,000 results). I've tried to describe what it can't do as plainly as what it can.

What you can do with the data

  • A daily coverage log. Run a search for your brand every morning with "Last 24 hours", append new articles to one sheet, and you have a running record of press coverage.
  • Competitor share of voice. Search several brand names in one run. The searchQuery field tells you which brand each article belongs to, so counting articles per brand per week is one line of pandas.
  • Coverage of a past event. Give a date range and the Actor searches day by day, so you can collect far more than 100 articles about a launch, a recall or an earnings report.
  • Input for other tools. The real article links can go into an article text extractor, a summarizer or a sentiment model.

Step 1: Set up a run in Apify Console

Open Google News Scraper and click Try for free. For monitoring you only need a few fields:

Field What it does
What to scrape (mode) search for keywords. Also topStories, topic (World, Business, Technology and others) and location (local news for a city).
Search keywords (searchQueries) One search per line. Google operators work: "exact phrase", OR, -word, intitle:word.
Max results (maxItems) Upper limit for the run. Default 100.
Time range (timeRange) any, 1h, 1d, 7d, 30d or 1y.
Date from / Date to (dateFrom, dateTo) A date range like 2026-08-01 to 2026-08-31. The Actor then searches week by week and day by day to get past the 100-article limit.
Only these publishers (sites) Keep only articles from sites like reuters.com.
Language / Country (language, country) The Google News edition. Defaults are en and US.
Get the real article links (resolveUrls) Adds the publisher's own URL as articleUrl. Off by default because it takes one extra request per article.

No Google account, API key or proxy is needed.

Step 2: Run it and export

Click Start. When the run finishes, open the Output tab and export as JSON, CSV or Excel.

For this article I searched for one brand over the last week, with real links turned on:

{
  "mode": "search",
  "searchQueries": ["Duolingo"],
  "timeRange": "7d",
  "maxItems": 20,
  "resolveUrls": true
}
Enter fullscreen mode Exit fullscreen mode

It saved 20 articles in about 22 seconds, and all 20 publisher links were decoded. Here are two of them. I removed the articleId field, and the long googleNewsUrl from the first one; the other values are unchanged:

[
  {
    "title": "Duolingo slapped with class-action lawsuit alleging AI efforts, relentless ads hurt user growth",
    "publisher": "New York Post",
    "publisherUrl": "https://nypost.com",
    "publishedAt": "2026-10-27T16:04:00Z",
    "articleUrl": "https://nypost.com/2026/10/27/business/duolingo-slapped-with-class-action-lawsuit-alleging-ai-efforts-relentless-ads-hurt-user-growth/",
    "description": "Duolingo slapped with class-action lawsuit alleging AI efforts, relentless ads hurt user growth New York Post",
    "relatedArticles": [],
    "searchQuery": "Duolingo",
    "topic": null,
    "location": null,
    "language": "en",
    "country": "US",
    "scrapedAt": "2026-11-02T13:26:06Z"
  },
  {
    "title": "Is Duolingo down for you? Here's what's going on (Update: Back)",
    "publisher": "Android Authority",
    "publisherUrl": "https://www.androidauthority.com",
    "publishedAt": "2026-10-27T21:37:00Z",
    "googleNewsUrl": "https://news.google.com/rss/articles/CBMiaEFVX3lxTE55bG42bUlDM0xjQmx6MVFfYjFwT2dETkJWclBLY01RRV9MY2FoTEdLcHBScXJ4T1VJRTBRSXFTWXRHQnAxdEU5cDdMMzctVWluY2JSQ3N5OUthc0xuS0M4VGRpWFN0NFBp?oc=5",
    "articleUrl": "https://www.androidauthority.com/duolingo-outage-3728718/",
    "description": "Is Duolingo down for you? Here's what's going on (Update: Back) Android Authority",
    "relatedArticles": [],
    "searchQuery": "Duolingo",
    "topic": null,
    "location": null,
    "language": "en",
    "country": "US",
    "scrapedAt": "2026-11-02T13:26:06Z"
  }
]
Enter fullscreen mode Exit fullscreen mode

Even 20 rows show why you want this in a table. Four of the 20 were press releases from law firms looking for shareholders after a class action was filed, and three more were stock quote or forecast pages rather than news stories. Mixed in with those were an app outage, a lawsuit story in the New York Post, and marketing pieces about a Duolingo campaign. The results also don't come in date order: the feed is ranked by Google, so sort by publishedAt yourself.

Note that description is not a summary. Google News only puts the headline and the publisher there.

Step 3: A daily monitoring sheet in Python

Install the official client and pandas:

pip install apify-client pandas
Enter fullscreen mode Exit fullscreen mode

This script searches the last 24 hours, tags each headline with simple keyword rules, and appends only new articles to one CSV file. Run it once a day, from cron or a scheduler of your choice:

import os
from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("pipiagent/google-news-scraper").call(run_input={
    "mode": "search",
    "searchQueries": ["Duolingo"],
    "timeRange": "1d",
    "maxItems": 100,
    "resolveUrls": True,
})
df = pd.DataFrame(client.dataset(run.default_dataset_id).iterate_items())
if df.empty:
    raise SystemExit("No articles in the last 24 hours.")
df["publishedAt"] = pd.to_datetime(df["publishedAt"])
df["link"] = df["articleUrl"].fillna(df["googleNewsUrl"])

# Simple keyword tags on the headline. Adjust them to your brand.
TAGS = {
    "legal": r"lawsuit|class action|securities|investors",
    "stock": r"\bstock\b|\bshares\b|earnings|\bDUOL\b|forecast",
    "product": r"\bapp\b|outage|\bdown\b|feature|launch",
}
for tag, pattern in TAGS.items():
    df[tag] = df["title"].str.contains(pattern, case=False, regex=True)

MASTER = "duolingo_news.csv"
cols = ["publishedAt", "publisher", "title", "link", "searchQuery", *TAGS]

if os.path.exists(MASTER):
    old = pd.read_csv(MASTER)
    new_rows = df[~df["link"].isin(old["link"])]
    pd.concat([old, new_rows[cols]]).to_csv(MASTER, index=False)
else:
    new_rows = df
    df[cols].to_csv(MASTER, index=False)

print(f"{len(new_rows)} new articles")
for _, r in new_rows.sort_values("publishedAt", ascending=False).iterrows():
    print(r["publishedAt"].strftime("%Y-%m-%d %H:%M"), r["publisher"], "-", r["title"])
Enter fullscreen mode Exit fullscreen mode

Use the publisher link as the key, as above, rather than the headline: publishers sometimes edit headlines after the first version.

Once the file has a few weeks of rows, a weekly summary is short:

log = pd.read_csv(MASTER, parse_dates=["publishedAt"])
print(log["publisher"].value_counts().head(10))
print(log.set_index("publishedAt").resample("W")[list(TAGS)].sum())
Enter fullscreen mode Exit fullscreen mode

For share of voice, put several brands in searchQueries and group by searchQuery instead of the tags. If you prefer Google Sheets, upload the CSV, or write the DataFrame with a library such as gspread.

What it costs

You pay per saved article: $1.50 per 1,000 results, with no start fee and no monthly rental. Turning on real links doesn't change the price.

  • My 20-article test: $0.03.
  • A daily brand search that returns 100 articles every day for 30 days: $4.50.
  • One month of coverage for a narrow topic: the README lists a test of "quantum computing" for August 2026 that returned 1,216 articles, which is $1.82.

You can also set a maximum charge per run in the run options, and the Actor stops when it reaches that amount.

Limitations

  • About 100 articles per search. This is Google's limit. Without a date range you get the roughly 100 articles Google ranks highest. A date range gets more, but each single day is also capped at about 100, so very busy keywords still miss some articles on busy days. The run log lists the days that hit the cap.
  • No summary or full text. The feed has the headline, publisher, date and link. Use the real links with a separate article extractor if you need the text.
  • Real links are slower. Decoding takes one extra request per article, about 60 to 70 seconds per 100 articles. If Google refuses those requests, the article is still saved with an empty articleUrl.
  • Old articles have a date but no exact time. For articles older than a few weeks, Google gives only the day.
  • Results depend on the edition. The same keyword returns different articles for different language and country settings.
  • Top stories, topics and locations have no history. They show what's on Google News right now, so schedule them if you want a record.

Related tools

If you want to see whether interest in a topic is rising, not just how much is written about it, Google Trends Scraper exports interest over time, by region and related queries. For newsletters, Substack Scraper collects posts with full text from Substack publications.

You can try a search for your own brand on the Google News Scraper store page. If something doesn't work the way this article says, open an issue there and I'll look into it.

Top comments (0)