Monitor Google News for your brand, and get the real article URLs instead of redirects

Google News has no official API anymore, but it still serves RSS feeds for any search, topic and country. That is enough to build a brand-mention monitor, a competitor alert or a news dataset. There are two catches: the feed caps at about 100 items per request, and every link is a news.google.com/rss/articles/... redirect rather than the publisher's URL.

Step 1: the free DIY feed

# pip install feedparser
import feedparser, urllib.parse

q = urllib.parse.quote('"Nvidia" when:7d')
url = f"https://news.google.com/rss/search?q={q}&hl=en-US&gl=US&ceid=US:en"
for e in feedparser.parse(url).entries[:10]:
    print(e.published, "|", e.source.title, "|", e.title)

Operators like site:reuters.com, "exact phrase", intitle: and when:7d work inside the query. Change hl, gl and ceid for another country and language (for example hl=de&gl=DE&ceid=DE:de). Topic feeds use https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en.

Step 2: the redirect problem

e.link points to Google, and older tricks that base64-decode the URL no longer work for most new articles. Resolving them means calling a Google web endpoint per article, which takes a couple of requests each and gets throttled if you go fast. If you only need to read the links in a browser, you can skip decoding: the Google link works there.

Step 3: the managed version

I built a Google News scraper actor that wraps the above and adds the parts that are tedious to maintain: many countries per run, URL decoding, day-by-day splitting to go past 100 results, deduplication, and a "only new" mode for schedules.

No code: open the actor, add queries, pick countries and a time range, tick decodeUrls if you want publisher URLs, Start, export.

Python:

# pip install apify-client
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("bachiembmt/google-news-rss-scraper").call(run_input={
    "queries": ['"Nvidia"', "Nvidia Blackwell"],
    "countries": ["us", "gb"],
    "timeRange": "7d",
    "maxArticlesPerQuery": 100,   # per query and country
    "decodeUrls": True,
    "onlyNew": True,
})
for a in client.dataset(run.default_dataset_id).iterate_items():
    print(a["publishedAt"][:10], a["source"], "|", a["title"])
    print("   ", a["articleUrl"] or a["googleUrl"])

Output fields: query, country, title, source, sourceUrl, publishedAt, googleUrl, articleUrl (filled when decodeUrls is on), snippet. Bing News is available as a second engine (sources: ["google", "bing"]) and returns the publisher URL inside a url= query parameter of its redirect link, but only about 10 to 15 fresh items per query and country.

A daily brand alert

In the Console, schedule the actor daily with onlyNew: true. The actor remembers what each query and country already returned (it stores this in a key-value store named google-news-seen), so each run contains only fresh items. Then pull them:

last = client.actor("bachiembmt/google-news-rss-scraper").last_run(status="SUCCEEDED")
new = list(last.dataset().iterate_items())
if new:
    text = "\n".join(f'- {a["source"]}: {a["title"]} {a["articleUrl"] or a["googleUrl"]}' for a in new)
    print(text)  # post to Slack/Discord/email here

Apify also has built-in integrations (Slack, email, webhooks, Make, n8n) if you prefer to avoid code.

Full text

extractContent: true fetches each article and extracts main text, author, image and language. In my tests about 55% of pages yield text. Paywalls, consent walls and JavaScript-only pages return nothing and are not charged for the content part.

Honest limits

• The feed shows what Google ranks, not a full archive; old dates are patchy.
• Google RSS returns about 100 items per request. splitByDay gets past that for keyword queries, at one request per day.
• URL decoding takes about 1.5 s per article. It is meant for hundreds of articles per run, not thousands. If Google changes or throttles the endpoint, articleUrl is null and googleUrl still works.
• The feed has no summaries for most results (snippet is usually empty).
• Respect publishers' copyright and Google's terms; this only collects public feed metadata.

Price

Pay per event: about $1 per 1,000 articles, plus about $2 per 1,000 for extracted full text (check the Store page for current numbers). A daily monitor with two queries and two countries returning a few dozen new items per day is a few cents a month. Disclosure: I built the actor; the DIY feed in step 1 is free and good enough for a single query.

Try it: Google News scraper