DEV Community

Ken-Mutisya
Ken-Mutisya

Posted on

Scrape Google News for Any Topic With Keyless RSS

#api #javascript #webdev #showdev

Media monitoring tools are expensive, and most "news API" plans bill per call for data Google already gives away. If you just need recent articles for a topic, brand or ticker, Google News has a keyless RSS feed that covers it.

The feed

Every Google News search has an RSS version:

GET https://news.google.com/rss/search?q=tesla&hl=en-US&gl=US&ceid=US:en
Enter fullscreen mode Exit fullscreen mode

No key, no header, no login. It returns up to about 100 <item> entries, each with a title, link, pubDate, and a <source url="..."> publisher. Three params control locale:

  • hl is the language, e.g. en-US
  • gl is the country, e.g. US
  • ceid is country:language, e.g. US:en

Narrow by recency

The single most useful trick: Google News honors search operators inside q. Append when:1d or when:7d to get only recent items, which is exactly what monitoring needs:

?q=tesla%20when:1d
Enter fullscreen mode Exit fullscreen mode

Parse it cleanly

RSS is regular, so a small parser is enough. With cheerio in XML mode:

import * as cheerio from 'cheerio';
const xml = await (await fetch(url)).text();
const $ = cheerio.load(xml, { xmlMode: true });
const rows = $('item').toArray().map((el) => {
  const it = $(el);
  const source = it.find('source').text().trim();
  let title = it.find('title').text().trim();
  if (source && title.endsWith(` - ${source}`)) title = title.slice(0, -(source.length + 3));
  return { title, source, url: it.find('link').text().trim(), publishedAt: it.find('pubDate').text().trim() };
});
Enter fullscreen mode Exit fullscreen mode

One detail worth handling: Google News titles end with - Source, and the publisher is also in the <source> tag, so strip the suffix to get a clean headline.

I packaged it

I turned this into a small Actor on Apify so it is callable from code without wiring the feeds and parsing by hand. You pass search terms, a language and country, and a recency window, and it returns one row per article with headline, source, link and date. It joins a growing set of keyless scrapers I have been shipping. The first rows of every run are free so you can check the output first.

For brand monitoring, competitor tracking, or feeding a ticker news panel, this is usually all you need, and it costs nothing to run.

Top comments (0)