DEV Community

dodou
dodou

Posted on

Fetch Google News via a SERP API in 15 Lines of Node

#node #api #news #webscraping

I wanted a cheap way to keep an eye on what the press says about a niche I follow — new product launches, funding rounds, the occasional drama. Setting up Google Alerts felt passive, and scraping the news tab meant fighting CAPTCHAs for data I only needed as structured rows. A SERP API with a news endpoint turned out to be the shortest path: one POST, get JSON back.

Here's the whole thing in Node, no dependencies beyond the built-in fetch (Node 18+).

The code 

const resp = await fetch("https://api.serpbase.dev/google/news", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "X-API-Key": process.env.SERPBASE_API_KEY,
  },
  body: JSON.stringify({ q: "serp api", hl: "en", gl: "us" }),
});

const data = await resp.json();

for (const item of data.news.slice(0, 10)) {
  console.log(`${item.rank}. [${item.source}] ${item.title}`);
  console.log(`   ${item.link} (${item.time})`);
}
Enter fullscreen mode Exit fullscreen mode

That's the news tab of Google, as rows. The /google/news endpoint costs 1 credit per call, same as web search.

What comes back

Each item in the news array carries the fields that matter for monitoring:

Field Use
rank Position on the news page — who's winning the story today
title / link The headline and where it points
source Publisher name, so you can group by outlet
time Relative age ("3 hours ago") — freshness check for free
snippet First lines of the story, enough for triage
thumbnail_url If you're building a digest email

I verified this structure against the official docs today. One detail worth knowing: there's no num parameter — paging is done with page (1-based), so a deep crawl is just a loop over pages.

What I do with it

A daily cron hits the endpoint with five keywords, appends the rows into a SQLite table keyed on link, and flags anything younger than 24 hours. Because each response is structured JSON, deduplication is a one-line INSERT OR IGNORE instead of DOM diffing. Failed requests aren't billed, so a flaky cron doesn't cost money.

Next step

Set SERPBASE_API_KEY (new accounts get 100 free searches, no card) and run the snippet against a keyword you actually track. Parameter details and the full response schema are in the SerpBase /google/news endpoint docs. The first useful signal is always the same: which outlets keep showing up for your niche.

Top comments (0)