How to Build a News Aggregator with Web Scraping (5 Free Sources)

#javascript #beginners #tutorial #webdev

A news aggregator that pulls from multiple sources is easy to build with free APIs.

The 5 Free News Sources

Google News RSS — news.google.com/rss/search?q=QUERY
Hacker News Algolia — hn.algolia.com/api/v1/search?query=QUERY
Reddit — reddit.com/r/SUBREDDIT.json
Wikipedia Current Events — en.wikipedia.org/w/api.php
arXiv — export.arxiv.org/api/query

All free. No API keys.

The Aggregator

async function aggregateNews(topic) {
  const sources = [
    fetchGoogleNews(topic),
    fetchHN(topic),
    fetchReddit(topic),
    fetchWikipedia(topic),
    fetchArxiv(topic)
  ];

  const results = await Promise.allSettled(sources);
  const articles = results
    .filter(r => r.status === "fulfilled")
    .flatMap(r => r.value);

  // Sort by date
  articles.sort((a, b) => new Date(b.date) - new Date(a.date));
  return articles;
}

Output

[
  { "title": "...", "source": "Google News", "date": "2026-03-21" },
  { "title": "...", "source": "Hacker News", "date": "2026-03-21" },
  { "title": "...", "source": "Reddit", "date": "2026-03-21" }
]