How to Scrape News Websites and Build a Custom RSS Feed

#javascript #beginners #tutorial #webdev

Most news sites dont offer RSS anymore. Build your own.

Strategy 1: Find Hidden RSS

Many sites still have RSS feeds:

https://site.com/feed
https://site.com/rss
https://site.com/feed.xml
https://site.com/atom.xml

Strategy 2: Google News RSS (Any Topic)

https://news.google.com/rss/search?q=YOUR+TOPIC&hl=en

Free, no key, 100 articles per query.

Strategy 3: Build Custom RSS from HTML

const cheerio = require("cheerio");

async function scrapeToRSS(url, selectors) {
  const res = await fetch(url);
  const $ = cheerio.load(await res.text());

  const items = $(selectors.item).map((i, el) => ({
    title: $(el).find(selectors.title).text().trim(),
    link: $(el).find("a").attr("href"),
    date: new Date().toISOString()
  })).get();

  // Generate RSS XML
  const rss = `<?xml version="1.0"?>\n<rss version="2.0">\n<channel>\n<title>Custom Feed</title>\n${items.map(i => `<item><title>${i.title}</title><link>${i.link}</link></item>`).join("\n")}\n</channel>\n</rss>`;

  return rss;
}