DEV Community

Алексей Спинов
Алексей Спинов

Posted on

Scrape Any Website in 3 Steps: URL Code Data (Beginner Template)

Here is a copy-paste template to scrape any website in 3 steps.

Step 1: Choose Your URL

const URL = "https://example.com/products"; // Change this
Enter fullscreen mode Exit fullscreen mode

Step 2: Copy This Code

const cheerio = require("cheerio");
const fs = require("fs");

async function scrape(url) {
  console.log(`Scraping: ${url}`);

  const res = await fetch(url, {
    headers: { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" }
  });

  if (!res.ok) {
    console.error(`Error: ${res.status}`);
    return [];
  }

  const $ = cheerio.load(await res.text());

  // CHANGE THESE SELECTORS for your target site
  const items = [];
  $("SELECTOR_HERE").each((i, el) => {
    items.push({
      title: $(el).find("TITLE_SELECTOR").text().trim(),
      link: $(el).find("a").attr("href"),
      // Add more fields as needed
    });
  });

  return items;
}

async function main() {
  const data = await scrape(URL);
  console.log(`Found ${data.length} items`);
  fs.writeFileSync("output.json", JSON.stringify(data, null, 2));
  console.log("Saved to output.json");
}

main();
Enter fullscreen mode Exit fullscreen mode

Step 3: Find Your Selectors

  1. Open the target page in Chrome
  2. Right-click the data you want → Inspect
  3. Copy the CSS selector
  4. Replace SELECTOR_HERE in the code

Run It

npm init -y && npm install cheerio
node scraper.js
Enter fullscreen mode Exit fullscreen mode

Need Help?


Rather have someone do it for you? $20. Any website. Email: Spinov001@gmail.com | Hire me

Top comments (0)