If you have spent any time building scrapers, you know the hard part is rarely the parsing. It is the wall in front of the page: DataDome on retail sites, reCAPTCHA on communities, GeeTest on marketplaces, Akamai on finance portals. Every project rewrites the same glue — launch a real browser, rotate fingerprints, get past the check, retry on failure — before it can return a single row of data.
We have been building that glue for years, and somebody finally put it all in one place: GitHub organization 2scraper (https://github.com/2scraper) now holds 66 open-source scrapers for the sites people actually scrape.
The coverage follows real demand. Retail and marketplaces: Amazon, Shein, Etsy, StockX, Farfetch, Lidl, Mediamarkt, Tokopedia, Dubizzle, Avito, Flippa. Social: YouTube, TikTok (profiles, videos, ads and shop), Instagram, Snapchat, Weibo, Facebook pages, marketplace and ad library, plus Goodreads, Quora, Medium, Hacker News and Perplexity. Money: Binance, Google Finance, TipRanks, screener.in and Polymarket. Real estate: Rosreestr and Zimmo. Cars: Autotrader and Webmotors. Travel: Skyscanner, MakeMyTrip, Vrbo and Foodpanda.
What makes the collection a system rather than a pile of scripts is that every scraper ships the same way. If the site allows a no-browser HTTP path, you get it. Otherwise the same scraper runs on Playwright, Selenium or Puppeteer. And the protection that specific site actually uses — reCAPTCHA, DataDome, Akamai, GeeTest and friends — is already handled inside, along with proxies and browser fingerprints where they matter. Output is JSON or CSV, so a scraper is a small program you pipe into your pipeline, not a notebook.
Pick the site you need and you start from a scraper that has already survived the wall.
Top comments (0)