DEV Community

# webscraping

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
arXiv Scraper: build a RAG-ready paper corpus for $1.50/1K

arXiv Scraper: build a RAG-ready paper corpus for $1.50/1K

Comments
7 min read
Google Ads Transparency Scraper: pull any competitor's ads for $1.20/1K

Google Ads Transparency Scraper: pull any competitor's ads for $1.20/1K

Comments
7 min read
Building Instagram Data Workflows with HikerAPI (Without Maintaining Scrapers)

Building Instagram Data Workflows with HikerAPI (Without Maintaining Scrapers)

Comments
3 min read
Choosing the Right Scraping Interface for LLM Workflows

Choosing the Right Scraping Interface for LLM Workflows

1
Comments
4 min read
USPTO Patent Search API: scrape US patents to clean JSON for $3/1K

USPTO Patent Search API: scrape US patents to clean JSON for $3/1K

1
Comments
7 min read
I Tested Every Web Scraping Tool Against Lazada — Here's What Actually Works (May 2026)

I Tested Every Web Scraping Tool Against Lazada — Here's What Actually Works (May 2026)

Comments
7 min read
The 7 Best Reddit Scrapers in 2026 (Free & Paid, Tested)

The 7 Best Reddit Scrapers in 2026 (Free & Paid, Tested)

Comments
5 min read
How I Built a Google Shopping Scraper with Python & Playwright

How I Built a Google Shopping Scraper with Python & Playwright

Comments
1 min read
How to scrape Nextdoor for hyper-local demographics and community sentiment

How to scrape Nextdoor for hyper-local demographics and community sentiment

5
Comments
9 min read
How China-focused funds turn Weibo into alt-data (Python, 2026)

How China-focused funds turn Weibo into alt-data (Python, 2026)

Comments
4 min read
Scraping Without Tests Is Gambling. And the House Always Wins.

Scraping Without Tests Is Gambling. And the House Always Wins.

1
Comments
3 min read
Give Your AI Agent a Web-Fetch Tool: a 60-Line MCP Server (Free, Self-Hosted)

Give Your AI Agent a Web-Fetch Tool: a 60-Line MCP Server (Free, Self-Hosted)

Comments 2
10 min read
Anonymous Proxies: How Modern Websites Decide Whether to Trust Your Traffic

Anonymous Proxies: How Modern Websites Decide Whether to Trust Your Traffic

Comments
7 min read
We patched Chromium with 49 C++ hooks to beat Cloudflare — here's how BrowserHand works

We patched Chromium with 49 C++ hooks to beat Cloudflare — here's how BrowserHand works

Comments
1 min read
How to scrape Google Play data with Node.js (no API key needed)

How to scrape Google Play data with Node.js (no API key needed)

1
Comments
4 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.