DEV Community

#webscraping

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Building a Python-based Forex Scraper and MCP Server: Forex-Pytory

Building a Python-based Forex Scraper and MCP Server: Forex-Pytory

Comments
1 min read
Why Data Collection Systems Work for 10 Minutes and Fail After 10,000 Requests

Why Data Collection Systems Work for 10 Minutes and Fail After 10,000 Requests

Comments
6 min read
I added a new feature to a live paid actor — the data was already there, I just wasn't reading it

I added a new feature to a live paid actor — the data was already there, I just wasn't reading it

Comments
2 min read
Best Python Libraries for Web Scraping Without Getting Blocked in 2026

Best Python Libraries for Web Scraping Without Getting Blocked in 2026

Comments
7 min read
Choosing Between Scraping APIs, Browser Sessions, and Self-Managed Playwright

Choosing Between Scraping APIs, Browser Sessions, and Self-Managed Playwright

Comments
5 min read
EffatĂ : Chronicle of a Human-AI Collaboration for a Different Kind of Search Engine by DeepSeek, AI assistant

EffatĂ : Chronicle of a Human-AI Collaboration for a Different Kind of Search Engine by DeepSeek, AI assistant

1
Comments
7 min read
Building a Resilient Meta Ads Scraper: What Breaks (and What I Learned Fixing It)

Building a Resilient Meta Ads Scraper: What Breaks (and What I Learned Fixing It)

Comments
4 min read
Google Trends will 429 you into the ground — here's how I got past it

Google Trends will 429 you into the ground — here's how I got past it

4
Comments 3
5 min read
What breaks when you scrape Steam Community Market at scale

What breaks when you scrape Steam Community Market at scale

1
Comments 6
2 min read
Extracting and Organizing Content from Older Websites: A Solution for Structured Documentation Including Mouse-Over Images

Extracting and Organizing Content from Older Websites: A Solution for Structured Documentation Including Mouse-Over Images

Comments
7 min read
Async Scraping Jobs Are Usually a Better Fit for RAG Ingestion Than Blocking Requests

Async Scraping Jobs Are Usually a Better Fit for RAG Ingestion Than Blocking Requests

Comments
4 min read
I scraped live betting odds from 9 countries and never once hit a real anti-bot wall

I scraped live betting odds from 9 countries and never once hit a real anti-bot wall

Comments
3 min read
Building an Instagram-powered app without managing scraping infrastructure

Building an Instagram-powered app without managing scraping infrastructure

Comments
2 min read
Stop parsing HTML to get product data — the JSON is already in the page

Stop parsing HTML to get product data — the JSON is already in the page

2
Comments 2
4 min read
Reading Google Play and App Store reviews straight from their JSON, no browser

Reading Google Play and App Store reviews straight from their JSON, no browser

Comments 2
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.