DEV Community

# webscraping

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Robots are not that bad

Robots are not that bad

1
Comments
2 min read
Benchmarking LLM Search APIs: Tavily vs Web Search Plus vs OpenAI Web Search

Benchmarking LLM Search APIs: Tavily vs Web Search Plus vs OpenAI Web Search

Comments
4 min read
Transformando Trabalho em Conteúdo: Gerando Pautas para Lives com Python, RSS e Reddit

Transformando Trabalho em Conteúdo: Gerando Pautas para Lives com Python, RSS e Reddit

Comments
4 min read
Building a Lightning-Fast NSE Stock Data Scraper: From API Calls to Full-Stack Web App

Building a Lightning-Fast NSE Stock Data Scraper: From API Calls to Full-Stack Web App

1
Comments
6 min read
Simple Auto Login Bot

Simple Auto Login Bot

11
Comments 2
2 min read
HasData vs Oxylabs vs ScrapingBee: Cost, Reliability & Output Quality

HasData vs Oxylabs vs ScrapingBee: Cost, Reliability & Output Quality

1
Comments
2 min read
🕵️‍♂️ Social Media First Post Finder

🕵️‍♂️ Social Media First Post Finder

Comments
1 min read
🔎 CloakDetect: Revealing the Hidden Side of the Web

🔎 CloakDetect: Revealing the Hidden Side of the Web

1
Comments
1 min read
Building a Production-Ready Apple Store Reviews Scraper with Python & Scrapy

Building a Production-Ready Apple Store Reviews Scraper with Python & Scrapy

Comments
7 min read
Scylla: A Proxy Pool Solution for Web Scraping

Scylla: A Proxy Pool Solution for Web Scraping

Comments
1 min read
G2.com Scraping: Handling Anti-Bot Measures with Scrapy

G2.com Scraping: Handling Anti-Bot Measures with Scrapy

Comments
4 min read
How I Built a WhatsApp Ad Leads Finder with Apify

How I Built a WhatsApp Ad Leads Finder with Apify

Comments 1
2 min read
Scraping eBay's 1.2 Billion Listings: A Scrapy Tale

Scraping eBay's 1.2 Billion Listings: A Scrapy Tale

Comments
4 min read
Building a Discord Controlled Web Scraper with Scrapy & Zyte API

Building a Discord Controlled Web Scraper with Scrapy & Zyte API

Comments 1
12 min read
A Faster Approach to Email Validation: Why We Ditched Regex

A Faster Approach to Email Validation: Why We Ditched Regex

Comments
3 min read
How to Block AI Web Scrapers on Your WordPress Site with robots.txt (2025 Update)

How to Block AI Web Scrapers on Your WordPress Site with robots.txt (2025 Update)

Comments
2 min read
Pinterest Scraping in 2025: Why I Built a Production-Ready Scrapy Spider (And You Should Too)

Pinterest Scraping in 2025: Why I Built a Production-Ready Scrapy Spider (And You Should Too)

Comments
4 min read
Data Extraction in Automated Workflows: The Competitive Edge

Data Extraction in Automated Workflows: The Competitive Edge

39
Comments 16
5 min read
Scraping Twitter in 2025: A Developer's Guide to Surviving the API Apocalypse

Scraping Twitter in 2025: A Developer's Guide to Surviving the API Apocalypse

5
Comments
3 min read
Building a Production-Ready Target Scraper: 5 Spiders That Actually Work in 2024

Building a Production-Ready Target Scraper: 5 Spiders That Actually Work in 2024

Comments
6 min read
Automating my University's STS Course & Lecturer Evaluation Process with Python + Selenium

Automating my University's STS Course & Lecturer Evaluation Process with Python + Selenium

Comments 3
1 min read
10 Best AI Web Scraping Tools of 2025

10 Best AI Web Scraping Tools of 2025

4
Comments
11 min read
BeautifulSoup vs lxml: A Practical Performance Comparison

BeautifulSoup vs lxml: A Practical Performance Comparison

2
Comments
3 min read
Proxies Alone Aren’t (Always) the Answer

Proxies Alone Aren’t (Always) the Answer

Comments
4 min read
🕸️ Web Scraping & Automation: How to Ethically Collect Web Data Without Getting Blocked (or Banned)

🕸️ Web Scraping & Automation: How to Ethically Collect Web Data Without Getting Blocked (or Banned)

5
Comments 3
3 min read
loading...