DEV Community

# webscraping

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
10 Best AI Web Scraping Tools of 2025

10 Best AI Web Scraping Tools of 2025

10
Comments
11 min read
BeautifulSoup vs lxml: A Practical Performance Comparison

BeautifulSoup vs lxml: A Practical Performance Comparison

2
Comments
3 min read
Using FireCrawl MCP Server with Claude for Web Scraping

Using FireCrawl MCP Server with Claude for Web Scraping

25
Comments 3
5 min read
How to scrape YouTube using Python [2025 guide]

How to scrape YouTube using Python [2025 guide]

7
Comments
18 min read
I Built a Node.js App That Tracks UK Supermarket Prices and Saved Me £300 This Month (Also My Neighbors Love It)

I Built a Node.js App That Tracks UK Supermarket Prices and Saved Me £300 This Month (Also My Neighbors Love It)

22
Comments 1
6 min read
Building a YouTube Scraper with Scrapy: A Developer's Journey

Building a YouTube Scraper with Scrapy: A Developer's Journey

3
Comments 1
4 min read
Scraping Twitter in 2025: A Developer's Guide to Surviving the API Apocalypse

Scraping Twitter in 2025: A Developer's Guide to Surviving the API Apocalypse

6
Comments
3 min read
Building a Production-Ready LinkedIn Scraper with Python Scrapy 🐍

Building a Production-Ready LinkedIn Scraper with Python Scrapy 🐍

1
Comments 1
7 min read
🕸️ Web Scraping in Python: A Practical Guide for Data Scientists

🕸️ Web Scraping in Python: A Practical Guide for Data Scientists

3
Comments
3 min read
AI-Ready Congressional Data: Building a Python Tool for Plaintext Extraction

AI-Ready Congressional Data: Building a Python Tool for Plaintext Extraction

3
Comments
5 min read
Why your site needs a sitemap?

Why your site needs a sitemap?

31
Comments 4
3 min read
🧑‍💻 Web Scraping Indeed Remote Jobs and Storing in PostgreSQL

🧑‍💻 Web Scraping Indeed Remote Jobs and Storing in PostgreSQL

7
Comments 1
2 min read
How to Hide Cookie Banners with Puppeteer

How to Hide Cookie Banners with Puppeteer

6
Comments 1
3 min read
What Are Today’s Top Movie Plots? I Wrote a Python Scraper to Find Out

What Are Today’s Top Movie Plots? I Wrote a Python Scraper to Find Out

2
Comments 1
2 min read
How to Extract Links from a Sitemap

How to Extract Links from a Sitemap

2
Comments
5 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.