DEV Community

#webscraping

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
SEC EDGAR Rate Limit and User-Agent: Fair Access Rules

SEC EDGAR Rate Limit and User-Agent: Fair Access Rules

1
Comments 1
6 min read
JobSpy doesn't scrape company career sites. Add Greenhouse, Lever and Workday jobs in the same DataFrame

JobSpy doesn't scrape company career sites. Add Greenhouse, Lever and Workday jobs in the same DataFrame

Comments 1
4 min read
Idealo price API: compare shop offers across 6 European markets

Idealo price API: compare shop offers across 6 European markets

Comments
7 min read
I ran the same Nike search on the Meta Ad Library three times and got three different answers

I ran the same Nike search on the Meta Ad Library three times and got three different answers

Comments
6 min read
How Sinner won Wimbledon 2025, in break points

How Sinner won Wimbledon 2025, in break points

Comments
4 min read
Track competitor prices on any Shopify store with products.json (and catch restocks)

Track competitor prices on any Shopify store with products.json (and catch restocks)

1
Comments
1 min read
Java proxy setup: HttpClient, Authenticator and SOCKS5

Java proxy setup: HttpClient, Authenticator and SOCKS5

Comments
2 min read
11 scrapers for Chinese platforms in 8 days, and my first $4

11 scrapers for Chinese platforms in 8 days, and my first $4

Comments 2
4 min read
Asking KAYAK for 10 Flights Instead of 50 Made the Same Seat 30% Dearer

Asking KAYAK for 10 Flights Instead of 50 Made the Same Seat 30% Dearer

Comments
6 min read
What a data-quality page for scraped price data looks like (and the numbers behind ours)

What a data-quality page for scraped price data looks like (and the numbers behind ours)

1
Comments 7
5 min read
How to get AliExpress product variants, stock and shipping as JSON with Python

How to get AliExpress product variants, stock and shipping as JSON with Python

Comments
4 min read
My Research Journal: I Thought Web Scraping Was the Hard Part

My Research Journal: I Thought Web Scraping Was the Hard Part

Comments
2 min read
Three Supermarkets Spell One Brand Three Ways, So My Price Comparison Never Happened

Three Supermarkets Spell One Brand Three Ways, So My Price Comparison Never Happened

Comments 2
7 min read
Agoda reviews API: Booking.com reviews via Agoda and event costs

Agoda reviews API: Booking.com reviews via Agoda and event costs

1
Comments
4 min read
The Same Song in Two TikToks Came Back as 'Made My Night' and 'Made My Life'

The Same Song in Two TikToks Came Back as 'Made My Night' and 'Made My Life'

Comments
6 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.