DEV Community

Sophia
Sophia

Posted on

Scraping Product Data with Python for Price Comparison Tools

As a developer, I'm always looking for ways to automate repetitive tasks. Recently, I needed to scrape product details from a UK home decor site for a price comparison tool. Here's a quick Python script using BeautifulSoup and requests to extract product names and prices from a door accessories collection page.

python
import requests
from bs4 import BeautifulSoup

url = 'https://infinitydecor.co.uk/collections/door-accessories'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

products = []
for item in soup.select('.product-item'):
name = item.select_one('.product-title').text.strip()
price = item.select_one('.price').text.strip()
products.append({'name': name, 'price': price})

print(products[:5])

This gave me a clean list of front door hardware options in seconds. The trick is inspecting the HTML structure first—most e-commerce sites use predictable class names. Have you tried scraping product data for any projects?

Top comments (0)