How to Build a Price Monitoring System with Python in 2026

You want to track competitor prices automatically. Maybe you run an e-commerce store and need to stay competitive, or you're a deal hunter who wants alerts when prices drop. Either way, a price monitoring system is one of the most practical things you can build with Python.

I've built these systems for clients and for my own projects. Here's what actually works in 2026, what breaks, and how to keep it running reliably.

The Architecture

A price monitoring system has four parts:

1. Scraper — fetches product pages and extracts prices
2. Storage — saves price history over time
3. Scheduler — runs the scraper at regular intervals
4. Alerting — notifies you when prices change

Let's build each one.

Part 1: The Scraper

Start simple. For most product pages, requests + BeautifulSoup gets the job done:

import requests
from bs4 import BeautifulSoup
from datetime import datetime

def scrape_price(url, css_selector):
    """Scrape a single product price from a URL."""
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                      "AppleWebKit/537.36 (KHTML, like Gecko) "
                      "Chrome/131.0.0.0 Safari/537.36"
    }

    response = requests.get(url, headers=headers, timeout=15)
    response.raise_for_status()

    soup = BeautifulSoup(response.text, "html.parser")
    price_element = soup.select_one(css_selector)

    if not price_element:
        raise ValueError(f"Price not found at {url}")

    # Clean the price text: remove currency symbols, commas
    price_text = price_element.get_text(strip=True)
    price_text = price_text.replace("$", "").replace(",", "").replace("£", "").replace("€", "")

    return {
        "url": url,
        "price": float(price_text),
        "timestamp": datetime.now().isoformat(),
    }

This works for static pages. But here's the reality check: many e-commerce sites in 2026 render prices with JavaScript. If your target site uses React, Next.js, or any SPA framework, requests will return an empty price container.

When You Need a Browser

For JS-rendered pages, use Playwright:

from playwright.sync_api import sync_playwright

def scrape_price_js(url, css_selector):
    """Scrape price from a JS-rendered page."""
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()
        page.goto(url, wait_until="networkidle")

        price_element = page.query_selector(css_selector)
        if not price_element:
            browser.close()
            raise ValueError(f"Price not found at {url}")

        price_text = price_element.inner_text().strip()
        price_text = price_text.replace("$", "").replace(",", "")

        browser.close()
        return float(price_text)

Playwright is heavier — slower and uses more RAM. For monitoring 10 products, it's fine. For 500+, you need a different approach.

When to Use a Scraping API

If you're monitoring prices across major e-commerce sites (Amazon, Walmart, Best Buy), you'll hit anti-bot systems fast. CAPTCHAs, IP bans, fingerprint detection — these sites invest millions in blocking scrapers.

This is where scraping APIs save you time:

• ScraperAPI — handles proxy rotation and CAPTCHA solving. Good for e-commerce sites. 5,000 free API calls to start.
• Scrape.do — similar approach with competitive pricing and JS rendering support.
• ScrapeOps — proxy aggregator that lets you compare providers. Useful if you want to optimize cost per request.

Here's how price monitoring looks with ScraperAPI:

import requests

SCRAPERAPI_KEY = "your_api_key"

def scrape_price_api(url, css_selector):
    """Scrape price using ScraperAPI to handle anti-bot measures."""
    api_url = f"http://api.scraperapi.com?api_key={SCRAPERAPI_KEY}&url={url}&render=true"

    response = requests.get(api_url, timeout=60)
    response.raise_for_status()

    soup = BeautifulSoup(response.text, "html.parser")
    price_element = soup.select_one(css_selector)

    if not price_element:
        raise ValueError(f"Price not found at {url}")

    price_text = price_element.get_text(strip=True)
    price_text = price_text.replace("$", "").replace(",", "")
    return float(price_text)

My recommendation: start free with requests. When you start getting blocked, switch to a scraping API. Don't over-engineer it upfront.

Part 2: Storage

SQLite is perfect for this. No server to manage, and it handles thousands of price records easily:

import sqlite3

def init_db(db_path="prices.db"):
    conn = sqlite3.connect(db_path)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS prices (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            product_name TEXT NOT NULL,
            url TEXT NOT NULL,
            price REAL NOT NULL,
            timestamp TEXT NOT NULL
        )
    """)
    conn.commit()
    return conn

def save_price(conn, product_name, url, price):
    conn.execute(
        "INSERT INTO prices (product_name, url, price, timestamp) VALUES (?, ?, ?, ?)",
        (product_name, url, price, datetime.now().isoformat())
    )
    conn.commit()

def get_price_history(conn, product_name, days=30):
    """Get price history for a product over the last N days."""
    cursor = conn.execute(
        """SELECT price, timestamp FROM prices
           WHERE product_name = ?
           AND timestamp > datetime('now', ?)
           ORDER BY timestamp""",
        (product_name, f"-{days} days")
    )
    return cursor.fetchall()

Part 3: Scheduling

Use schedule for simplicity or cron for reliability:

import schedule
import time

# Define your products to monitor
PRODUCTS = [
    {
        "name": "Sony WH-1000XM5",
        "url": "https://example.com/product/sony-xm5",
        "selector": "span.price",
    },
    {
        "name": "MacBook Air M3",
        "url": "https://example.com/product/macbook-air-m3",
        "selector": "div.product-price",
    },
]

def monitor_all():
    conn = init_db()
    for product in PRODUCTS:
        try:
            price = scrape_price(product["url"], product["selector"])
            save_price(conn, product["name"], product["url"], price["price"])
            print(f"[OK] {product['name']}: ${price['price']}")
            check_price_alert(conn, product["name"], price["price"])
        except Exception as e:
            print(f"[ERROR] {product['name']}: {e}")
    conn.close()

# Run every 6 hours
schedule.every(6).hours.do(monitor_all)

if __name__ == "__main__":
    monitor_all()  # Run once immediately
    while True:
        schedule.run_pending()
        time.sleep(60)

For production, run this as a systemd service or a cron job. The schedule library is fine for development, but cron is more robust for long-running monitors.

Part 4: Alerting

Send yourself an email or Slack message when prices drop:

import smtplib
from email.mime.text import MIMEText

def check_price_alert(conn, product_name, current_price, threshold_pct=5):
    """Alert if price dropped more than threshold_pct from the last recorded price."""
    cursor = conn.execute(
        """SELECT price FROM prices
           WHERE product_name = ?
           ORDER BY timestamp DESC LIMIT 1 OFFSET 1""",
        (product_name,)
    )
    row = cursor.fetchone()
    if not row:
        return  # No previous price to compare

    previous_price = row[0]
    change_pct = ((current_price - previous_price) / previous_price) * 100

    if change_pct <= -threshold_pct:
        send_alert(
            subject=f"Price Drop: {product_name}",
            body=f"{product_name} dropped {abs(change_pct):.1f}%!\n"
                 f"Was: ${previous_price:.2f}\n"
                 f"Now: ${current_price:.2f}"
        )

def send_alert(subject, body):
    """Send email alert. Replace with Slack webhook if preferred."""
    msg = MIMEText(body)
    msg["Subject"] = subject
    msg["From"] = "[e-mail]"
    msg["To"] = "[e-mail]"

    with smtplib.SMTP("smtp.yourdomain.com", 587) as server:
        server.starttls()
        server.login("[e-mail]", "your_password")
        server.send_message(msg)

Putting It All Together

Here's the complete flow:

1. Define your products with URLs and CSS selectors
2. The scheduler runs every 6 hours (adjust based on how often prices change)
3. Each run scrapes all products, saves to SQLite
4. If a price drops more than 5%, you get an alert

Scaling tips:

• Add random delays (2-5 seconds) between requests to avoid detection
• Rotate User-Agent strings
• For 50+ products across major retailers, use ScraperAPI or Scrape.do — the time you save debugging blocks is worth the cost
• Use ScrapeOps to monitor your scraper's success rates over time

Common Pitfalls

Selectors break. Sites redesign. Your CSS selector that worked yesterday returns nothing today. Build in error handling and alerts for failed scrapes, not just price drops.

Prices vary by location. Some sites show different prices based on your IP's geolocation. If you're using proxies, be consistent with the region.

Dynamic pricing is real. Some sites change prices based on visit frequency. Don't scrape the same product from the same IP every 5 minutes.

Want the Full System?

I wrote a complete guide covering advanced patterns — handling login-protected prices, building price comparison dashboards, and deploying monitors on cheap VPS instances.

Get the complete Web Scraping Playbook — $9 on Gumroad

It includes ready-to-deploy code for price monitoring, SERP tracking, and more.

---

Questions or need help adapting this to your use case? Reach me at [e-mail].