DEV Community

Quill Datafix
Quill Datafix

Posted on

Watching product pages for price drops with ~80 lines of Python (CSV history + Telegram alerts)

Watching product pages for price drops is one of those tasks where every SaaS wants $10/month for something that is, fundamentally, one cron entry and ~80 lines of Python. Here is the whole pattern — no framework, no database, no paid plan. The state lives in a CSV file. The alert goes to Telegram. The scheduler is cron.

The one trick that makes it reliable: JSON-LD

Most tutorials grab prices with regex over raw HTML and then break on every site redesign. The sturdier approach: the overwhelming majority of shop platforms (Shopify, WooCommerce, Magento, BigCommerce, pretty much every serious storefront) embed schema.org structured data in a <script type="application/ld+json"> block. That block has a machine-readable offers.price field. Parse that first; fall back to regex only when it's missing.

import re, json, csv, time, sys, requests
from pathlib import Path

UA = {"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36"}
HIST = Path("price_history.csv")

def extract_price(html):
    # 1) schema.org JSON-LD — most shops embed it
    for m in re.findall(r'<script[^>]+type="application/ld\+json"[^>]*>(.*?)</script>', html, re.S):
        try:
            data = json.loads(m)
        except json.JSONDecodeError:
            continue
        for n in (data if isinstance(data, list) else [data]):
            if not isinstance(n, dict):
                continue
            offers = n.get("offers")
            if isinstance(offers, list) and offers:
                offers = offers[0]
            if isinstance(offers, dict) and "price" in offers:
                try:
                    return float(offers["price"]), "jsonld"
                except (TypeError, ValueError):
                    pass
    # 2) markup fallback
    m = re.search(r'[$€£]\s?(\d+(?:[.,]\d{2})?)', html)
    if m:
        return float(m.group(1).replace(",", ".")), "regex"
    return None, None

def load_state():
    """url -> last seen price, from the history file itself (no separate db)."""
    state = {}
    if HIST.exists():
        for row in csv.reader(HIST.open()):
            if len(row) == 4:
                state[row[1]] = row[2]
    return state

def telegram_alert(token, chat_id, text):
    requests.post(
        f"https://api.telegram.org/bot{token}/sendMessage",
        data={"chat_id": chat_id, "text": text},
        timeout=15,
    )

def main():
    urls = [u.strip() for u in Path(sys.argv[1]).read_text().splitlines() if u.strip()]
    token, chat_id = sys.argv[2], sys.argv[3]
    known = load_state()
    with HIST.open("a", newline="") as f:
        w = csv.writer(f)
        for url in urls:
            try:
                html = requests.get(url, headers=UA, timeout=20).text
            except requests.RequestException as e:
                print("fetch failed:", url, e)
                continue
            price, src = extract_price(html)
            if price is None:
                print("no price found:", url)
                continue
            w.writerow([time.strftime("%Y-%m-%d %H:%M:%S"), url, price, src])
            prev = known.get(url)
            if prev and float(prev) != price:
                telegram_alert(token, chat_id, f"Price change: {url}\n{prev} -> {price}")
                print("ALERT", url, prev, "->", price)
            time.sleep(3)  # be polite: don't hammer

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Run it like this:

echo "https://example.com/product/widget" > urls.txt
# token from @BotFather, chat id from @userinfobot
*/30 * * * * python3 watcher.py urls.txt 123456:ABC-DEF your_chat_id
Enter fullscreen mode Exit fullscreen mode

That's it: one CSV that doubles as your price history (open it in any spreadsheet and chart it), one Telegram message whenever a price actually changes, one cron line.

Gotchas I hit while running this

  1. Missing User-Agent = instant 403. The default python-requests UA is blocked by most CDNs. The UA string above is load-bearing, don't trim it.
  2. Comma vs dot decimals. European shops write 1.299,00, US shops write 1,299.00. Normalizing before float() is not optional — the regex fallback above only handles the simple case; JSON-LD sidesteps it entirely, which is one more reason to prefer it.
  3. JS-rendered storefronts. If the HTML you fetch has no JSON-LD and no price, the page probably renders client-side. Swap requests for a headless browser (Playwright's page.content() after networkidle) — same extraction code works unchanged.
  4. Poll interval. Below ~15 minutes you start looking like a bot to shop CDNs, and (depending on the site's terms) you might be. 30 minutes catches every real price change I've ever cared about; flash sales lasting under 30 minutes are rare and not the point of monitoring.
  5. HTML "changes" without price changes (banners, A/B tests, CSRF tokens) — this is why the code compares extracted prices, never raw HTML hashes. Comparing hashes of the full page gives false alerts every few hours.

Why I'm writing this down

I run a tiny $1 software store (datafixtools.sell.app) where everything costs one dollar on purpose — I'm collecting the first round of honest user reviews before charging real prices. The packaged version of the above (multi-product config file, CSV history that charts cleanly, Telegram + email alerts, a Playwright variant for JS-heavy pages) is my Price Monitor Kit. The core pattern is fully in this post and free; the kit is for people who'd rather not assemble the edges themselves. If it disappoints, roast me in the product Q&A — that is literally what the price is for.

Top comments (0)