DEV Community

coreclaw
coreclaw

Posted on

Etsy Price Tracker: How to Monitor Listing Price Changes with Python

Etsy Price Tracker: How to Monitor Listing Price Changes with Python

The reliable way to track price changes on Etsy is to snapshot public listing data on a schedule and diff each snapshot against the previous one. You can build that workflow in a couple hundred lines of Python around the open-source Etsy scraper maintained by the data-scrape profile, which handles the collection side while your code handles storage, comparison, and alerting. This article shows the full loop: what to store, how to diff, and how to turn raw changes into a report you can actually act on.

TL;DR

  • Use the data-scrape/etsy-scraper repository to collect public listing records (title, price, availability, shop, URL) for the listings and shops you care about.
  • Append each run's output to a dated JSONL snapshot file — never overwrite, because the history is the product.
  • Diff consecutive snapshots in Python to detect price drops, price increases, and sold-out or relisted items.
  • Apply the same snapshot-diff pattern to other marketplaces; the AliExpress scraper repository follows the same repository-led approach for cross-marketplace monitoring.
  • Respect Etsy's terms, robots directives, and applicable privacy law. This workflow is for public marketplace research, not for bulk harvesting personal data.

Why manual Etsy price tracking breaks down

If you watch five handmade listings, a spreadsheet works. If you watch five hundred — competitor shops, a product niche, or your own catalog across seasons — manual tracking fails for predictable reasons:

  • Prices move quietly. Etsy sellers reprice for sales, holidays, and material costs. A weekly manual check misses most intraweek changes.
  • Listings disappear and return. A sold-out listing that relists at a higher price looks like two unrelated records unless you track listing IDs over time.
  • Copy-paste loses provenance. Without a timestamp and source URL on every record, you cannot reconstruct when a change happened or trust the history.
  • One-off exports go stale immediately. A CSV saved today is a fossil next week.

The fix is not a bigger spreadsheet. It is a small, boring pipeline: collect → snapshot → diff → report. The analysis is easy; the collection and consistency are the hard parts, which is exactly what a maintained scraper repository removes.

What the repository-led workflow looks like

The data-scrape/etsy-scraper repository provides runnable code for collecting public Etsy listing and shop data. Treat it as the collection layer of your pipeline: you run it with your own documented inputs, it produces structured records, and everything downstream — storage, diffing, reporting — is plain Python you own.

The pipeline has four stages:

  1. Collect. Run the repository's documented workflow against your target listings or shop pages. Output: one structured record per listing.
  2. Snapshot. Normalize each record to a stable schema and append it to snapshots/YYYY-MM-DD.jsonl. One file per run, append-only.
  3. Diff. Compare the newest snapshot against the previous one on listing_id. Produce a change list: price up, price down, delisted, relisted, availability changed.
  4. Report. Write the changes to a CSV and, optionally, a markdown summary you can paste into Slack or email.

Because every record carries a source URL and a run date, you can always answer "what did this listing cost on which day" — the property that manual tracking never gives you.

Step 1 — Set up the environment

Clone the repository and create a working directory for snapshots:

git clone https://github.com/data-scrape/etsy-scraper.git
cd etsy-scraper
mkdir -p ../etsy-price-tracker/snapshots
Enter fullscreen mode Exit fullscreen mode

Configure any account-specific values through environment variables rather than hard-coded strings. The exact variable names depend on the repository's current README — read it before running, because documented inputs change over time:

export ETSY_TARGETS_FILE="../etsy-price-tracker/targets.txt"   # listing or shop URLs you follow
export SNAPSHOT_DIR="../etsy-price-tracker/snapshots"
Enter fullscreen mode Exit fullscreen mode

Keep a plain targets.txt with one public listing or shop URL per line. Starting from a file instead of hard-coded URLs means adding a competitor shop is a one-line change, and the file doubles as documentation of what you monitor.

Step 2 — Run a collection pass and snapshot the output

Run the repository's documented collection workflow once. Its exact invocation is in the README; the important discipline is what you do with the output: normalize it into a stable schema before storing it, so that diffing code never has to know about raw-format quirks.

A normalization wrapper looks like this:

import json
import pathlib
from datetime import datetime, timezone

def normalize_record(raw: dict) -> dict:
    """Map a raw collected record to the tracker's stable schema.

    Field names in `raw` depend on the repository's current output format.
    Adjust the keys after inspecting one real record — do not guess.
    """
    return {
        "listing_id": str(raw["listing_id"]),
        "title": raw.get("title", ""),
        "price": raw.get("price"),              # keep as-is; parsed below
        "currency": raw.get("currency", "USD"),
        "availability": raw.get("availability", "unknown"),
        "shop": raw.get("shop", ""),
        "url": raw.get("url", ""),
        "collected_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
    }

def append_snapshot(records: list[dict], snapshot_dir: str) -> pathlib.Path:
    snap_dir = pathlib.Path(snapshot_dir)
    snap_dir.mkdir(parents=True, exist_ok=True)
    today = datetime.now(timezone.utc).date().isoformat()
    path = snap_dir / f"{today}.jsonl"
    with path.open("a", encoding="utf-8") as f:
        for rec in records:
            f.write(json.dumps(rec, ensure_ascii=False) + "\n")
    return path
Enter fullscreen mode Exit fullscreen mode

Two design decisions matter here:

  • Append, never overwrite. If you run the collection twice in one day, both passes land in the same dated file. The diff step handles duplicates by keeping the last record per listing per day.
  • Normalize at ingestion. A price stored as "$24.00" on Monday and 24.0 on Tuesday will silently break your diff. Parse and canonicalize once, at the boundary.

Step 3 — Diff consecutive snapshots

The core of the tracker is a pure function: given yesterday's records and today's records, produce a change list. No network, no parsing — which makes it trivially testable.

import csv
import json
import pathlib
from decimal import Decimal, InvalidOperation

def load_latest_records(snapshot_dir: str, date: str) -> dict[str, dict]:
    """Load one day's JSONL file, keeping the LAST record per listing."""
    path = pathlib.Path(snapshot_dir) / f"{date}.jsonl"
    records: dict[str, dict] = {}
    if not path.exists():
        return records
    with path.open(encoding="utf-8") as f:
        for line in f:
            rec = json.loads(line)
            records[rec["listing_id"]] = rec
    return records

def parse_price(value) -> Decimal | None:
    if value is None:
        return None
    try:
        return Decimal(str(value).replace("$", "").replace(",", "").strip())
    except InvalidOperation:
        return None

def diff_snapshots(old: dict[str, dict], new: dict[str, dict]) -> list[dict]:
    changes = []
    for lid, new_rec in new.items():
        old_rec = old.get(lid)
        if old_rec is None:
            changes.append({"listing_id": lid, "change": "new_listing",
                            "url": new_rec["url"], "detail": new_rec["title"]})
            continue
        old_price, new_price = parse_price(old_rec["price"]), parse_price(new_rec["price"])
        if old_price is not None and new_price is not None and old_price != new_price:
            direction = "price_increase" if new_price > old_price else "price_decrease"
            changes.append({"listing_id": lid, "change": direction,
                            "url": new_rec["url"],
                            "detail": f"{old_price} -> {new_price} {new_rec['currency']}"})
        if old_rec.get("availability") != new_rec.get("availability"):
            changes.append({"listing_id": lid, "change": "availability_changed",
                            "url": new_rec["url"],
                            "detail": f"{old_rec.get('availability')} -> {new_rec.get('availability')}"})
    for lid, old_rec in old.items():
        if lid not in new:
            changes.append({"listing_id": lid, "change": "delisted",
                            "url": old_rec["url"], "detail": old_rec["title"]})
    return changes

def write_change_report(changes: list[dict], out_path: str) -> None:
    with open(out_path, "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=["listing_id", "change", "detail", "url"])
        writer.writeheader()
        writer.writerows(changes)
Enter fullscreen mode Exit fullscreen mode

Glue it together with a small daily runner:

import sys
from datetime import date, timedelta

def main(snapshot_dir: str) -> None:
    today = date.today().isoformat()
    yesterday = (date.today() - timedelta(days=1)).isoformat()
    new = load_latest_records(snapshot_dir, today)
    old = load_latest_records(snapshot_dir, yesterday)
    if not new:
        sys.exit(f"No snapshot found for {today} — run the collection step first.")
    changes = diff_snapshots(old, new)
    out = f"changes-{today}.csv"
    write_change_report(changes, out)
    print(f"{len(changes)} changes written to {out}")

if __name__ == "__main__":
    main(sys.argv[1] if len(sys.argv) > 1 else "snapshots")
Enter fullscreen mode Exit fullscreen mode

Representative output

The change report is deliberately boring and machine-friendly. Example rows (illustrative, not from a live run):

listing_id,change,detail,url
1284402917,price_decrease,32.00 -> 24.00 USD,https://www.etsy.com/listing/1284402917
1290773451,price_increase,18.50 -> 21.00 USD,https://www.etsy.com/listing/1290773451
1281023764,availability_changed,in_stock -> sold_out,https://www.etsy.com/listing/1281023764
1275539980,delisted,Handmade ceramic mug set,https://www.etsy.com/listing/1275539980
1300182734,new_listing,Personalized dog bandana,https://www.etsy.com/listing/1300182734
Enter fullscreen mode Exit fullscreen mode

From this one file you can answer the questions manual tracking never could: which competitor started a sale, which niche is seeing price creep, and which listings rotate in and out of stock.

Comparison: build vs. spreadsheet vs. managed service

Dimension This repository workflow Spreadsheet + manual checks Managed scraping platform
Best for Developers tracking a defined list of listings Fewer than ~10 listings, one-off checks Teams needing scale, uptime, and support
Setup model Clone repo, run documented workflow, own the code None Account, plan selection, endpoint config
Maintenance burden You adapt when page layouts change None, but coverage decays Provider maintains parsers
Cost Code is open source; your time Free, but hours of labor Verify current pricing on the provider's official page
Data provenance Full: URL + timestamp on every record Poor Usually good; depends on provider
Change detection Built in (this article's diff step) Manual Varies by provider

The honest trade-off: a repository-led workflow costs you maintenance attention in exchange for full control and provenance. If you only need a handful of price checks per month, a spreadsheet wins. If you need auditable history across hundreds of listings, the pipeline pays for itself.

Use cases this supports

  • Competitive repricing. Handmade and craft sellers watch adjacent shops; a price-decrease alert is often the first visible signal of a promotion.
  • Resale and arbitrage research. Track listing prices over weeks to distinguish stable prices from temporary discounts before making inventory decisions.
  • Niche market research. Aggregate price distributions per category over time to see whether a niche is moving up- or down-market. The same snapshot-diff pattern extends to other marketplaces via the AliExpress scraper for cross-marketplace product data.
  • Catalog QA for your own shop. Run the tracker against your own public listings to catch accidental price edits or unintended sold-out states.

Limits, compliance, and maintenance

Be realistic about all three:

  • Layout changes are inevitable. When a diff suddenly reports everything as "delisted," suspect a parsing break upstream, not a mass exodus of sellers. Keep the collection layer isolated so fixes stay contained.
  • Pace your collection. Run daily or a few times per day, not per minute. Low-frequency snapshots capture nearly all economically meaningful price changes and keep load on target servers minimal.
  • Public data only. Track public listing pages. Do not attempt to access private data, bypass access controls, or evade anti-bot protections — those approaches create legal exposure and produce unreliable data anyway.
  • Respect the platform. Review Etsy's terms and robots directives, comply with applicable privacy and consumer-protection law, and use the data for research and competitive analysis rather than republishing sellers' content wholesale.
  • Verify before production. The repository's documented inputs and output fields evolve. Inspect a real record before wiring assumptions into your schema, and pin the commit you deploy so upgrades are deliberate.

FAQ

Is there an official Etsy API for prices?
Etsy offers an official API for sellers and approved applications with its own scope and authentication requirements. The repository workflow in this article is for public-page research where the official API does not fit your use case; check Etsy's developer documentation for current API access rules before choosing.

What fields does each snapshot contain?
The normalized schema used here: listing_id, title, price, currency, availability, shop, url, and collected_at. Adjust after inspecting the repository's real output — do not assume field names.

How often should the workflow run?
Once daily is enough for most price-monitoring goals. Increase frequency only if you have evidence that intraday changes matter for your niche, and keep frequency proportionate to avoid unnecessary load.

What happens when Etsy changes its page layout?
The collection step (repository) may need updating; your diff and reporting code keeps working because it only sees normalized records. This separation is the main reason to keep raw collection isolated from analysis.

Can this send alerts to Slack or email?
Yes. The change report is a plain CSV and list of dicts — add a webhook call or SMTP send in the runner's main() after write_change_report(). Keep alert logic separate from diff logic so you can test each independently.

Can I track other marketplaces with the same pattern?
Yes. The snapshot-diff-report skeleton is marketplace-agnostic; only the collection layer changes. The AliExpress repository linked above uses the same repository-led approach for a different marketplace.

Do I need a paid service to run this?
No managed service is required — the code runs locally on a schedule you control (cron, Task Scheduler, or a CI cron job). Costs are your own compute time. Commercial alternatives exist and can be worth it at scale; verify current pricing on any provider's official page before committing.

Next step

Clone the data-scrape/etsy-scraper repository, run one documented collection pass against a small targets.txt, and get a two-day snapshot loop working before scaling up. Browse the rest of the data-scrape repository profile for sibling scrapers covering other marketplaces, platforms, and data sources you may want to fold into the same pipeline.

Top comments (0)