DEV Community

Cover image for Building a Price & Competitor Monitor That Diffs Shopify and AliExpress Prices
Oaida Adrian
Oaida Adrian

Posted on

Building a Price & Competitor Monitor That Diffs Shopify and AliExpress Prices

#webscraping #python #automation #ecommerce

Most price watching is a spreadsheet: paste a URL, check it today, forget it tomorrow. The moment you want a history — what did this product cost a month ago, did it dip last Tuesday — snapshots are useless. I built an actor that keeps a real per-URL price timeline, diffs every run against the last one, and pings a webhook the instant something changes.

The design

For each URL the actor:

  1. Fetches the product — Shopify stores expose /products/<handle>.json plus /cart.js for the store currency; AliExpress item pages are client-rendered, so the price comes from the search page matched back by product id
  2. Reads the previous snapshot from a durable key-value store
  3. Computes the diff — NEW, PRICE_CHANGED, AVAILABILITY_CHANGED, or UNCHANGED, with absolute and percentage change
  4. Appends the new point to history, pruning anything older than historyDays
  5. Emits one row per URL with snapshot + diff + recent history, and POSTs a compact JSON alert to your webhook if the row changed

The first run of a URL is the baseline (NEW, no alert). Alerts fire only on real changes after that.

The persistence trap

This is the part that bit: Apify's default key-value store is per-run for API-started runs — it's purged at run start and deleted after the run. A naive implementation would lose its history between runs and re-emit everything as NEW every time. The fix is a named store: Actor.open_key_value_store(name='price-competitor-monitor-history') (the name= argument is keyword-only in SDK 3.4). That store persisted across three cloud runs in testing, which is what turns scheduled runs into a real timeline.

The smoke test

Three URLs — two Shopify stores and an AliExpress dock:

  • Run 1: allbirds NEW 91.00 USD, colourpop NEW 29.00 USD, non-zero ✓ (the AliExpress item page was x5sec-challenged from the datacenter egress — honest error row, no false price)
  • Run 2: both Shopify products UNCHANGED — durable history proven in the cloud
  • Run 3 (with a modified snapshot): colourpop PRICE_CHANGED, previous 36.25 → 29.00, −20%, direction down, alert true ✓

A fetch that fails cleanly (blocked page, deleted product) emits a status: "error" row and leaves the previous snapshot untouched — a transient block never masquerades as a price crash.

The honest bits

  • AliExpress item pages are fully client-rendered: the price is recovered from the search page by product id, so an item absent from search is reported without a price until it reappears.
  • Availability on AliExpress isn't reported yet (available: null).
  • History is capped at 500 points per URL as a safety limit.

Try it

👉 Price & Competitor Monitor on Apify Store

More from me

While you're here, these might be worth a read:

Top comments (0)