DEV Community

Matvey
Matvey

Posted on

My Shopee Price Alert Delivered One Row. previousPrice Was Null

I run a visa agency in Bali, and the Apify Actors I publish are the other half of my week. Price tracking is the request I get most often for Shopee, and it sounds like the easy one: read the price, remember it, compare it next time. The remembering is easy. The part nobody warns you about is deciding what you are remembering, and I only learned it properly by running my own tracker five times in a row and reading the output as a stranger would.

The input was frozen for every run, the default demo:

{
  "searchTerms": ["iphone 15"],
  "country": "ID",
  "maxItems": 40,
  "priceHistoryStore": "devto-watch-73"
}
Enter fullscreen mode Exit fullscreen mode

priceHistoryStore is the price memory — a named key-value store in my own Apify account. Each run writes the price of every product it saw; the next run with the same name fills in previousPrice, priceChange, priceChangePercent, priceTrend, and the lowest and highest price the memory has ever seen. No database, no schema migration. Four runs, at 21:08, 21:09, 21:11 and 21:12 on 29 October — all four inside four minutes.

160 rows, 63 products

Every run returned exactly 40 product rows, which is what I asked for. Pool them and you do not get 40 products tracked four times. You get 63 distinct itemId values:

appeared in 4 of 4 runs   22 products
appeared in 3 of 4        7
appeared in 2 of 4        17
appeared in 1 of 4        17
Enter fullscreen mode Exit fullscreen mode

Just over half the catalogue I "tracked" showed up once or twice and then left. The pairwise overlaps say the same thing from the other side:

run1 vs run2   32/40 shared
run1 vs run3   27/40
run1 vs run4   27/40
run2 vs run3   24/40
run2 vs run4   26/40
run3 vs run4   34/40
Enter fullscreen mode Exit fullscreen mode

Nothing is broken here. iphone 15 on Shopee Indonesia matches thousands of listings, and the first 40 are whatever the ranker preferred in that second. Ask twice a minute apart and a quarter of the page is different.

Not one price moved

So I checked the only comparison that is actually valid: products present in both of two consecutive runs.

run1 -> run2   32 shared products   0 price changes
run2 -> run3   24 shared            0
run3 -> run4   34 shared            0
Enter fullscreen mode Exit fullscreen mode

Ninety shared product-pairs, zero movement. Correct, and expected — Shopee sellers do not reprice twice a minute. Which makes the trend column interesting, because it was not quiet at all:

run1   new 40
run2   same 32   new 8
run3   same 29   new 11
run4   same 36   new 4
Enter fullscreen mode Exit fullscreen mode

Twenty-three new rows across runs 2 to 4. I checked every one against the set of IDs the memory had already stored, and all 23 were genuinely first-time sightings. The memory is not misreporting. new means "this country:itemId is not in my store", and on a keyword search that overwhelmingly means the ranker moved, not a product was listed. Those are two different events and the column cannot tell them apart.

Then I turned on the alert

onlyPriceChanges is the feature people actually want: write and charge only products whose price moved since the last run, plus products seen for the first time. Skip the unchanged ones free of charge. Same memory, one flag added, run five at 21:13.

One row. Here it is, trimmed:

{
  "title": "IPHONEE 15 128GB/256GB FULLSET SECOND LIKE NEW NO SIM LOCK ...",
  "itemId": "29591951945",
  "price": 6984000,
  "originalPrice": 11000000,
  "discountPercent": 37,
  "previousPrice": null,
  "priceChange": null,
  "priceChangePercent": null,
  "priceTrend": "new",
  "lowestPrice": 6984000,
  "highestPrice": 6984000,
  "timesChecked": 1
}
Enter fullscreen mode Exit fullscreen mode

previousPrice: null. priceChange: null. timesChecked: 1. This is not a price change. It is a listing that drifted into the top 40 for the first time, and it is the entire output of a run whose job was to tell me about price changes. Thirty-nine unchanged products were skipped, correctly and for free, and the one thing delivered was a new arrival wearing the same badge.

The billing worked beautifully: $0.0004 for that run against $0.016 for each full 40-row run — forty times cheaper, and the free-skip accounting was exact every time. The economics are fine; the semantics are the trap. If you wire onlyPriceChanges straight into Slack and call the channel #price-drops, most of what lands there will be churn, and priceChange: null is the only thing distinguishing it.

Filter on the fields, not on the fact that a row arrived:

def is_real_move(row):
    return (row.get("priceTrend") in ("up", "down")
            and row.get("previousPrice") is not None)
Enter fullscreen mode Exit fullscreen mode

timesChecked is the honest column

The memory keeps a per-product counter, and it is the field that tells you whether a price history is worth anything. Here it is across the four search runs:

run1   timesChecked 1 ×40
run2   1 ×8    2 ×32
run3   1 ×11   2 ×7    3 ×22
run4   1 ×4    2 ×7    3 ×7    4 ×22
Enter fullscreen mode Exit fullscreen mode

By run four, only 22 of the 40 rows had been priced all four times. The other 18 carried a history one, two or three points deep, mixed into the same column as the good rows. Any "lowest price in the last 30 days" built on this needs a floor — timesChecked >= n — or you will compare a product watched thirty times against one watched twice and announce a record low.

The fix is boring: stop tracking a search

I took the 6 products that had appeared in all four runs, fed their URLs instead of the keyword, pointed at a fresh memory, and ran it three times:

{ "startUrls": ["https://shopee.co.id/Apple-iPhone-15-i.255563049.23157471408", "..."],
  "priceHistoryStore": "devto-fixed-73" }
Enter fullscreen mode Exit fullscreen mode
run1   6 rows   trend: new ×6     timesChecked 1 ×6    9s   $0.0024
run2   6 rows   trend: same ×6    timesChecked 2 ×6   11s   $0.0024
run3   6 rows   trend: same ×6    timesChecked 3 ×6    9s   $0.0024
Enter fullscreen mode Exit fullscreen mode

Identical ID sets all three times, the counter marching 1 → 2 → 3, zero spurious new. That is a price history. It is also 9–11 seconds against 47–78 seconds for the 40-row searches, because resolving six known URLs beats ranking a keyword. Use keywords to discover a watch list once, then pin the URLs and track those. A search is a sampler; only a URL list is a watch list.

Two field quirks from the same five runs

The discount is the seller's opinion. Six different listings in run one carried the identical price of 6,950,000 IDR:

price 6,950,000   originalPrice  7,100,000       2%
price 6,950,000   originalPrice 12,000,000      42%
price 6,950,000   originalPrice 21,000,000      67%
price 6,950,000   originalPrice  7,100,000       2%
price 6,950,000   originalPrice       null     null
price 6,950,000   originalPrice       null     null
Enter fullscreen mode Exit fullscreen mode

Same money, discounts of 2%, 42% and 67%. originalPrice is a number each seller types in, so discountPercent ranks sellers' marketing, not value. Sort by price. originalPrice was null on 8 of 40 cards, and discountPercent with it.

Detail rows drop originalPrice but keep discountPercent. Product URLs return type: "product_detail", and on all 6 of mine originalPrice was null while discountPercent survived intact — 18, 10, 10, 15, 8, 8, matching the search cards for the same six products exactly. If you need both, reconstruct: originalPrice ≈ price / (1 - discountPercent/100), or keep the card row. Those rows said so themselves, in a note field on 6 of 6: Shopee publishes no full product record for these items, so price and rating came from the product's own search card. description, brand, breadcrumb, categoryPath and imageCount were null on all six; ratingCount was null on 32 of 40 search cards, and brand, priceMin and priceMax on all 40.

Calling it

Under a minute at this size, so the synchronous endpoint is fine:

curl -X POST "https://api.apify.com/v2/acts/lergassy~shopee-price-tracker/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"startUrls":["https://shopee.co.id/Apple-iPhone-15-i.255563049.23157471408"],
       "priceHistoryStore":"my-watch","onlyPriceChanges":true}'
Enter fullscreen mode Exit fullscreen mode

The Actor is Shopee Price Tracker, $0.40 per 1,000 products with no per-run fee, 11 Shopee marketplaces. Every run appends a free notice row with its own cost — filter type != "notice" and you have your billing audit without opening the console.

When it is the wrong tool

It will not give you yesterday. The memory starts empty on your first run, so a 30-day low means thirty days of waiting; there is no backfill, by me or by Shopee. It will not catch flash sales — a daily schedule sees the price at the moment of the run, and Shopee moves prices inside hours. And it is still the wrong shape for "has anything in this category changed", because as shown above a category is a search, and a search churns faster than prices do. For that, scrape the category wide, diff it yourself, and accept that you are measuring the ranker as much as the sellers.

The general lesson I keep relearning: a diff is only meaningful against a stable key set. Give a tracker a moving list and it will dutifully report change — just not the change you asked about.

Top comments (0)