DEV Community

ivanpetrus
ivanpetrus

Posted on

I stopped checking competitor Shopify stores by hand. Here's what I use instead

If you sell on Shopify, you probably have a few competitors you keep an eye on. Maybe every Monday you open their stores in five tabs and scroll. New products? Price drop? Something sold out? Then you write it down somewhere. It's slow, it's boring, and you still miss things.

I built a tool to do this because it felt like a job a computer should do. This post is how it works, what I learned building it, and where it breaks. Fair warning: the tool at the end is mine (Shopify Store Monitor on Apify). You can also just use the trick below with plain curl.

The trick: products.json

Most Shopify stores publish their whole catalog as JSON. No login, no API key. Try it:

curl -s "https://allbirds.com/products.json?limit=250&page=1" | head -c 600
Enter fullscreen mode Exit fullscreen mode

You get titles, handles, vendors, variants, prices, compare-at prices and an available flag for each variant. That's basically everything you'd scroll for by hand.

The catch is that a snapshot is boring. Nobody cares that a shoe costs $100. You care that it cost $125 yesterday.

What "changed" actually means

When I sat down to write the diff, the list got longer than I expected:

  • a new product showed up
  • a product disappeared
  • the price went up or down
  • a sale started or ended (that's the compare-at price)
  • a size sold out, or came back
  • variants were added or removed

Sales were the sneaky one. A store often keeps the price and just sets a compare-at price, so the product page suddenly shows "$125 $99". If you only diff price, you miss half the action.

The monitor

I wrapped this into an Apify actor. You give it a list of stores. The first run saves a baseline. Every next run gives you only the changes. Put it on a daily schedule and you have a competitor feed.

Input looks like this:

{
  "stores": ["allbirds.com", "gymshark.com"],
  "changeTypes": ["priceChange", "compareAtChange", "stockChange", "newProduct"],
  "minPriceChangePct": 5
}
Enter fullscreen mode Exit fullscreen mode

minPriceChangePct is there because some stores move prices by a few cents all the time. Nobody needs a ping for $0.30.

A real change row from a test run (shortened):

{"type": "change", "changeType": "priceChange", "store": "Allbirds",
 "title": "Men's Canvas Runner NZ - Deep Navy Stripes (Blizzard Sole)",
 "direction": "down", "oldPrice": 125.0, "newPrice": 100.0, "changePct": -20.0,
 "url": "https://allbirds.com/products/mens-canvas-runner-nz"}
Enter fullscreen mode Exit fullscreen mode

That's the whole point. One line, one fact, one link.

Each store also gets a free summary row: catalog size, how many items are on sale or sold out, price range, newest launches. Handy for a quick "how big is this guy, really" check.

Pulling it into Python

If you want a weekly log, here's the short version with the official client:

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("ivora/shopify-store-monitor").call(run_input={
    "stores": ["allbirds.com", "gymshark.com"],
})

rows = client.dataset(run["defaultDatasetId"]).list_items().items
df = pd.DataFrame([r for r in rows if r.get("type") == "change"])
print(df.groupby(["store", "changeType"]).size())
Enter fullscreen mode Exit fullscreen mode

Add a webhook in Apify and the same rows land in Slack, email, Zapier, Make or n8n. Getting the list in Slack with your morning coffee beats scrolling five stores.

What it costs

Pay per use, no subscription:

  • $0.003 per run start
  • $0.004 per store checked
  • $0.0005 per change row

Five stores checked every day comes out to well under $1 a month. Stores the actor can't read get a free error row and aren't charged.

Where it breaks (honestly)

  • Not every store works. In my cloud test, 12 of 13 big brand stores were fine. Some stores turn off the feed or run a custom frontend. bombas.com said no even through a residential proxy.
  • Stock is yes or no. The public feed shows available, not "3 left".
  • Prices are the base currency. Market-specific prices and checkout discounts aren't visible.

That's it. No magic, just a JSON file most people don't know exists, and a diff.

If you sell on Shopify, try the curl line on your biggest competitor right now. It's a little weird how much is out there.

More details and pricing: ivora.life/products/shopify-store-monitor. The actor: apify.com/ivora/shopify-store-monitor.

Got a store where it doesn't work? Tell me in the comments. Those reports help me fix things.

Top comments (0)