DEV Community

Web Obsidians
Web Obsidians

Posted on

How to Get Product Prices and Stock from Shopify and WooCommerce Stores (Python and No-Code)

Competitor price tracking, product research and catalog migrations all start with the same question: how do I get every product of a store, with prices and stock, without copying it by hand? For the two most common e-commerce platforms you usually do not need to parse HTML at all.

Option 1: WooCommerce's public Store API in Python

Most WooCommerce stores answer at /wp-json/wc/store/v1/products (the Store API that WooCommerce's block-based cart and checkout use). It needs no key, returns up to 100 products per page, and includes prices, stock and categories.

import json
import urllib.request


def get(url):
    req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0 (price-check-script)"})
    with urllib.request.urlopen(req) as r:
        return json.load(r)


def woo_products(store, pages=2):
    out = []
    for page in range(1, pages + 1):
        items = get(f"https://{store}/wp-json/wc/store/v1/products?per_page=100&page={page}")
        if not items:
            break
        for p in items:
            pr = p["prices"]
            unit = 10 ** pr["currency_minor_unit"]  # prices come in cents as strings
            out.append({
                "name": p["name"],
                "price": int(pr["price"]) / unit,
                "regular_price": int(pr["regular_price"]) / unit,
                "currency": pr["currency_code"],
                "in_stock": p["is_in_stock"],
                "url": p["permalink"],
            })
    return out


for p in woo_products("woocommerce.com", pages=1)[:5]:
    print(p["price"], p["currency"], "|", p["name"], "|", "in stock" if p["in_stock"] else "out of stock")
Enter fullscreen mode Exit fullscreen mode

Two details that trip people up: prices are strings in the currency's minor unit (cents), and product names come HTML-escaped (– and friends), so unescape them with html.unescape before you compare titles.

Shopify: products.json, and the 429 wall

Every Shopify store serves its catalog at /products.json?limit=250&page=N: title, handle, vendor, product type, tags, images and a variants list with price, compare_at_price, sku and available. For a single collection, use /collections/<handle>/products.json.

The catch is rate limiting. While writing this post, four different Shopify stores answered 429 Too Many Requests to the very first request from my own connection. Once you scrape more than a few stores, you need retries with back-off, pauses between pages, and ideally a different network than your laptop.

Other things you will end up handling:

  • Which platform is it? A list of competitor domains does not tell you whether a store runs Shopify or WooCommerce.
  • Variants: a T-shirt is one product with many sizes and colors, each with its own price, SKU and stock.
  • Sale prices: compare_at_price on Shopify and regular_price vs price on WooCommerce.
  • Change tracking: to know what changed since yesterday, you have to keep yesterday's prices.

Option 2: a ready-made Actor that also tracks changes

I packaged all of that as an Apify Actor: Shopify & WooCommerce Products Scraper. You paste store domains or URLs, it detects the platform, and it returns one clean row per product. A real item from a run (shortened):

{
  "store": "www.beardbrand.com",
  "platform": "shopify",
  "title": "Bold Fortune Classic Beard Bundle",
  "brand": "Beardbrand",
  "productType": "Bundles",
  "url": "https://www.beardbrand.com/products/bold-fortune-classic-beard-bundle",
  "currency": "USD",
  "price": 119.85,
  "compareAtPrice": 141,
  "onSale": true,
  "available": false,
  "variantCount": 1,
  "change": null
}
Enter fullscreen mode Exit fullscreen mode

What it adds:

  • Filters for keywords, excluded words, minimum and maximum price, in stock only and on sale only
  • Variants with their own price, SKU and stock, and the product description as plain text, when you turn them on
  • Return only changes: schedule it daily and each run returns only new products and products whose price or stock changed, with previousPrice and previousAvailable
  • Export to JSON, CSV or Excel, or send results to Google Sheets, Slack or a webhook

From code:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("webobsidians/shopify-woocommerce-products-scraper").call(
    run_input={"stores": ["www.beardbrand.com", "woocommerce.com"], "maxProductsPerStore": 100, "onlyOnSale": True}
)
for p in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(p["store"], "|", p["title"], "|", p["price"], p["currency"])
Enter fullscreen mode Exit fullscreen mode

It costs $1.00 per 1,000 products plus $0.00005 per run start, with platform usage included, and Apify's free plan includes monthly credit to try it.

Which one should you use?

  • One WooCommerce store, once: the Python script above.
  • Many stores, Shopify at scale, or daily price-change alerts: the Actor.

Either way, you are reading catalogs that the stores publish for every visitor, so keep request rates polite.

Top comments (0)