How to Scrape Any Shopify Store in Seconds (No API Key, No Captcha)

Every Shopify storefront exposes its full product catalog through a public endpoint — and most paid "product research tools" just repackage it for $50+/month. Here's how to do it yourself, for free.

The secret: /products.json

Every Shopify store has a public endpoint:

https:///products.json

No auth. No API key. No captcha. The store's own frontend uses this endpoint to render — so it's open by default.

Try it right now — open this in your browser:

https://allbirds.com/products.json?limit=5

You'll see JSON with products, prices, variants, compare-at values, tags, and images.

How it works

250 products per page — paginate with ?page=2, ?page=3...
Compare-at price = the sale signal. When compare_at > price, that product is actively on sale — the clearest "this seller is pushing a winner" signal in product research.
Works on huge stores — I tested gymshark.com: 9,466 products / 57,532 SKUs, paginated cleanly.

A minimal scraper (Python, ~40 lines)

import requests

def fetch_products(store, max_pages=50):
    products = []
    for page in range(1, max_pages + 1):
        r = requests.get(f"https://{store}/products.json", params={"limit": 250, "page": page}, timeout=30)
        if r.status_code != 200:
            break
        items = r.json().get("products", [])
        products.extend(items)
        if len(items) < 250:
            break
    return products

products = fetch_products("allbirds.com")
print(f"{len(products)} products fetched")

What you can do with it

Dropshipping research: scan competitor stores → find price gaps and what they're pushing
Competitor price monitoring: snapshot daily, diff the data → catch price changes within 24h
Product research: full catalog audits with price distribution

Production-ready version

I packaged this into an open-source tool: Shopify Scout

Features:

Automatic pagination (handles 10,000+ product stores)
Price intelligence: range, median, average
Compare-at sale signal detection
Keyword and price filters
Clean Markdown report output
MIT licensed — free to use and modify

pip install requests
python shopify_scout.py allbirds.com --sort price-desc

Caveats

Some stores disable the endpoint — the script tells you clearly.
Use only for stores you have permission to research, and respect their terms.

If this saved you time, star the repo: https://github.com/zhaoheng588-tech/shopify-scout 🚀

DEV Community