DEV Community

Emir Mrkaljević
Emir Mrkaljević

Posted on Originally published at emirmrkaljevic.github.io

Find which domains in a list run Shopify (Python, no paid API)

I often get a list of domains from a CRM export or a trade show list and need to know one thing first: which of these sites run on Shopify? Shopify app developers, theme agencies and ecommerce consultants all start from that question.

You do not need a paid lookup service for a few hundred domains. A Shopify storefront leaves very clear fingerprints in its homepage, and a short Python script can check them.

What gives a Shopify store away

When you load the homepage of a Shopify store, you usually see:

  • a powered-by: Shopify response header
  • _shopify_y / _shopify_s cookies
  • assets loaded from cdn.shopify.com
  • a global JavaScript object, window.Shopify and Shopify.theme
  • the store's permanent something.myshopify.com domain somewhere in the source

Any one of these is a strong signal. Seeing several at once is close to certain. The myshopify.com domain is also handy later, because it keeps working even when the custom domain changes.

The script

It reads a text file with one domain per line (full URLs are fine), checks the homepages in parallel and writes a CSV to stdout. The only dependency is requests.

import csv
import re
import sys
from concurrent.futures import ThreadPoolExecutor

import requests

HEADERS = {"User-Agent": "Mozilla/5.0 (compatible; shopify-check/1.0)"}

# Signals that only a Shopify storefront sends
HTML_SIGNALS = [
    r"cdn\.shopify\.com",
    r"Shopify\.theme",
    r"window\.Shopify",
    r"myshopify\.com",
]


def clean(domain):
    domain = domain.strip().lower()
    domain = re.sub(r"^https?://", "", domain)
    return domain.split("/")[0]


def check(domain):
    row = {"domain": domain, "shopify": False, "evidence": "", "myshopify": "", "error": ""}
    try:
        r = requests.get(f"https://{domain}", headers=HEADERS, timeout=15)
    except requests.RequestException as e:
        row["error"] = type(e).__name__
        return row

    evidence = []
    # 1. Response headers (most stores send some of these)
    for h in ("x-shopid", "x-shopify-stage", "powered-by"):
        value = r.headers.get(h, "")
        if value and (h != "powered-by" or "shopify" in value.lower()):
            evidence.append(f"header:{h}")
    if "_shopify_y" in r.headers.get("set-cookie", "") or "_shopify_s" in r.headers.get("set-cookie", ""):
        evidence.append("cookie:_shopify")

    # 2. HTML fingerprints
    for pattern in HTML_SIGNALS:
        if re.search(pattern, r.text):
            evidence.append(f"html:{pattern.replace(chr(92), '')}")

    # 3. The permanent myshopify.com domain is often in the page source
    m = re.search(r"([a-z0-9-]+\.myshopify\.com)", r.text)
    if m:
        row["myshopify"] = m.group(1)

    row["shopify"] = bool(evidence)
    row["evidence"] = ";".join(evidence)
    return row


def main(path):
    with open(path) as f:
        domains = sorted({clean(line) for line in f if line.strip()})

    with ThreadPoolExecutor(max_workers=8) as pool:
        rows = list(pool.map(check, domains))

    writer = csv.DictWriter(sys.stdout, fieldnames=rows[0].keys())
    writer.writeheader()
    writer.writerows(rows)


if __name__ == "__main__":
    main(sys.argv[1])
Enter fullscreen mode Exit fullscreen mode

Run it:

pip install requests
python find_shopify.py domains.txt > shopify.csv
Enter fullscreen mode Exit fullscreen mode

Output

This is a real run on a small test list:

domain,shopify,evidence,myshopify,error
gymshark.com,True,header:powered-by;html:cdn.shopify.com;html:Shopify.theme;html:window.Shopify;html:myshopify.com,gymsharkusa.myshopify.com,
kyliecosmetics.com,True,header:powered-by;html:cdn.shopify.com;html:Shopify.theme;html:window.Shopify;html:myshopify.com,kylie-jenner-us.myshopify.com,
nalgene.com,False,,,
not-a-real-domain-xyz123.com,False,,,SSLError
python.org,False,,,
wikipedia.org,False,,,
www.allbirds.com,True,header:powered-by;html:cdn.shopify.com;html:Shopify.theme;html:window.Shopify;html:myshopify.com,weareallbirds.myshopify.com,
Enter fullscreen mode Exit fullscreen mode

The evidence column matters. When someone asks "why is this one marked as Shopify?", you can point to exactly what was found instead of trusting a black box.

Things to watch out for

  • Keep the error column. A domain that times out or has a broken certificate is "unknown", not "not Shopify". Retry those later instead of dropping them.
  • Headless stores. Some big brands use Shopify only as a backend (Hydrogen or a custom frontend). Their homepage may show none of the classic signals. A follow-up check of https://domain/products.json often still answers for those.
  • Bot protection. A few stores sit behind Cloudflare challenges and return a challenge page. Low concurrency and a normal user agent help, but you will not get 100% this way.
  • Be polite. One request per domain, a few workers, and no hammering. That is enough for this job.

Next step: the products

Once you know which domains are Shopify, most Shopify stores expose a public /products.json feed with titles, prices, variants and stock flags, so you can go from "list of domains" to "list of products" with one more script.

When the list gets big

For a few hundred domains the script above is all you need. If you want to check other platforms too (WooCommerce, HubSpot, Klaviyo and thousands more) or run it on a schedule without keeping a machine on, I built a hosted version on Apify: Bulk Tech Stack Detector takes a domain list plus an onlyIfUses: ["Shopify"] filter and saves only the matching domains, and Shopify Products Scraper exports the catalogs of those stores. Unreachable domains are not charged.

Disclosure: I built and maintain both of these Apify actors.

Top comments (0)