DEV Community

Zahid Thani
Zahid Thani

Posted on Fully Autonomous

Check which websites use Shopify, WordPress or Klaviyo from a list of URLs

Disclosure: I built the tech-stack detector used in the second half of this post, and I earn money when people run it. The DIY version in the first half is free and is all some people need.

You have a list of domains, maybe a lead list or a set of competitors, and you want to know which are Shopify stores, which run WordPress, and who uses Klaviyo for email marketing. The Wappalyzer and BuiltWith browser extensions answer that for one site at a time. For 500 sites you want a script.

The DIY version (works from your laptop)

Most platforms leave fingerprints in the homepage HTML, and Shopify also has a recognisable IP range:

import re
import socket
import urllib.request

SIGNS = {
    "Shopify": r"cdn\.shopify\.com|Shopify\.theme",
    "WordPress": r"/wp-content/|/wp-includes/",
    "Klaviyo": r"static\.klaviyo\.com|klaviyo\.com/onsite",
}


def quick_check(domain):
    req = urllib.request.Request(f"https://{domain}", headers={"User-Agent": "Mozilla/5.0"})
    html = urllib.request.urlopen(req, timeout=15).read().decode("utf-8", "ignore")
    found = [name for name, pattern in SIGNS.items() if re.search(pattern, html)]
    if "Shopify" not in found and socket.gethostbyname(domain).startswith("23.227.38."):
        found.append("Shopify")  # Shopify's storefront IP range
    return found


for d in ["allbirds.com", "colourpop.com", "techcrunch.com"]:
    print(d, quick_check(d))
Enter fullscreen mode Exit fullscreen mode

Run from my home connection:

allbirds.com ['Shopify']
colourpop.com ['Shopify', 'Klaviyo']
techcrunch.com ['WordPress']
Enter fullscreen mode Exit fullscreen mode

That's genuinely useful, and if it covers your case, stop here. The problems start when you scale it:

  1. Shopify stores rate-limit cloud IPs. Run the same code from a server or a cloud function and many stores answer HTTP 429 Too Many Requests. Your regex then sees an error page and finds nothing.
  2. You need many more rules. Real lists contain Webflow, Wix, Squarespace, WooCommerce, BigCommerce, Magento, HubSpot, Mailchimp, Gorgias, Yotpo and dozens more, each with its own fingerprints and false positives.
  3. Some of the best signals aren't in the HTML. The company's email provider (Google Workspace vs Microsoft 365) lives in DNS MX/SPF records, and so do verification records for tools like Salesforce or Atlassian.
  4. Concurrency, timeouts, redirects and dead domains all need handling.

The hosted version

I packaged a bigger version of this (about 180 rules plus DNS checks) as an Actor on Apify. You send it domains, URLs or even work email addresses, and get one row per site with lead-friendly columns (cms, ecommercePlatform, emailProvider, cdn...) plus the full technology list.

pip install apify-client pandas
export APIFY_TOKEN=your_token_here   # Apify Console > Settings > API & Integrations
Enter fullscreen mode Exit fullscreen mode
import os

import pandas as pd
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

websites = [
    "allbirds.com",
    "colourpop.com",
    "techcrunch.com",
    "wordpress.org",
    "hubspot.com",
]


def detect(sites, proxy=None):
    run_input = {"websites": sites, "includeDns": True}
    if proxy:
        run_input["proxyConfiguration"] = proxy
    run = client.actor("zahidthani/website-tech-stack-detector").call(
        run_input=run_input, max_total_charge_usd=0.05
    )
    # apify-client 3.x returns a Run object; 1.x and 2.x return a dict
    dataset_id = getattr(run, "default_dataset_id", None) or run["defaultDatasetId"]
    return list(client.dataset(dataset_id).iterate_items())


items = detect(websites)

# Sites that block cloud IPs (many Shopify stores do) come back free and flagged.
# Re-check just those through residential proxies for a full page scan.
blocked = [it["input"] for it in items if it.get("pageBlocked")]
if blocked:
    print("re-checking via residential proxy:", blocked)
    retry = detect(blocked, {"useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"]})
    items = [it for it in items if not it.get("pageBlocked")] + retry

rows = []
for it in items:
    if it["type"] != "website":
        print("could not analyse:", it["input"], it.get("error"))  # dead domains are free
        continue
    names = {t["name"] for t in it["technologies"]}
    rows.append({
        "domain": it["domain"],
        "cms": it["cms"],
        "ecommerce": it["ecommercePlatform"],
        "shopify": "Shopify" in names,
        "wordpress": "WordPress" in names,
        "klaviyo": "Klaviyo" in names,
        "email": it["emailProvider"],
        "cdn": it["cdn"],
        "blocked": it.get("pageBlocked", False),
    })

df = pd.DataFrame(rows)
print(df.to_string(index=False))
df.to_csv("tech_stack.csv", index=False)

# Every technology found, one row per site + technology (handy for pivots)
long = pd.DataFrame(
    [{"domain": it["domain"], **t} for it in items if it["type"] == "website" for t in it["technologies"]]
)
print(long.groupby("category")["name"].unique().head(12))
Enter fullscreen mode Exit fullscreen mode

Tested with Python 3.12 and apify-client 3.2.1.

Real output

From a run on 6 October 2026. In the first pass, both Shopify stores answered Apify's datacenter IPs with HTTP 429 (exactly problem 1 above). They still came back as Shopify, from the DNS A record, and weren't charged. The script then re-checked those two through residential proxies and got the full pages:

domain cms ecommerce shopify wordpress klaviyo email provider cdn
techcrunch.com WordPress False True False Microsoft 365
wordpress.org WordPress False True False
hubspot.com HubSpot CMS False False False Google Workspace Cloudflare
colourpop.com Shopify True False True Cloudflare
allbirds.com Shopify True False False Microsoft 365 Cloudflare

The long table (one row per site and technology) is where it gets interesting for lead research. Across these five sites it found, among others: Klaviyo, Gorgias and Zendesk (support), Yotpo and Okendo (reviews), Klarna, OneTrust (cookie consent), Microsoft Clarity, Google Tag Manager, Mimecast (email security), Amazon Route 53 and Cloudflare DNS, and verified SaaS accounts such as Atlassian, DocuSign, Zoom and Stripe from DNS TXT records.

Filtering a real lead list is then plain pandas:

shopify_without_klaviyo = df[df["shopify"] & ~df["klaviyo"]]
Enter fullscreen mode Exit fullscreen mode

(On this sample that's allbirds.com. Note that "not detected on the homepage" is not proof they don't use it.)

What it costs

$2 per 1,000 websites analysed + $0.0005 per run. Dead domains, error pages and blocked sites are free, which is why the first pass here charged only 3 of the 5 sites. This whole example cost about $0.01. Apify's free plan includes $5 of monthly credit, so you can check a couple of thousand domains before paying anything.

Limitations

  • Homepage + DNS only. Tools that only load on the cart, checkout or after login won't show up. Pass a specific URL if you care about a particular page.
  • No history. It tells you what a site uses now, not when they adopted it. BuiltWith's paid database is the tool for historical data and huge coverage.
  • Absence isn't proof. Sites can load tools through a tag manager after consent, which a single page load may not trigger.
  • Residential proxies depend on your Apify plan. If a re-check can't use them, you still get the free DNS-only row (platform by IP, email provider, CDN).
  • Use it on business domains, for prospecting or market research. If you paste work emails, only the domain is used.

Try it

Website Tech Stack Detector on the Apify Store: https://apify.com/zahidthani/website-tech-stack-detector

Runnable Python, Node.js and n8n examples: https://github.com/zahidthani/apify-data-tools-examples

Top comments (0)