DEV Community

Cover image for Find out what any website is built with, in bulk, without a Wappalyzer subscription
Joshua Smith
Joshua Smith

Posted on

Find out what any website is built with, in bulk, without a Wappalyzer subscription

#seo #webdev #opensource #api

You have a list of a few hundred domains and you want to know which run WordPress, which are on Shopify, who uses HubSpot, who is behind Cloudflare. The browser extensions do this one tab at a time. Wappalyzer's API is subscription-only and BuiltWith starts at $295 a month. Here is a way to do it for less than a cent per site.

How technology detection works

Every technology leaves traces in a page: a Set-Cookie: PHPSESSID, a <meta name="generator" content="WordPress 6.6">, a script URL under /wp-content/, a Server: cloudflare header, an x-shopify-stage header. The open-source webappanalyzer project maintains 7,600 such fingerprints (it is the community continuation of the original Wappalyzer rule set). Matching them is mostly regular expressions against headers, cookies, meta tags, script sources and the DOM, plus a step that resolves implied technologies (WordPress implies PHP and MySQL) and removes contradictions.

Doing it yourself means fetching each page politely, handling redirects and timeouts, keeping the fingerprint bundle fresh, and implementing the implication logic. That is a weekend of work and then maintenance.

The two-minute version

Website Tech Stack Detector is an Apify Actor that does exactly the above over plain HTTP (no browser), and I published the source under GPL-3.0 at github.com/josh99smith/tech-stack-detector.

  1. Open the Actor page and paste your URLs into Website URLs, one per line. The scheme is optional.
  2. Optionally turn on Include detection evidence to see which header, cookie or script triggered each match.
  3. Click Start. Download the results as JSON, CSV or Excel, or send them to Google Sheets from the Integrations tab.

From code

One HTTP call runs the Actor and returns the dataset:

curl -X POST "https://api.apify.com/v2/acts/josh99smith~tech-stack-detector/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "urls": ["https://www.shopify.com", "wordpress.org", "https://www.nytimes.com"], "detectViaDns": true }'
Enter fullscreen mode Exit fullscreen mode

Python, with the official client:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_API_TOKEN>")
run = client.actor("josh99smith/tech-stack-detector").call(
    run_input={"urls": ["https://www.shopify.com", "wordpress.org"], "detectViaDns": True}
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["url"], item.get("byCategory", {}).get("CMS"), item.get("byCategory", {}).get("CDN"))
Enter fullscreen mode Exit fullscreen mode

What you get back

One record per site (trimmed):

{
    "url": "https://wordpress.org/",
    "success": true,
    "statusCode": 200,
    "title": "Blog Tool, Publishing Platform, and CMS",
    "technologyCount": 13,
    "technologies": [
        { "name": "WordPress", "categories": ["CMS", "Blogs"], "version": "7.2", "confidence": 100 },
        { "name": "Nginx", "categories": ["Web servers", "Reverse proxies"], "version": null, "confidence": 100 },
        { "name": "Google Tag Manager", "categories": ["Tag managers"], "version": null, "confidence": 100 }
    ],
    "byCategory": { "CMS": ["WordPress"], "Web servers": ["Nginx"], "Tag managers": ["Google Tag Manager"] },
    "server": "nginx",
    "responseTimeMs": 474
}
Enter fullscreen mode Exit fullscreen mode

Sites that cannot be reached come back as { "success": false, "errorType": "dns" | "timeout" | "blocked" | ... } so nothing silently disappears from your list.

Cost and limits

Pricing is pay-per-event: $0.008 per successfully analysed site, nothing for failures, no start fee, and a cost cap you set per run. A thousand domains cost about $8. Apify's free plan includes $5 of monthly usage, which covers roughly 600 sites before you pay anything.

The honest limitation: detection is HTTP-only. Technologies that only reveal themselves after JavaScript runs in a real browser are missed. In practice CMS, ecommerce platform, CDN, hosting, analytics and tag managers are almost always visible in the raw HTML and headers; some client-side widgets are not.

Use it from an AI agent

The Actor is a tool in the Apify MCP server. Add https://mcp.apify.com?tools=josh99smith/tech-stack-detector to Claude Desktop, Cursor or Claude Code and ask "what is shopify.com built with?".

Disclosure: I built this Actor. Bugs and misdetections go in the Issues tab on the Actor page or as GitHub issues; fingerprint fixes belong upstream at enthec/webappanalyzer.

Top comments (0)