DEV Community: PennyData

Detect the tech stack of 500 websites in one run (no browser, about $2.50)

PennyData — Mon, 19 Oct 2026 20:00:16 +0000

Say you have a list of 500 company websites and you want to know which run WordPress, which are on Shopify, who uses Cloudflare or Google Tag Manager. That's a typical prospecting, competitor-research or migration-audit question. Services like BuiltWith are great but priced for heavier use. For a quick, coarse pass, a single HTTP request per site gets you surprisingly far.

This post walks through doing that with an Apify actor I built, Website Tech Stack Detector. Disclosure: it's my actor and I earn a little when it's used.

How it works

For each URL it makes one plain HTTP request (no browser, no JavaScript execution), follows up to 5 redirects, and looks at:

response headers (server, cf-ray, x-vercel-id, ...)
<meta name="generator">
script URLs and a few well-known HTML markers (__NEXT_DATA__, /wp-content/)
cookie names (values are never read or stored)

About 60 technologies are recognised: CMS (WordPress, Drupal, Wix...), ecommerce (Shopify, WooCommerce, Magento...), analytics/tag managers, CDN/hosting, web servers, JS frameworks. Every detection includes the evidence that triggered it, so you can judge it yourself. It respects robots.txt by default.

Price

Pay-per-event: $0.005 per URL analyzed (at time of writing), so 500 sites is about $2.50. You're only charged for URLs fetched successfully (HTTP below 400). Timeouts, DNS failures, 4xx/5xx, non-public URLs and robots.txt skips are free. Compute is included.

Input

{
  "urls": [
    { "url": "https://wordpress.org" },
    { "url": "https://nextjs.org" },
    { "url": "https://www.drupal.org" },
    { "url": "example.com" }
  ],
  "respectRobotsTxt": true,
  "maxConcurrency": 5,
  "timeoutSecs": 20
}

Bare domains are accepted. For 500 sites, paste or generate the urls array from your spreadsheet; maxConcurrency goes up to 20.

Real output

I ran the actor's source locally on 10 URLs: 8 analyzed, 2 not analyzed and not charged (a 404 page and http://127.0.0.1, which is rejected as non-public). Two of the results:

[
  {
    "input": "https://nextjs.org",
    "finalUrl": "https://nextjs.org",
    "httpStatus": 200,
    "technologies": [
      {
        "name": "Vercel",
        "category": "Hosting",
        "evidence": "header x-vercel-id"
      },
      {
        "name": "Next.js",
        "category": "JS framework",
        "evidence": "script src /\\/_next\\/static\\//"
      }
    ],
    "capturedAt": "2026-10-16T05:53:25.120Z"
  },
  {
    "input": "https://www.drupal.org",
    "finalUrl": "https://new.drupal.org/",
    "httpStatus": 200,
    "technologies": [
      {
        "name": "Drupal",
        "category": "CMS",
        "evidence": "meta generator \"Drupal 11 (https://www.drupal.org)\""
      },
      {
        "name": "Google Tag Manager",
        "category": "Tag manager",
        "evidence": "html GTM noscript iframe"
      },
      {
        "name": "Fastly",
        "category": "CDN",
        "evidence": "header x-served-by"
      },
      {
        "name": "Apache",
        "category": "Web server",
        "evidence": "header server"
      },
      {
        "name": "jQuery",
        "category": "JS library",
        "evidence": "script src /jquery([.-]\\d[\\d.]*)?(\\.min)?\\.js/"
      }
    ],
    "capturedAt": "2026-10-16T05:53:25.120Z"
  }
]```
{% endraw %}


Failed rows come back with an {% raw %}`error`{% endraw %} field instead of technologies, so you can retry or drop them.

## Be realistic about accuracy

This is a coarse tool, and my test run shows why:

- {% raw %}`https://www.shopify.com`{% endraw %} returned only **Cloudflare**. A single request saw nothing else. Absence of a detection is never proof a technology isn't used.
- {% raw %}`github.com`{% endraw %} came back as GitHub hosting plus Contentful, not a full stack.
- No JavaScript is executed, so tech that appears only after client-side rendering, or only on other pages, is missed. No version numbers are reported (except what a generator tag states).
- Sites behind bot walls or logins may return a block page. No proxies or captcha solving are used.

It's best for questions like "which of these 500 are on WordPress / Shopify / Cloudflare?", where the signal is in headers or the HTML, and weaker for building a complete technographic profile. I didn't run a 500-URL batch for this post, so I won't quote timing for one. Try 20 of your own sites first and compare with what you know.

## Call it from code

Small lists, one synchronous request:
{% raw %}


```bash
curl -X POST \
  "https://api.apify.com/v2/acts/pennydata~website-tech-stack-detector/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":[{"url":"https://wordpress.org"},{"url":"https://nextjs.org"}]}'

For 500 URLs use an async run with the Python client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
urls = [{"url": u.strip()} for u in open("sites.txt") if u.strip()]
run = client.actor("pennydata/website-tech-stack-detector").call(
    run_input={"urls": urls, "maxConcurrency": 10}
)
rows = client.dataset(run["defaultDatasetId"]).list_items().items

wordpress = [r["input"] for r in rows if "WordPress" in r.get("technologyNames", [])]
print(len(wordpress), "WordPress sites out of", len(rows))

technologyNames is a flat list for easy filtering; technologies has name, category and evidence per detection.

Wrap-up

For a few dollars you can tag a whole prospect list by CMS, ecommerce platform, CDN and analytics, with the evidence attached. Treat it as a first-pass filter and verify anything important by hand.

Written with AI (Claude).

Validate a list of 10,000 emails without SMTP for about $10

PennyData — Mon, 19 Oct 2026 07:03:48 +0000

Before you import a lead list or send a campaign, you want to drop the junk: typos, fake domains, throwaway inboxes. Many verifiers do this by opening SMTP connections to each mail server, which is slow and often gets IPs blocked. A cheaper first pass skips SMTP and checks everything you can know from the address and DNS alone.

This post shows how to run that pass on a CSV or a list of up to 10,000 addresses with an Apify actor I built, Email Verifier & Validator (No SMTP). Disclosure: it's my actor and I earn a little when it's used. The checks themselves are plain DNS, so you could also reimplement them yourself.

What it checks (and what it doesn't)

Per address, with no SMTP probing:

Syntax (length limits, bad characters, internationalized domains)
DNS: MX records, with A/AAAA fallback, and null-MX detection
Disposable-domain list, role accounts (info@, sales@), free providers (gmail etc.)
Typo suggestions for common providers (yaho.com -> yahoo.com)

What it can't do: it cannot tell if the mailbox jane@company.com exists. valid: true means "well-formed, and the domain accepts mail". Catch-all domains, full mailboxes and made-up users on real domains all pass. If you need mailbox-level verification, you need an SMTP-based verifier (more expensive, and results are often "unknown" anyway). Use this as a cheap first filter.

Price

Pay-per-event: $0.001 per email checked (at time of writing), so 10,000 emails is about $10, 1,000 is about $1. Duplicates in your list are removed before billing. Addresses whose DNS lookup errored out (status unknown) are not charged; invalid-syntax rows are charged like any other row. Compute is included in the per-event price.

Input

{
  "emails": ["jane.doe@gmail.com", "bob@yaho.com", "temp@10minutemail.com", "bad@@example.com"],
  "concurrency": 10
}

For a CSV, give a public URL instead (it uses the column whose header contains "email", or csvColumn):

{ "csvUrl": "https://example.com/leads.csv", "csvColumn": "email", "maxEmails": 10000 }

maxEmails defaults to 10,000 (maximum 100,000) and is a safety cap: you can also set a max-cost limit on the run in Apify, and the actor stops when it's reached.

Real output

I ran the actor's source locally on a 15-address test list (7 deliverable-domain, 3 disposable, 2 bad domain, 3 bad syntax). Four of the rows (mxRecords shortened):

[
  {
    "input": "jane.doe@gmail.com",
    "email": "jane.doe@gmail.com",
    "valid": true,
    "status": "deliverable_domain",
    "reason": null,
    "domain": "gmail.com",
    "mxRecords": [
      "gmail-smtp-in.l.google.com",
      "alt1.gmail-smtp-in.l.google.com",
      "..."
    ],
    "hasMx": true,
    "hasARecord": true,
    "isDisposable": false,
    "isRole": false,
    "isFreeProvider": true,
    "didYouMean": null,
    "checkedAt": "2026-10-16T05:53:24Z"
  },
  {
    "input": "bob@yaho.com",
    "email": "bob@yaho.com",
    "valid": false,
    "status": "invalid_domain",
    "reason": "likely_typo_no_mx",
    "domain": "yaho.com",
    "mxRecords": [],
    "hasMx": false,
    "hasARecord": true,
    "isDisposable": false,
    "isRole": false,
    "isFreeProvider": false,
    "didYouMean": "bob@yahoo.com",
    "checkedAt": "2026-10-16T05:53:24Z"
  },
  {
    "input": "temp@10minutemail.com",
    "email": "temp@10minutemail.com",
    "valid": false,
    "status": "disposable",
    "reason": "disposable_domain",
    "domain": "10minutemail.com",
    "mxRecords": [],
    "hasMx": false,
    "hasARecord": true,
    "isDisposable": true,
    "isRole": false,
    "isFreeProvider": false,
    "didYouMean": null,
    "checkedAt": "2026-10-16T05:53:24Z"
  },
  {
    "input": "bad@@example.com",
    "email": null,
    "valid": false,
    "status": "invalid_syntax",
    "reason": "invalid_local_part",
    "domain": null,
    "mxRecords": [],
    "hasMx": false,
    "hasARecord": false,
    "isDisposable": false,
    "isRole": false,
    "isFreeProvider": false,
    "didYouMean": null,
    "checkedAt": "2026-10-16T05:53:24Z"
  }
]

Note the statuses: deliverable_domain, disposable, invalid_domain (with a didYouMean hint) and invalid_syntax. Filter on valid == true to keep the good ones. I did not run a 10,000-row batch for this post, so I won't quote timing for one. Run time depends mostly on how many distinct domains you have, since DNS results are cached per domain.

Call it from code

Run synchronously and get the rows back in one request (fine for small lists; use async runs for big ones):

curl -X POST \
  "https://api.apify.com/v2/acts/pennydata~bulk-email-validator-no-smtp/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"emails":["jane.doe@gmail.com","bob@yaho.com"]}'

Python, with the official client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("pennydata/bulk-email-validator-no-smtp").call(
    run_input={"csvUrl": "https://example.com/leads.csv", "csvColumn": "email"}
)
rows = client.dataset(run["defaultDatasetId"]).list_items().items
good = [r["input"] for r in rows if r["valid"] and not r["isDisposable"]]
print(len(good), "of", len(rows), "kept")

Pipeline suggestion

Run this pass over the whole list (about $0.001 each).
Drop invalid_syntax, invalid_domain, disposable; apply didYouMean fixes if you like.
Optionally treat isRole addresses separately for cold outreach.
Send only the survivors to a more expensive SMTP verifier, if you need one at all.

That keeps the pricey step small. Try it on a few hundred addresses first and compare against your own bounce data before trusting it on the full list.

Written with AI (Claude).