DEV Community: SwiftKit

Validate email lists without sending a single email (DNS, typos, disposables, duplicates)

SwiftKit — Fri, 02 Oct 2026 05:26:13 +0000

Most "email verification" tutorials end with the same trick: open an SMTP connection to the recipient's mail server and ask whether the mailbox exists. It works less and less. Gmail, Outlook and most corporate servers accept everything (catch-all) or answer vaguely, and probing from a cloud IP is a quick way to get that IP blocklisted.

The good news: you can catch most of the junk in a list without contacting any mail server at all. Here's the checklist I ended up with.

1. Syntax, but realistic

The full RFC 5322 grammar allows things no real provider accepts (quoted local parts, comments, IP literals). For list cleaning, a practical rule is better: one @, a local part of normal characters, and a domain with a real TLD. Lowercase the domain, trim whitespace, and strip mailto: that people paste in.

2. Does the domain accept mail?

import { Resolver } from 'node:dns/promises';

const dns = new Resolver();
dns.setServers(['1.1.1.1', '8.8.8.8']);

async function acceptsMail(domain) {
  try {
    const mx = await dns.resolveMx(domain);
    // A "null MX" (RFC 7505) means: this domain never receives email.
    if (mx.length === 1 && mx[0].exchange === '') return false;
    return mx.length > 0;
  } catch (e) {
    if (e.code === 'ENODATA') {
      // No MX: mail falls back to the A record.
      try { return (await dns.resolve4(domain)).length > 0; } catch { return false; }
    }
    if (e.code === 'ENOTFOUND') return false; // domain doesn't exist
    throw e; // timeout etc.: report "unknown", don't guess
  }
}

Two lessons:

Treat timeouts as unknown, not invalid. Marking a real customer invalid because a DNS server hiccuped is worse than leaving one bad address in.
Cache per domain. A 50k list usually has only a few thousand distinct domains.

3. Typos of big providers

gmial.com, hotmal.com, gmail.con. These often do resolve (someone registered them), so DNS alone won't catch them. Compare the domain with a short list of popular providers using edit distance:

function editDistance(a, b) {
  const d = Array.from({ length: a.length + 1 }, (_, i) => [i]);
  for (let j = 1; j <= b.length; j++) d[0][j] = j;
  for (let i = 1; i <= a.length; i++)
    for (let j = 1; j <= b.length; j++)
      d[i][j] = Math.min(d[i - 1][j] + 1, d[i][j - 1] + 1,
        d[i - 1][j - 1] + (a[i - 1] === b[j - 1] ? 0 : 1));
  return d[a.length][b.length];
}
// distance 1–2 from gmail.com, outlook.com, yahoo.com… → suggest the fix

Only suggest when the domain is not itself a known provider, or you'll "correct" gmx.com to gmail.com.

4. Disposable and role addresses

Disposable: the community-maintained disposable-email-domains list (public domain, about 9,000 entries) catches most throwaway services. Refresh it regularly.
Role accounts: info@, sales@, support@, noreply@. They're real, but they're shared inboxes, and many email tools treat them as a risk. Flag them instead of deleting them.

5. Duplicates that don't look like duplicates

For Gmail, dots and +tags don't matter, so these are all the same inbox:

john.smith+news@gmail.com
johnsmith@googlemail.com
JohnSmith@gmail.com

Build a canonical form (lowercase, remove dots and the +tag for Gmail, map googlemail.com to gmail.com) and dedupe on that. Only remove +tags for providers where you know they're aliases.

6. Cheap extra signals

Domain age from the registry's RDAP service: a domain registered last week on a B2B sign-up form deserves a second look.
Parked domains: name servers like sedoparking.com mean "this domain is for sale", not a real company.
SPF / DMARC presence: not proof of anything alone, but real sending domains usually have them.

What this can't tell you

Honestly: none of this proves that a specific mailbox exists. "Valid" here means "well formed, the domain is real and accepts mail, and nothing looks off." For list hygiene before an import, that's usually the part that matters, and it's fast, cheap and doesn't touch your sender reputation.

If you'd rather not build it

I packaged this checklist as an Apify Actor: Bulk Email Validator. Paste emails or a CSV link, and you get valid / risky / invalid with the reason, a typo suggestion, disposable/role/free flags, mail provider, domain age, parked and SPF/DMARC flags, and duplicate detection. It never sends a test email. Pay-per-email, about $0.50 per 1,000.

What's the weirdest thing you've found in an email list?

Written with AI assistance; the snippets and numbers were tested before publishing.

How to detect any website's tech stack in bulk (and why a real browser finds 2x more)

SwiftKit — Fri, 02 Oct 2026 03:14:59 +0000

"What is this site built with?" sounds like a one-off question, until you need the answer for 5,000 domains. Sales teams want every Shopify store in a list. Agencies want to audit prospects. Analysts want to count which CMS wins in a niche.

Here's how technology detection actually works, and what I learned building an open-source detector.

Fingerprints: the core idea

Every tool leaves traces. The open-source webappanalyzer project (a community continuation of the old Wappalyzer data) catalogs 7,600+ technologies with patterns like:

"WordPress": {
  "meta": { "generator": "^WordPress ?([\\d.]+)?\\;version:\\1" },
  "html": "<link rel=[\"']stylesheet[\"'] [^>]+/wp-(?:content|includes)/",
  "implies": ["PHP", "MySQL"]
}

A detector loads the page and tests each pattern against:

Signal	Example
HTML	`/wp-content/` paths
Script URLs	`cdn.shopify.com/s/…`
Meta tags	`<meta name="generator" content="WordPress 6.8">`
Response headers	`server: cloudflare`, `x-powered-by: Next.js`
Cookies	`_shopify_y`, `hubspotutk`
DNS	MX records → Google Workspace or Microsoft 365, SPF includes
SSL issuer	Let's Encrypt, DigiCert…

Then it resolves implies (WordPress → PHP + MySQL), requires and excludes, and keeps only matches above a confidence threshold.

The catch: half the fingerprints need JavaScript

When I counted, 3,342 of 7,628 technologies have js patterns: things like window.ABTasty, window.Intercom or AFRAME.version. These only exist after the page's JavaScript runs. A plain HTTP request never sees them, and neither do tools that load later through tag managers.

So I ran the same sites both ways:

Site	HTML + headers + DNS	Headless Chrome
notion.so	18	38
allbirds.com	16	29
hubspot.com	33	56

Roughly 2× more detections with a real browser: analytics, chat widgets, A/B testing, consent tools and front-end frameworks.

Tips if you build this yourself:

Block images, fonts and media in the browser. You don't need them, and it makes each page much cheaper.
Evaluate all JS chains in one page.evaluate call, wrapped in try/catch (some getters throw).
Also feed the browser's network requests into the script-URL and XHR patterns. Many tools load from tag managers you won't see in the initial HTML.
Respect robots.txt, and expect some big sites to answer 403 to automation.

Open source + a hosted version

The detector is open source (GPL-3.0, like the fingerprint data):
github.com/swiftkit-dev/website-tech-stack-detector

If you'd rather not run browsers yourself, the same code runs on Apify as
Website Tech Stack Detector: paste a list of domains,
choose fast or deep scan, and get JSON/CSV with technologies grouped by category, versions and the
email provider. It's pay-per-site, and unreachable sites are free.

What would you use bulk tech detection for? I'm curious which categories people care about most.

Written with AI assistance; the endpoints, numbers and examples were tested before publishing.

How to get every open job from Greenhouse, Lever, Ashby and Workday with one API call

SwiftKit — Fri, 02 Oct 2026 03:14:42 +0000

If you've ever built a niche job board, a hiring-signal feed for sales, or just wanted to track who's hiring data engineers, you've probably hit the same wall: every company's careers page looks different.

The good news is that under the hood, most of them run on a small number of applicant tracking systems (ATS), and many of those publish open jobs through public, unauthenticated endpoints. They exist precisely so companies can show their openings anywhere. You don't need to scrape HTML.

Here's how the big ones work.

Greenhouse

curl "https://boards-api.greenhouse.io/v1/boards/airbnb/jobs?content=true"

The airbnb part is the company's board token, the same slug you see in boards.greenhouse.io/airbnb. You get title, location, departments, first-published date and (with content=true) the full description as HTML-escaped text.

Lever

curl "https://api.lever.co/v0/postings/palantir?mode=json"

Lever gives you categories.commitment (full-time, contract…), workplaceType (remote/hybrid/onsite), and a salaryRange object when the company publishes pay.

Ashby

curl "https://api.ashbyhq.com/posting-api/job-board/openai?includeCompensation=true"

Ashby is the friendliest for salary data: includeCompensation=true returns structured min/max, currency and interval. In a quick test, OpenAI's "Data Engineer" role came back as $235K – $385K.

Workday

Workday is the one everyone asks about, and the trickiest. Each company's career site exposes a JSON endpoint behind its public page:

curl -X POST "https://nvidia.wd5.myworkdayjobs.com/wday/cxs/nvidia/NVIDIAExternalCareerSite/jobs" \
  -H "Content-Type: application/json" \
  -d '{"appliedFacets":{},"limit":20,"offset":0,"searchText":""}'

Things to know:

You can't guess the URL from a company name. Tenant (nvidia), data center (wd5) and site name (NVIDIAExternalCareerSite) all vary, so start from the company's careers link.
Pages are capped at 20 jobs, and the total field is only filled in on the first page. If you read it on page 2, you'll stop early. (Ask me how I know.)
Dates come as text like "Posted 3 Days Ago", so you only get day precision.

The annoying part: normalizing everything

Each system has different field names, date formats, location shapes and salary structures. If you want one clean table, you end up writing a small adapter per ATS, plus:

detecting which ATS a company uses,
handling pagination and rate limits politely,
deduplicating, and
remembering which jobs you've already seen if you run it daily.

A shortcut

I packaged all of this as an Apify Actor: Company Jobs. You give it a list of companies (names, websites or careers URLs) and it returns one normalized row per open job from Workday, Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Teamtailor and Rippling:

{
  "company": "openai",
  "title": "Data Engineer",
  "location": "San Francisco; Mountain View",
  "salaryMin": 235000,
  "salaryMax": 385000,
  "salaryCurrency": "USD",
  "platform": "ashby",
  "url": "https://jobs.ashbyhq.com/openai/…"
}

Two features I use most:

"Only new jobs since my last run": put it on a daily schedule and get just the new postings. Great for hiring alerts or "which of my target accounts just opened a data team?"
Title, location and remote filters, so you only pay for jobs you actually want.

It's pay-per-result (about $1 per 1,000 jobs), and it only reads the public job boards that companies publish themselves.

Be a good citizen

Whatever route you take:

Stick to the official public endpoints above, not HTML scraping.
Keep concurrency low and back off on 429s.
Don't collect recruiter names or emails that some boards include. You don't need them for job data.

If there's an ATS you'd like covered next, tell me in the comments.

Written with AI assistance; the endpoints, numbers and examples were tested before publishing.