How to Check Domain Health (DNS, SSL, SPF, DMARC) in Bulk Without a Paid API

#apify #webscraping #api #seo

The problem

"Is this domain healthy?" sounds like one question until you actually check it. The answer lives in a dozen places: A and AAAA records, name servers, MX routes, the SPF TXT record, the DMARC policy, and the TLS certificate on port 443 — each with its own command, its own output format, and its own way of misleading you. A Null MX record means the domain deliberately accepts no mail, which looks identical to a broken MX if you do not know the convention. Two SPF records is a configuration defect, not redundancy. A certificate can be present but untrusted, or valid but expiring next week.

Run that checklist across a client portfolio or a CRM export of fifty domains and the manual terminal work — dig here, openssl s_client there, squint at a DMARC p= tag — eats the afternoon. And a transient resolver timeout is easy to misread as "the domain is down" when it is really "the lookup failed."

What the actor does

The Domain Health Checker performs direct DNS lookups and a live TLS handshake on port 443 for up to 100 domains per run, and returns one timestamped evidence card per unique domain. Each complete card includes:

DNS inventory — A, AAAA, NS, priority-sorted MX and the www CNAME;
email authentication — SPF record contents and record count, the DMARC record and its exact organizational p= policy, plus Null MX recognition, so "this domain explicitly accepts no mail" is never mistaken for a working exchanger;
TLS metadata — issuer, expiry date, whole days remaining, trust status (ssl.authorized) and any authorization error;
a 0–100 health score across seven deterministic checks — DNS resolution, email routing, exactly one SPF record, DMARC enforcement (p=quarantine or p=reject), TLS trust, more than 30 days of TLS runway, and DNS authority — with a plain-English issues list and a recommendedAction such as FIX_TLS_CERTIFICATE, ENFORCE_DMARC_POLICY, PUBLISH_SPF_POLICY or RESTORE_DNS_RESOLUTION, plus an actionPriority.

The score is only published when every required source is known. A resolver timeout or SERVFAIL makes the audit partial, the score null, and the row free — a source outage is never sold as a defect in your domain. A blocked state means the name resolved to a private or otherwise unsafe address (refused, free). Known absence is treated differently from failure: a clean NXDOMAIN answer is a complete, actionable observation — "this domain does not resolve" — and is billed as one.

Input is a list of 1–100 domains or HTTP(S) URLs; scheme, path, query, trailing dot and a leading www. are normalized away, Unicode names are converted to Punycode, and custom ports, embedded credentials and IP literals are rejected. Duplicates are audited once. A run-level OUTPUT receipt reconciles requested, unique, duplicate, delivered, paid, free and withheld rows, plus fatal state and replay safety.

Example: input and output

{
  "domains": [
    "apify.com",
    "github.com"
  ],
  "maxConcurrency": 2
}

An example evidence card from the README (the structure is illustrative; DNS records and certificate dates change over time — use the values from your own run):

{
  "domain": "brand.example",
  "found": true,
  "auditState": "complete",
  "healthScore": 71,
  "records": {
    "a": ["203.0.113.10"],
    "mx": ["mail.brand.example"],
    "ns": ["ns1.provider.example", "ns2.provider.example"]
  },
  "email": {
    "canReceive": true,
    "nullMx": false,
    "spf": "v=spf1 include:sender.example -all",
    "spfRecordCount": 1,
    "dmarc": "v=DMARC1; p=none; rua=mailto:dmarc@brand.example",
    "dmarcRecordCount": 1,
    "dmarcPolicy": "none"
  },
  "ssl": {
    "issuer": "Example CA",
    "validTo": "2026-11-30T23:59:59.000Z",
    "daysLeft": 111,
    "authorized": true
  },
  "issues": [
    "DMARC policy is \"none\" — monitoring only; failing mail is not quarantined or rejected."
  ],
  "recommendedAction": "ENFORCE_DMARC_POLICY",
  "actionPriority": "high",
  "safeToAutomate": false
}

Pricing and the free limit

Pay per event: $0.005 per run start plus $0.005 per unique complete domain audit delivered. Partial, blocked and withheld rows are free, and duplicates never create a second audit. One hundred complete unique audits cost about $0.505 including the start.

Apify's free plan gives $5 of usage credits per month. At this tariff, $5 covers up to 999 domain audits in a single run (0.005 + 0.005 × 999 = $5.00) — or nine full 100-domain runs at about $0.505 each.

Try it

Paste a domain list, start the run, and sort the dataset by actionPriority: Domain Health Checker

For AI agents and MCP

The actor takes JSON in and returns structured JSON evidence cards plus the OUTPUT receipt, so an agent can call it through the Apify API and check replaySafe and fatal before importing anything — then route rows by auditState and recommendedAction instead of inventing its own diagnosis. The README confirms the actor is callable from an AI agent through the Apify API, SDK or MCP integration, and documents the MCP server setup alongside JavaScript, Python and CLI examples. The review boundary is explicit: safeToAutomate is always false, so evidence routes to a human before anyone touches DNS, certificates or mail policy.

DEV Community