DEV Community

Ben Barazandeh
Ben Barazandeh

Posted on

Domain Age, Email Provider and DMARC From a List of Domains: RDAP + DNS-over-HTTPS in Python

#python #dns #security #api

Disclosure: I'm Ben, and I built the Apify Actor mentioned later in this post. I charge per result for it. Everything in the tutorial part works with free tools too.

A list of domains says a lot more than it looks. From public data alone you can tell how old a domain is, when it expires, who hosts its DNS, whether the company is on Google Workspace or Microsoft 365, and whether it has a DMARC policy. That's useful for fraud checks ("this domain is 12 days old"), deliverability audits, and segmenting a lead list ("Microsoft 365 shops with no DMARC").

You don't need a WHOIS API subscription for most of it. You need two open protocols: RDAP and DNS.

RDAP in one paragraph

RDAP is the official, structured JSON successor to WHOIS, run by the same registries. IANA publishes a bootstrap file (data.iana.org/rdap/dns.json) that tells you which RDAP server handles each TLD. You look up the TLD, then call {server}/domain/{name}. A 404 means the registry has no record, so the domain is probably available.

Step 1: Registration dates + DNS in ~30 lines

This ran as-is on 25 Oct 2026. DNS goes through Cloudflare's public DNS-over-HTTPS JSON endpoint, so there's nothing to install beyond requests.

import requests

def rdap(domain):
    tld = domain.rsplit(".", 1)[-1]
    services = requests.get("https://data.iana.org/rdap/dns.json", timeout=30).json()["services"]
    base = next((urls[0] for tlds, urls in services if tld in tlds), None)
    if not base:
        return None  # this TLD has no RDAP server: fall back to port-43 WHOIS
    r = requests.get(f"{base.rstrip('/')}/domain/{domain}", timeout=30)
    if r.status_code == 404:
        return {"registered": False}
    ev = {e["eventAction"]: e["eventDate"] for e in r.json().get("events", [])}
    return {"registered": True, "created": ev.get("registration"), "expires": ev.get("expiration")}

def dns(name, rtype):
    r = requests.get("https://cloudflare-dns.com/dns-query", params={"name": name, "type": rtype},
                     headers={"accept": "application/dns-json"}, timeout=30).json()
    return [a["data"] for a in r.get("Answer", [])]

for d in ["stripe.com", "spiegel.de"]:
    print(d, rdap(d))
    print("  MX   ", dns(d, "MX")[:2])
    print("  DMARC", dns(f"_dmarc.{d}", "TXT"))
Enter fullscreen mode Exit fullscreen mode

Output (trimmed):

stripe.com {'registered': True, 'created': '1995-09-12T04:00:00Z', 'expires': '2027-09-11T04:00:00Z'}
  MX    ['10 aspmx.l.google.com.', '20 alt1.aspmx.l.google.com.']
  DMARC ['"v=DMARC1; p=reject; pct=100; fo=1; rua=mailto:dmarc-reports@stripe.com; ...']
spiegel.de None
  MX    ['0 spiegel-de.mail.protection.outlook.com.']
  DMARC ['"v=DMARC1; p=quarantine; rua=mailto:report.dmarc@spiegel.de; ...']
Enter fullscreen mode Exit fullscreen mode

Two facts fall straight out: Stripe's mail goes to Google (aspmx.l.google.com), Der Spiegel's to Microsoft 365 (mail.protection.outlook.com). And .de has no RDAP server in IANA's bootstrap, so rdap() returned None.

Step 2: The pitfalls

  • ccTLDs are a patchwork. Some have RDAP, some only port-43 WHOIS with their own text format, and some publish no dates at all. .de doesn't publish registration dates. A few registries (for example .es) offer no public RDAP or WHOIS.
  • "Registrable domain" isn't "last two labels". news.bbc.co.uk belongs to bbc.co.uk, not co.uk. Use the Public Suffix List, not string splitting. The same goes for turning emails and URLs into domains.
  • Registrant data is mostly redacted since GDPR. Expect the organisation and country at best.
  • DNS answers need cleaning. MX records come with a priority prefix and trailing dot. TXT records come quoted and can be split into chunks.
  • Email provider is inference. You map MX hosts (google.com, outlook.com, zoho…) to providers, and SPF include:s to sending tools (HubSpot, Salesforce, SendGrid…). It's reliable for the big providers and fuzzy for self-hosted setups.
  • DKIM can't be listed. You can only probe common selectors, and some domains answer every selector (wildcards), which looks like a false positive.
  • Rate limits. Registry RDAP servers throttle bulk lookups. Add concurrency limits and backoff.

Step 3: Or use the packaged version

I packaged all of this as an Apify Actor, Bulk WHOIS & DNS Lookup. You paste domains, URLs or email addresses, and it normalises them to the registrable domain (co.uk, com.au…) and dedupes.

{
  "domains": ["stripe.com", "https://www.apify.com/store", "jane@hubspot.com", "bbc.co.uk", "notion.so"],
  "includeWhois": true, "includeDns": true, "includeWebsite": true, "concurrency": 5
}
Enter fullscreen mode Exit fullscreen mode

One row per domain: registrar and IANA ID, created/updated/expiry dates, domain age in days, days until expiry, EPP status, nameservers, A/AAAA/MX/NS/TXT, SPF and its includes, DMARC record and policy, optional DKIM probing (with wildcard detection), email provider, email security gateway, email senders, DNS provider, about 50 SaaS tools detected from DNS verification records, and an optional website check (status, redirects, title, basic tech like WordPress or Shopify). It's RDAP first with WHOIS fallback.

A real run on 18 Oct 2026 with 7 inputs and every check on took about 46 seconds:

Domain Registrar Created Email provider DMARC DNS provider SaaS found
stripe.com SafeNames Ltd. 1995-09-12 Google Workspace reject Amazon Route 53 12
openai.com MarkMonitor Inc. 2007-01-19 Google Workspace reject Azure DNS 19
hubspot.com MarkMonitor Inc. 2005-02-06 Google Workspace reject Cloudflare 17
bbc.co.uk British Broadcasting Corpor… 1994-12-13 Other / self-hosted reject – 10
spiegel.de – – Microsoft 365 quarantine – 6

ceo@hubspot.com and https://www.apify.com in the input were normalised to their domains. Same .de story as my script: DNS and email data, no WHOIS dates.

Useful filters once it's in a sheet: domainAgeDays < 90 for new-domain risk, daysUntilExpiry < 30 for portfolio renewals, emailProvider plus dmarcPolicy for deliverability or security outreach.

Limits, honestly

  • WHOIS coverage depends on the registry. No dates for .de, nothing for TLDs with no public RDAP/WHOIS (you get a whoisError note plus DNS data).
  • Registrant fields are usually null because of privacy redaction.
  • Provider and SaaS detection come from DNS records, so they show what a domain has verified or routed, not what it actively uses.
  • The website check is slower. Turn it off for big lists and raise concurrency (up to 20).
  • registered: false is a strong hint, not a guarantee. Confirm at a registrar before you buy.

What it costs

$0.003 per domain ($3 per 1,000), with WHOIS/RDAP, DNS, email intelligence and the website check included. No start fee, and runs stop at your "maximum cost per run". The 7-domain test was about 2 cents. If you only need registrar and expiry, a WHOIS-only tool can be cheaper. And the script above is free.

Play fair

Some registries restrict reusing WHOIS data for marketing. Read their terms, and keep privacy laws in mind when you combine domain data with people.

What signal would you want added: MTA-STS, BIMI, CAA, something else? Tell me in the comments.

Top comments (0)