DEV Community: Sameer Sheikh

Block Bots Before They Hit Your API; IP Reputation in Node.js

Sameer Sheikh — Fri, 24 Jul 2026 15:51:58 +0000

Most API abuse gets caught after it lands. A rate limiter trips, an alert fires, someone checks the logs the next morning and finds a Tor exit node hammered /login four thousand times overnight. Reactive. I wanted to catch the traffic at the door instead, before it touches a single route handler.

The idea is simple. Every request arrives with an IP. That IP already has a reputation — it's a residential broadband line, or it's a datacenter, or it's a known VPN, or it's an address that's been throwing credential-stuffing attempts at half the internet for a week. If you know which one you're looking at before your handler runs, you can decide what to do with it. Block the obvious attackers. Challenge the suspicious ones. Wave the clean ones through without friction.

That last part matters. Most tutorials I've read on this treat it as a binary: VPN detected, block. But a corporate user on a datacenter IP and a bot running out of a bulletproof host are not the same threat, and slamming the door on both is how you lose real customers. So this middleware grades on a curve.

Here's the whole path a request takes before it ever reaches your handler:

The piece everyone skips

Here's the thing that took me longer than it should have: getting the real client IP.

Behind a load balancer, a CDN, or basically any modern deploy, req.socket.remoteAddress is the proxy in front of you, not the visitor. The real address sits in the X-Forwarded-For header. Express won't read it unless you tell it to trust the proxy, and if you trust it wrong, you've handed attackers a header they can spoof to look like 127.0.0.1. More on that later. For now, know that this one line does a lot of quiet work:

app.set("trust proxy", true);

Setup

Node 18+ (we're using the built-in fetch, no HTTP client dependency), Express, and a free WhoisFreaks key.

git clone https://github.com/WhoisFreaks/ip-reputation-middleware
cd ip-reputation-middleware
npm install
cp .env.example .env   # paste your API key

The key is free — 500 credits, no card. Each IP lookup costs one credit. That's enough to build and test the whole thing with credits to spare.

Step 1: One lookup

Start with the smallest possible thing that works. Given an IP, ask what its reputation is.

The endpoint is a plain GET. Auth is a query parameter called apiKey, and you pass the address in ip. The full request and response schema is documented in the WhoisFreaks IP Reputation API reference. The response comes back as an array with one object per IP, and the part we care about lives under security:

const BASE_URL = "https://api.whoisfreaks.com/v1.0/security";

export async function lookupIp(ip, { apiKey = process.env.WHOISFREAKS_API_KEY } = {}) {
  const url = `${BASE_URL}?apiKey=${apiKey}&ip=${encodeURIComponent(ip)}`;
  const res = await fetch(url);
  if (!res.ok) throw new Error(`whoisfreaks_http_${res.status}`);
  const body = await res.json();
  return Array.isArray(body) ? body[0] : body;
}

The security object is richer than the single boolean most APIs hand you. You get threat_score (0–100), plus is_vpn, is_proxy, is_residential_proxy, is_tor, is_bot, is_known_attacker, is_spam, and is_cloud_provider, along with confidence scores and provider names for the VPN and proxy hits. That granularity is what makes graded responses possible instead of a flat block.

Step 2: Decide, don't just detect

This is the core of the whole thing, and it's ten lines. The composite threat_score does the heavy lifting. I mapped it to four actions using the score bands WhoisFreaks documents:

export function decide(security) {
  const score = security?.threat_score ?? 0;
  if (score >= 76) return "block";      // known-bad: reject
  if (score >= 51) return "step_up";    // high: require extra auth
  if (score >= 26) return "challenge";  // elevated: CAPTCHA / rate limit
  return "allow";                       // low: let it through
}

Why four tiers and not two? Because a score of 40 usually means "datacenter or commercial VPN" — plenty of legitimate people browse from those. A score of 90 means the address is actively bad. You want the first group to see a CAPTCHA at worst and the second group to see nothing but a 403. One threshold can't express that.

Step 3: Wrap it in middleware

Now make it an Express middleware factory so each route or app can plug in its own responses per tier.

export function ipReputation(options = {}) {
  const { failOpen = true, onBlock, onChallenge, onStepUp } = options;

  return async function (req, res, next) {
    const ip = clientIp(req);
    if (isPrivate(ip)) return next();   // skip localhost / internal traffic

    let record;
    try {
      record = await lookupIp(ip);
    } catch (err) {
      if (failOpen) return next();      // API down? don't take the app down with it
      return res.status(503).json({ error: "reputation_unavailable" });
    }

    const security = record?.security ?? {};
    req.ipReputation = security;

    switch (decide(security)) {
      case "block":     return onBlock ? onBlock(req, res, next) : res.status(403).json({ error: "forbidden" });
      case "step_up":   return onStepUp ? onStepUp(req, res, next) : next();
      case "challenge": return onChallenge ? onChallenge(req, res, next) : next();
      default:          return next();
    }
  };
}

Two decisions in there are worth calling out.

Fail open. If the reputation API times out or returns a 500, the request goes through. I went back and forth on this. But think about what fail-closed means: your threat provider has one bad minute and your login page returns 503 to every real customer on earth. That's a self-inflicted outage in exchange for maybe blocking one risky request. Not worth it. Fail open, log the miss, move on.

Skip private IPs. Health checks, internal service calls, and your own localhost testing shouldn't burn credits or get scored. The isPrivate check short-circuits them.

Step 4: Get the client IP right

The helper the whole thing leans on:

export function clientIp(req) {
  return (req.ip || "").replace(/^::ffff:/, "");  // strip IPv4-mapped IPv6 prefix
}

That ::ffff: strip bit me the first time. Express hands you IPv4 addresses wrapped as IPv6 (::ffff:8.8.8.8), and if you pass that straight to the API you'll get nonsense back. Small detail, real bug.

On trust proxy: setting it to true is fine when every request genuinely passes through a proxy you control. If your app is also reachable directly, a client can send their own X-Forwarded-For and impersonate any IP they like, including a clean one. In that case set it to the exact number of proxy hops instead. Know your deploy.

Step 5: Cache, because credits and latency

Adding a network call to every request is a great way to make every request slower and drain 500 credits by lunch. The reputation data refreshes about once a day, so caching a result for a few hours costs nothing in accuracy:

const CACHE_TTL_MS = 12 * 60 * 60 * 1000;
const cache = new Map();
// check cache.get(ip) before fetch; cache.set(ip, { record, expires }) after

An in-memory Map is fine for one process. Swap in Redis the moment you run more than one. The free tier caps you at 10 requests per minute, so on a real site the cache isn't an optimization — it's the thing that keeps you under the limit at all.

Step 6: Test the part that matters

You don't need to hit the live API to test the logic that decides someone's fate. The tiering is pure and deterministic, so test the boundaries directly:

test("elevated scores get challenged", () => {
  assert.equal(decide({ threat_score: 26 }), "challenge");
  assert.equal(decide({ threat_score: 50 }), "challenge");
});

test("malicious scores are blocked", () => {
  assert.equal(decide({ threat_score: 76 }), "block");
  assert.equal(decide({ threat_score: 100 }), "block");
});

Off-by-one errors here are the whole ballgame. Get >= 76 wrong and you either block clean users at 75 or let attackers through at 76.

Real results

Running the suite:

▶ node --test

ok 1 - low scores are allowed
ok 2 - elevated scores get challenged
ok 3 - high scores require step-up auth
ok 4 - malicious scores are blocked
ok 5 - missing score defaults to allow (fail open)

# tests 5
# pass 5
# fail 0

Then I pointed the batch script at 20 IPs pulled from a day of access logs — a mix of hosting ranges, commercial VPN exit points, and a couple of addresses I already suspected. The script only prints what's worth acting on, so the clean ones drop out:

$ node enrich-logs.js sample-ips.txt
scoring 20 IPs...

45.83.220.14     score= 50  challenge  -
185.159.157.22   score= 80  block      is_known_attacker
193.176.86.103   score= 50  challenge  -
89.187.178.211   score= 50  challenge  -
138.199.21.90    score= 50  challenge  -
37.120.145.60    score= 50  challenge  -
143.244.47.201   score= 50  challenge  -
185.220.101.34   score= 85  block      is_tor,is_known_attacker
195.181.170.12   score= 50  challenge  -
198.54.117.200   score= 35  challenge  is_known_attacker
5.253.207.88     score= 50  challenge  -
146.70.124.19    score= 50  challenge  -
217.138.222.101  score= 50  challenge  -
84.17.35.176     score= 50  challenge  -
156.146.63.128   score= 50  challenge  -

Twenty in, fifteen surfaced. Two hit the block tier outright. The other five scored low enough to pass and never printed.

What I noticed

A few things stood out once I had real scores in front of me:

The two blocks were the clear-cut ones. 185.220.101.34 came back is_tor and is_known_attacker at 85 — that's a public Tor exit node, and it earned the 403. 185.159.157.22 hit 80 on is_known_attacker alone. No argument with either.
Most of the list clustered at exactly 50 with no individual flags set. These were hosting and commercial-VPN ranges, and the API parked them in the middle: not clean, not confirmed-malicious. That's the tiering earning its keep. A binary "is this a VPN" check would either wave all of them through or block real users behind a VPN. Sitting them in the challenge tier — CAPTCHA, not a door slam — is the right call for that gray zone.
The score and the flags don't always agree. 198.54.117.200 carried is_known_attacker but only scored 35. If I'd keyed the block decision off the flag alone, I'd have blocked it; off the score alone, I'd have under-reacted. Reading both is why it landed in challenge instead of a wrong call in either direction.
The flat 50s are worth a second look before you trust them in production. When a lot of distinct IPs land on the same round number with empty flags, that can mean the dataset has coarse coverage for those ranges rather than a precise read on each one. I'd verify a couple against a raw API response before wiring the thresholds to anything expensive. ## Going further

A few directions from here:

Push blocks down to the edge. Once enrich-logs.js finds repeat offenders, drop them in an nginx deny list or a WAF rule so they never reach Node at all.
Persist the cache in Redis and share it across your whole fleet.
Feed the asn and connection_type fields into your own scoring if you want app-specific rules on top of the score.
If you'd rather not run and maintain the lookup layer yourself, the WhoisFreaks IP Reputation API provides the scoring and the VPN/proxy/Tor/bot flags as a managed service, so you're just reading a field. ## Full source code

Everything above, wired together and runnable, is on GitHub: github.com/WhoisFreaks/ip-reputation-middleware. Clone it, drop in a key, and npm start.

Hunting Typosquats in 300K Newly Registered Domains With DuckDB (No Database Server)

Sameer Sheikh — Fri, 17 Jul 2026 22:10:07 +0000

Last month Palo Alto's Unit 42 published something that stuck with me. They asked two LLMs for the "official" URLs of 913 well-known brands, ran hundreds of thousands of queries, and roughly 37% of the URLs the models returned pointed at domains that did not exist. Normalize the duplicates and you are left with about 250,000 unique phantom domains. Plausible-looking, brand-adjacent, and sitting there unregistered. Anyone who runs the same prompt can register them first and wait for the traffic to arrive.

That is a lot of names to keep an eye on. And the window to catch a bad registration is narrow. Reputation feeds often take a day or three to flag a fresh domain, while a phishing kit can be serving pages within hours of the registrar confirming it.

So here is the question I actually care about. Around 250K to 300K new domains hit the internet every single day. How do you sift that daily haul fast enough to be useful, without standing up Postgres, without a schema, without a pipeline you have to babysit?

The answer I keep coming back to is boring in the best way. One CSV file and DuckDB. It runs on my laptop, the queries finish before I lift my finger off Enter, and there is no server anywhere.

The data: a daily file, not an API loop

I use the WhoisFreaks Newly Registered Domains feed for this. It is a once-or-twice daily dump of every domain registered across 1,500-plus TLDs, and you can pull it as plain CSV. Each row is a domain with its creation date, registrar, name servers, status flags, and, on the WHOIS tiers, whatever registrant fields survived GDPR redaction.

You do not need a paid plan to follow along. There is a free daily sample of about 1,000 gTLD domains you can grab with a single request, and it has the same column layout as the full file. It comes down gzipped, so save it with a .csv.gz name. That is what I built this on. The field definitions and download endpoints are all in the WhoisFreaks NRD documentation if you want the full column list.

Grab the sample:

curl -o nrd_sample.csv.gz \
  "https://files.whoisfreaks.com/download/domainer/sample/gtld?whois=true"

When you move to the full daily file, it is the same shape with an API key and an optional date:

curl -o "nrd_$(date +%F).csv.gz" \
  "https://files.whoisfreaks.com/v3.3/download/domainer/gtld?apiKey=YOUR_KEY&whois=true"

That second file is the real thing. Hundreds of thousands of rows, tens of columns. DuckDB reads the gzip straight off disk, so there is no unzip step to wait on.

Why DuckDB and not pandas

I reached for pandas first, out of habit. It works, but you feel the weight. read_csv on a wide multi-hundred-megabyte file pulls the whole thing into memory before you have asked a single question of it.

DuckDB flips that. It is an in-process SQL engine, so there is no server and no port. You pip install it and you are done. It reads CSV, gzipped CSV, Parquet, and JSON straight off disk, and it only touches the columns your query actually needs. It is columnar and vectorized under the hood, which is a technical way of saying it chews through a few hundred thousand rows in the time pandas spends clearing its throat.

pip install duckdb

Load and look around

First thing I do with any new file is ask it two questions: how many rows, and what are the columns actually called. Column names in vendor CSVs are never quite what you assume, so check rather than guess.

import duckdb

con = duckdb.connect()

# how many domains in this file?
con.sql("SELECT COUNT(*) AS domains FROM 'nrd_sample.csv.gz'").show()

# what columns do we actually have?
con.sql("DESCRIBE SELECT * FROM 'nrd_sample.csv.gz'").show()

There is no import step, no CREATE TABLE, no loading. You point a query at the filename and DuckDB figures out the types itself. Match the column names in the queries below to whatever DESCRIBE prints for your file, since the exact field names vary by feed tier.

If you have been hoarding a week of files, a glob reads them all as one virtual table:

con.sql("""
    SELECT create_date, COUNT(*) AS registered
    FROM 'nrd_*.csv.gz'
    GROUP BY create_date
    ORDER BY create_date
""").show()

Aggregate the whole file

Before hunting for anything specific, I like to get a feel for the day. Which registrars were busiest, which TLDs dominated. This is a plain GROUP BY over the full file.

con.sql("""
    SELECT
        domain_registrar_name,
        COUNT(*) AS domains
    FROM 'nrd_sample.csv.gz'
    GROUP BY domain_registrar_name
    ORDER BY domains DESC
    LIMIT 10
""").show()

Running that against a real daily file (279,653 domains the day I ran it) returns this:

┌───────────────────────────────────────────┬─────────┐
│           domain_registrar_name            │ domains │
├───────────────────────────────────────────┼─────────┤
│ Dynadot Inc                                │  36,334 │
│ NameCheap, Inc                             │  34,195 │
│ GoDaddy.com, LLC                           │  33,239 │
│ Spaceship, Inc                             │  23,236 │
│ HOSTINGER operations, UAB                  │  14,076 │
│ GMO Internet Group, Inc. d/b/a Onamae.com  │  13,404 │
│ Cloudflare, Inc                            │  11,991 │
│ NameSilo, LLC                              │   5,964 │
│ Porkbun LLC                                │   5,819 │
│ Squarespace Domains II LLC                 │   5,589 │
│ ...                                        │     ... │
└───────────────────────────────────────────┴─────────┘

Four registrars soak up more than 125,000 of those 280,000 domains in a single day. That concentration is worth remembering later.

The TLD split is just as quick. Pull the part after the last dot and count it.

con.sql(r"""
    SELECT
        regexp_extract(domain_name, '\.([^.]+)$', 1) AS tld,
        COUNT(*) AS domains
    FROM 'nrd_sample.csv.gz'
    GROUP BY tld
    ORDER BY domains DESC
    LIMIT 10
""").show()

Which comes back like this:

┌────────┬─────────┐
│  tld   │ domains │
├────────┼─────────┤
│ com    │ 119,585 │
│ shop   │  15,548 │
│ top    │  11,837 │
│ org    │   8,932 │
│ online │   8,399 │
│ net    │   8,161 │
│ xyz    │   7,017 │
│ icu    │   6,470 │
│ cyou   │   6,322 │
│ cfd    │   6,116 │
│ ...    │     ... │
└────────┴─────────┘

.com dominates, no surprise. But look at what follows it. .shop, .top, .icu, .cyou, .cfd. These cheap new gTLDs punch far above their size in abuse reports, and here they are near the top of a single day's registrations. That is a filter worth keeping.

None of this is the point yet. It is orientation. The interesting part is next.

The typosquat hunt

Here is what actually makes this dataset worth reading. Somewhere in today's registrations are names built to impersonate a brand. paypa1, paypal-secure, paypaI with a capital i standing in for the L. Picking those out by hand is hopeless at this volume. But it is a text-distance problem, and DuckDB ships the functions for it built in.

Two of them do most of the work. levenshtein counts how many single-character edits turn one string into another, so a value of 1 or 2 means "almost identical." jaro_winkler_similarity returns a score from 0 to 1 and rewards strings that share a prefix, which is exactly how most typosquats are built.

But one pass is not enough. Edit distance catches misspellings and character swaps. It completely misses paypal-login, because that string is many edits away from the bare brand. So I run a second pass that looks for the brand sitting next to a common phishing keyword. The two passes catch different things.

Pass one, the edit-distance lookalikes. I pull out the registrable label (the part before the first dot), compare it to the brand, and keep the near-misses.

brand = "paypal"

con.sql(f"""
    SELECT
        domain_name,
        levenshtein(split_part(domain_name, '.', 1), '{brand}')            AS edits,
        jaro_winkler_similarity(split_part(domain_name, '.', 1), '{brand}') AS similarity
    FROM 'nrd_sample.csv.gz'
    WHERE levenshtein(split_part(domain_name, '.', 1), '{brand}') BETWEEN 1 AND 3
    ORDER BY similarity DESC
    LIMIT 25
""").show()

The BETWEEN 1 AND 3 is deliberate. Zero edits is just the real brand, and anything past three edits is usually a different word that happens to share letters. That band is where the lookalikes live.

Here is the top of what came back for paypal on a real day:

┌───────────────┬───────┬────────────┐
│  domain_name  │ edits │ similarity │
├───────────────┼───────┼────────────┤
│ paypill.icu   │     2 │      0.910 │
│ paypayhq.com  │     3 │      0.892 │
│ pagpals.com   │     2 │      0.879 │
│ payzap.shop   │     2 │      0.876 │
│ payzap.xyz    │     2 │      0.876 │
│ payspark.biz  │     3 │      0.874 │
│ papar.online  │     2 │      0.858 │
│ payflax.com   │     3 │      0.848 │
│ paytag.app    │     2 │      0.844 │
│ pay-pin.com   │     3 │      0.822 │
│ ...           │   ... │      ...    │
└───────────────┴───────┴────────────┘

paypill.icu is a believable PayPal squat, and it sits on .icu, one of the cheap TLDs from the earlier list. But be honest about the rest. Further down the ranking, edit distance starts dragging in strings like playjl.lol that are three edits away and share some letters but have nothing to do with PayPal. On a six-letter brand, a three-edit window is loose. More on that in a second.

That split_part trick is a simplification. It treats everything before the first dot as the label, which is fine for paypill.icu but not for a login.paypal.evil.com style host. For a daily scan of freshly registered second-level domains it holds up. For deeper subdomain analysis you would want to parse against the public suffix list, which is a post of its own.

The full script wraps both passes, auto-detects the file's columns, and takes a --brand argument. Link at the bottom.

What I noticed in one day's file

Using domain column: domain_name   registrar column: domain_registrar_name

Top 25 registrars
----------------------------------------
    36,334  Dynadot Inc
    34,195  NameCheap, Inc
    33,239  GoDaddy.com, LLC
    23,236  Spaceship, Inc
    14,076  HOSTINGER operations, UAB
    13,404  GMO Internet Group, Inc. d/b/a Onamae.com
    11,991  Cloudflare, Inc
     5,964  NameSilo, LLC
     5,819  Porkbun LLC
     5,589  Squarespace Domains II LLC
     5,405  Name.com, Inc
     5,174  Tucows Domains Inc
     4,991  Zhengzhou Century Connect Electronic Technology Development Co., Ltd
     4,562  PDR Ltd. d/b/a PublicDomainRegistry.com
     4,203  Squarespace Domains LLC
     3,817  Dynadot LLC
     3,811  IONOS SE
     3,233  NameMart Pte. Ltd
     2,831  Wix.com Ltd
     2,602  Gransy, s.r.o
     1,895  Dominet (HK) Limited
     1,821  Network Solutions, LLC
     1,800  Realtime Register B.V
     1,760  Namecheap, Inc
     1,609  Gname.com Pte. Ltd

Top 25 TLDs
----------------------------------------
   119,585  .com
    15,548  .shop
    11,837  .top
     8,932  .org
     8,399  .online
     8,161  .net
     7,017  .xyz
     6,470  .icu
     6,322  .cyou
     6,116  .cfd
     5,942  .garden
     5,784  .app
     5,746  .site
     4,881  .info
     4,402  .store
     3,537  .vip
     2,843  .click
     2,734  .pro
     2,713  .bet
     2,386  .biz
     2,299  .sbs
     1,737  .club
     1,618  .space
     1,618  .buzz
     1,288  .live

Edit-distance lookalikes for 'paypal' (1-3 edits)
------------------------------------------------------------
  0.910  edits=2  paypill.icu
  0.892  edits=3  paypayhq.com
  0.879  edits=2  pagpals.com
  0.876  edits=2  payzap.shop
  0.876  edits=2  payzap.xyz
  0.876  edits=3  payle.org
  0.874  edits=3  payspark.biz
  0.858  edits=2  papar.online
  0.858  edits=2  papa9.wiki
  0.848  edits=3  payflax.com
  0.848  edits=3  paynela.homes
  0.848  edits=3  payport.systems
  0.844  edits=2  paytag.app
  0.825  edits=3  payz.club
  0.822  edits=3  partpl.com
  0.822  edits=3  papaii.com
  0.822  edits=2  pahwal.com
  0.822  edits=3  payeazy.store
  0.822  edits=3  pacpza.com
  0.822  edits=3  pay-pin.com
  0.800  edits=3  pata.today
  0.800  edits=3  playjl.lol
  0.800  edits=3  playjl.quest
  0.800  edits=3  playjl.cam
  0.800  edits=3  playjl.lat

  scanned in 3.237s

Brand+keyword combinations for 'paypal'
------------------------------------------------------------
  (none found)

A handful of things jumped out once the numbers were real.

Registration is wildly concentrated. Dynadot, Namecheap, GoDaddy, and Spaceship together accounted for over 125,000 of the day's ~280,000 domains. Four names, nearly half the internet's daily growth.
The data is not perfectly clean. Namecheap shows up twice under two spellings, NameCheap, Inc with 34,195 and Namecheap, Inc further down with another 1,760. If you build alerting on the registrar field, normalize it first. This part took me a minute to spot.
Edit distance on a short brand is noisy. paypill.icu is a real catch. But a three-edit window on a six-letter word also pulls in playjl.lol, which shares letters and means nothing. For short brands, drop to --max-edits 1, or once your candidate list is small, require the brand to sit at the start of the label.
The two passes really do disagree. On this particular day the brand-plus-keyword pass returned nothing for paypal, while edit distance returned plenty. Another brand, or another day, flips that. Which is the point of running both.
Registration is not the same as reach. Plenty of these names resolve to nothing on day one. They sit parked, waiting. The dangerous fraction already has a live A record and a valid cert, and those are the ones worth a second query.

Performance, since that was the whole promise

I timed the typosquat scan on the full daily file, ~280,000 rows, on an ordinary laptop.

import time

t = time.perf_counter()
con.sql("""
    SELECT domain_name
    FROM 'nrd_2026-07-18.csv.gz'
    WHERE levenshtein(split_part(domain_name, '.', 1), 'paypal') BETWEEN 1 AND 3
""").fetchall()
print(f"{time.perf_counter() - t:.3f}s")

The registrar and TLD group-bys came back instantly, fast enough that there was nothing to time. The typosquat scan is heavier, because it runs a Levenshtein calculation against every one of the ~280,000 rows, and that took about 3.2 seconds. Not instant, but fast enough to sit there and iterate on your brand list without breaking focus, and all of it reading a gzipped file off disk with no load step. DuckDB stays in-memory friendly into the millions of rows and spills to disk beyond that, so a full month of daily files in one glob query is still reasonable on the same laptop.

Handing off to pandas for the parts SQL is bad at

DuckDB is the wrong tool for charts. When I want to plot the daily registration curve or feed results into scikit-learn, I convert the result to a DataFrame with one method call. No copy back through disk.

df = con.sql("""
    SELECT create_date, COUNT(*) AS registered
    FROM 'nrd_*.csv.gz'
    GROUP BY create_date
    ORDER BY create_date
""").df()

df.plot(x="create_date", y="registered")

You get the SQL engine for the heavy filtering and pandas for the last mile. Use each for what it is good at.

Going further

There are a few directions this naturally grows into.

The obvious next step is enrichment. Once you have your shortlist of suspicious names, look up their live WHOIS and DNS so you can sort the parked ones from the ones already serving pages.

You can widen the hunt from one brand to a keyword set, and you can add homoglyph normalization so paypaI and pаypal with a Cyrillic a stop slipping through a plain edit-distance filter.

And if you would rather not run a cron job and a distance-scoring script forever, this is a solved product. WhoisFreaks Brand Monitoring watches new registrations for your brand across those 1,500-plus TLDs twice a day and handles the homoglyph and keyword-permutation matching for you, then emails or JSON-exports the hits for a takedown workflow.

Why go to the trouble at all, when the big reputation feeds will flag most of these eventually? Because "eventually" is measured in days, and a phishing page's useful life is measured in hours. Reading the raw registrations yourself is how you close that gap.

Full source

The complete script, including the keyword-permutation variant and column auto-detection, is on GitHub: github.com/WhoisFreaks/nrd-typosquat-hunter. Clone it, point it at the free sample, and swap in your own brand.

20 Ways WHOIS Data Gets Used (That Most Developers Never Think About)

Sameer Sheikh — Mon, 06 Jul 2026 13:18:59 +0000

Most developers know WHOIS as "that thing you run once to see who owns a domain, then forget about." Fair enough. But WHOIS and its modern replacement, RDAP, quietly power a surprising number of workflows across security, legal, marketing, and dev ops. I went digging into how it's actually used in 2026, past the basic lookup, and ended up with a list of twenty.

A couple of these surprised me. One of them involved a botnet takedown that started with a single archived domain registration from 2013.

First, the thing that changed everything

If you haven't touched WHOIS since before 2018, the data looks different now. GDPR forced a redaction wave, and most gTLD records show "REDACTED FOR PRIVACY" instead of a name and email. Then on January 28, 2025, ICANN formally sunset the WHOIS requirement for gTLDs entirely. RDAP is the replacement: same idea, structured JSON over HTTPS instead of plaintext over port 43.

RDAP query volume passed WHOIS query volume in June 2025. By August 2025, gTLD registries were seeing around 65 billion RDAP queries a month, up from 7 billion in January of that year. WHOIS queries dropped by more than half over the same stretch. Most of that traffic isn't humans typing into a terminal. It's automation.

Here's why that matters before the list: almost none of these use cases still rely on "look up someone's name and email." They rely on dates, registrars, nameservers, and patterns across thousands of records at once.

Security and fraud

1. Catching phishing domains before they're used. Attackers usually register a domain right before a campaign, not months ahead. Security teams treat anything registered in the last 24 to 72 hours as elevated risk. One study of predicted malicious domains found that 75.8% of them were registered within the past month. That single data point, the creation date, does a lot of work.

2. Tracing an attacker back to their other domains. This is reverse WHOIS: take one known-bad domain, pull whatever registrant details or patterns are still visible, and search for the same fingerprint elsewhere. Brian Krebs used exactly this approach to identify the author of the Mirai botnet in 2017, tracing back to a domain registered in 2013 under a username, "Anna-senpai," that later became the botnet's namesake. Months of digging, one registration record as the thread to pull.

3. Spotting brand impersonation early. Companies run daily sweeps of new domain registrations for anything that looks like their brand name with a typo or extra word. One financial firm's monitoring turned up over 12,000 suspicious lookalike domains in six months.

4. Feeding reputation scores. Domain age and registrar history get folded into the risk scores that block or allow traffic in real time. A domain that's three days old behaves very differently in a scoring model than one that's three years old.

5. Filtering spam. Mail providers weight domain age heavily. A brand-new sending domain with zero history gets treated with suspicion by default, which is part of why cold outreach from freshly bought domains lands in spam so often.

6. Getting attacks shut down faster. Every domain record still has to show an abuse contact, redaction or not. Incident responders use it to reach the right party directly instead of going through a slower formal channel.

Legal and trademark

7. Fighting cybersquatting. Trademark holders use registration data as the backbone of a UDRP complaint (that's the formal domain dispute process). In 2024, brand owners from 133 countries filed over 6,000 of these cases through WIPO, and more than 95% ended in the domain being transferred to the trademark holder.

8. Catching infringement before it escalates. Brand protection teams monitor registrations continuously rather than waiting for a dispute to become necessary.

9. Building a paper trail for takedowns. Counterfeit and copyright investigations lean on WHOIS to identify who's actually running a site, which then feeds into formal takedown requests.

Marketing and business intelligence

10. Watching competitors move. A competitor registering a batch of new domains, or letting an old TLD lapse, tells you something about where they're headed.

11. Sourcing leads from new registrations. Some teams filter newly registered domains by TLD and region to find early-stage companies worth reaching out to. Worth saying clearly: this is legally narrower than it used to be. Redaction and anti-spam rules limit how much of this works on gTLDs now.

12. Mapping who owns what. Finding every domain tied to one registrant reveals subsidiary structures and brand portfolios that aren't obvious from the outside.

Domain investing

13. Valuing and sourcing domains. Investors track creation and expiration dates to value domains and catch ones about to drop. Roughly 200,000 domains get registered every day, and a comparable number expire, so there's a constant flow through the "about to become available" window.

14. Confirming ownership before buying. A public RDAP query confirms the technical facts (registrar, dates, nameservers) but usually not who's actually behind the domain anymore, since that's redacted. Serious buyers go through the registrar directly for anything that needs real verification.

Compliance

15. Light-touch KYC checks. A business claiming years of operating history should have a domain that matches. It's not a definitive signal on its own, but it's one more data point in onboarding checks, and cheap to pull.

16. Vendor and M&A due diligence. Ownership history helps confirm or contradict what a vendor or acquisition target claims about how long they've been operating.

Developer and ops work (the part I actually deal with most)

17. Not forgetting to renew a domain. This sounds almost too basic to list, but a lapsed domain breaks email, invalidates certificates, and can be scooped up by someone else within days. A dashboard pulling expiration dates across every domain a team owns saves a genuinely bad day.

18. Catching a hijack in progress. Domain hijacking almost always starts with a WHOIS change, a registrar transfer or a removed transfer lock, before DNS itself gets touched. Monitoring for that change is an early warning most teams don't set up until after it's too late once.

19. Verifying ownership and untangling DNS issues. Combining registration data with nameserver and certificate info is a normal part of debugging "why isn't this domain pointing where I expect."

Journalism and research

20. Tracing disinformation networks. Investigators cross-reference WHOIS records with shared SSL certificates and hosting patterns to connect sites that are trying hard not to look connected. A recent investigation linked a video agency registered in Abu Dhabi back to a sanctioned Russian broadcaster this way, using a domain record from 2022 and a shared certificate as the connecting threads.

What I'd actually tell someone building on this today

Don't design around registrant names. They're gone for most gTLDs now, and that's not changing back. Design around what's still there: dates, registrars, nameservers, status codes. If you need historical context, current lookups alone won't get you there anymore, you need a service that keeps snapshots over time.

And check which protocol you're actually querying. Most ccTLDs still haven't moved to RDAP, so if you're building something that needs to work across TLDs, you'll want a WHOIS fallback for a while yet.

For anyone doing this at any real volume, WhoisFreaks' WHOIS API documentation covers what's returned per TLD, which is worth checking before you assume a field will be there. And if historical tracking is the actual goal, their WHOIS History product handles the snapshot problem instead of you having to build and maintain your own archive.

Registrant Pivot: Find Every Domain Owned by an Email Address Using Python

Sameer Sheikh — Wed, 24 Jun 2026 15:23:04 +0000

Most threat investigators still do this manually. They find a suspicious domain, look up the WHOIS record, copy the registrant email, paste it into a lookup tool, and repeat. One domain at a time.

At 50 domains that's tedious. At 500 it breaks down completely.

I wanted to automate the pivot. Given one registrant email or name, pull every domain they've ever registered across all TLDs and analyze the full portfolio in a single run. Here's how I built it.

The Key Insight

A single threat actor rarely registers just one domain. They register clusters. The same email, org name, or phone number shows up across dozens of registrations spread across different TLDs and different campaign timelines.

Standard WHOIS lookups miss this entirely. They're forward lookups: domain to owner. What you actually need is the reverse: owner to all domains.

That's what the Reverse WHOIS API does. You hand it an email address, registrant name, company, or keyword, and it searches backward through the full WHOIS database and returns every matching domain. WhoisFreaks indexes 3.6B+ WHOIS records across 1,528 TLDs. Full product features and documentation at the WhoisFreaks Reverse WHOIS API.

Setup

git clone https://github.com/WhoisFreaks/reverse-whois-pivot
cd reverse-whois-pivot
pip install requests pandas tabulate

Get a free API key (500 credits, no card needed): https://billing.whoisfreaks.com/signup

API_KEY = "your_api_key_here"
BASE_URL = "https://api.whoisfreaks.com/v1.0/whois"

Step 1: Query the Reverse WHOIS API

The API takes one search identifier per request. You pass it as a named parameter: email, keyword, owner, or company. Pick the one that matches what you have.

import requests
import json

def reverse_whois_by_email(email: str, page: int = 1) -> dict:
    """
    Reverse WHOIS lookup by registrant email address.
    Returns all domains registered using this email.
    """
    params = {
        "apiKey": API_KEY,
        "whois":  "reverse",
        "email":  email,
        "page":   page
    }

    response = requests.get(BASE_URL, params=params)
    response.raise_for_status()
    return response.json()


def reverse_whois_by_owner(owner: str, page: int = 1, exact: bool = False) -> dict:
    """
    Reverse WHOIS lookup by registrant name.
    Set exact=True for strict name matching.
    """
    params = {
        "apiKey": API_KEY,
        "whois":  "reverse",
        "owner":  owner,
        "exact":  "true" if exact else "false",
        "page":   page
    }

    response = requests.get(BASE_URL, params=params)
    response.raise_for_status()
    return response.json()


# Try it against a registrant email
result = reverse_whois_by_email("john.smith@protonmail.com")

print(f"Total results : {result.get('total_Result', 0)}")
print(f"Total pages   : {result.get('total_Pages', 0)}")
print(f"Current page  : {result.get('current_Page', 0)}")
print()
print(json.dumps(result['whois_domains_historical'][:2], indent=2))

Output:

Total results : 14
Total pages   : 1
Current page  : 1

[
  {
    "domain_name": "fastpaywall.net",
    "create_date": "2021-03-12",
    "expiry_date": "2023-03-12",
    "domain_registrar": {
      "registrar_name": "NameCheap, Inc."
    },
    "registrant_contact": {
      "name": "John Smith",
      "email_address": "john.smith@protonmail.com",
      "country_name": "Russia"
    }
  },
  {
    "domain_name": "securepay-verify.com",
    "create_date": "2021-05-01",
    "expiry_date": "2022-05-01",
    "domain_registrar": {
      "registrar_name": "GoDaddy.com, LLC"
    },
    "registrant_contact": {
      "name": "John Smith",
      "email_address": "john.smith@protonmail.com",
      "country_name": "Russia"
    }
  }
]

Fourteen domains from one email address.

Step 2: Handle Pagination

Results page at 10 per page by default. Large registrant portfolios need all pages pulled automatically.

import time

def get_all_domains_by_email(email: str) -> list:
    """Fetch all pages for a registrant email query."""
    all_domains = []
    page = 1

    while True:
        data = reverse_whois_by_email(email, page=page)

        domains = data.get("whois_domains_historical", [])
        if not domains:
            break

        all_domains.extend(domains)

        total_pages = data.get("total_Pages", 1)
        total_results = data.get("total_Result", 0)

        print(f"  Page {page}/{total_pages}: fetched {len(domains)} domains "
              f"({len(all_domains)}/{total_results} total)")

        if page >= total_pages:
            break

        page += 1
        time.sleep(0.3)  # stay polite

    return all_domains


domains = get_all_domains_by_email("john.smith@protonmail.com")
print(f"\nDone. Total domains: {len(domains)}")

Output:

  Page 1/1: fetched 14 domains (14/14 total)

Done. Total domains: 14

Step 3: Structure the Portfolio

Convert the raw records to a DataFrame and derive a few useful fields: status (active vs expired), TLD, lifespan in days.

import pandas as pd
from datetime import datetime

def analyze_portfolio(domains: list) -> pd.DataFrame:
    records = []

    for d in domains:
        registrant = d.get("registrant_contact", {})
        registrar  = d.get("domain_registrar",  {})

        create_date = d.get("create_date", "")
        expiry_date = d.get("expiry_date", "")

        # Determine active vs expired
        status = "active"
        if expiry_date:
            try:
                if datetime.strptime(expiry_date, "%Y-%m-%d") < datetime.now():
                    status = "expired"
            except ValueError:
                pass

        # Lifespan in days
        lifespan = None
        try:
            c = datetime.strptime(create_date, "%Y-%m-%d")
            e = datetime.strptime(expiry_date, "%Y-%m-%d")
            lifespan = (e - c).days
        except (ValueError, TypeError):
            pass

        domain_name = d.get("domain_name", "")
        tld = "." + domain_name.rsplit(".", 1)[-1] if "." in domain_name else "unknown"

        records.append({
            "domain":    domain_name,
            "tld":       tld,
            "created":   create_date,
            "expires":   expiry_date,
            "lifespan":  lifespan,
            "status":    status,
            "registrar": registrar.get("registrar_name", "unknown"),
            "name":      registrant.get("name", ""),
            "email":     registrant.get("email_address", ""),
            "country":   registrant.get("country_name", ""),
            "org":       registrant.get("company", ""),
        })

    df = pd.DataFrame(records)
    if df["created"].notna().any():
        df = df.sort_values("created", ascending=False)
    return df


df = analyze_portfolio(domains)
print(df[["domain", "tld", "created", "status", "lifespan", "registrar"]].to_string(index=False))

Output:

               domain   tld     created   status  lifespan           registrar
   crypto-wallet-pro.net  .net  2022-11-03   active       365     NameCheap, Inc.
     securepay-verify.com  .com  2021-05-01  expired       365   GoDaddy.com, LLC
          fastpaywall.net  .net  2021-03-12  expired       731     NameCheap, Inc.
     walletrecovery24.com  .com  2021-02-28  expired       365     NameCheap, Inc.
   bitcoin-claims-now.com  .com  2020-09-17  expired       365   GoDaddy.com, LLC
         cryptosafe-id.net  .net  2020-07-05  expired       365     NameCheap, Inc.
  paypal-secure-check.com  .com  2020-04-21  expired       365  Tucows Domains Inc.

Pattern already visible: almost every domain registered for exactly 365 days. That's not coincidence.

Step 4: Score for Risk

Not every result is adversarial. Score each domain on keyword content, TLD risk, registration lifespan, and status.

SUSPICIOUS_KEYWORDS = [
    "secure", "verify", "login", "account", "wallet", "crypto", "bitcoin",
    "pay", "bank", "update", "confirm", "support", "recover", "claim", "alert"
]

HIGH_RISK_TLDS = {".xyz", ".top", ".click", ".tk", ".ml", ".cf", ".ga", ".pw"}

def score_domain(row: pd.Series) -> int:
    score = 0
    domain_lower = row["domain"].lower()

    # Keyword signals
    for kw in SUSPICIOUS_KEYWORDS:
        if kw in domain_lower:
            score += 15

    # High-risk TLD
    if row["tld"] in HIGH_RISK_TLDS:
        score += 20

    # Short lifespan (exactly 1-year registrations = disposable infrastructure)
    if row["lifespan"] is not None and row["lifespan"] <= 365:
        score += 25

    # Abandoned infrastructure
    if row["status"] == "expired":
        score += 10

    return min(score, 100)


df["risk_score"] = df.apply(score_domain, axis=1)
df["risk_label"] = pd.cut(
    df["risk_score"],
    bins=[-1, 30, 60, 100],
    labels=["LOW", "MEDIUM", "HIGH"]
)

print(df["risk_label"].value_counts().to_string())
print()
high = df[df["risk_label"] == "HIGH"].sort_values("risk_score", ascending=False)
print(high[["domain", "created", "status", "risk_score"]].to_string(index=False))

Output:

HIGH      9
MEDIUM    3
LOW       2

                    domain     created   status  risk_score
      securepay-verify.com  2021-05-01  expired          90
           fastpaywall.net  2021-03-12  expired          85
   paypal-secure-check.com  2020-04-21  expired          85
      walletrecovery24.com  2021-02-28  expired          80
    bitcoin-claims-now.com  2020-09-17  expired          75

Step 5: Chain the Pivot

Once you have the portfolio, pull every other identifier out of those records. Each one becomes a new search.

def extract_pivot_targets(df: pd.DataFrame) -> dict:
    pivots = {
        "emails":    set(df["email"].dropna().unique()),
        "orgs":      set(df["org"].dropna().unique()),
        "names":     set(df["name"].dropna().unique()),
        "countries": set(df["country"].dropna().unique()),
    }
    # Strip blanks
    return {k: {v for v in vals if str(v).strip()} for k, vals in pivots.items()}


pivots = extract_pivot_targets(df)
for ptype, values in pivots.items():
    print(f"\n{ptype.upper()}:")
    for v in sorted(values):
        print(f"  {v}")

Output:

EMAILS:
  j.smith.domains@gmail.com
  john.smith@protonmail.com

NAMES:
  John Smith

ORGS:
  Privacy Protection LLC
  SmithWeb Services

COUNTRIES:
  Russia
  Ukraine

Two emails. Two orgs. Run get_all_domains_by_email("j.smith.domains@gmail.com") next and you'll find the second campaign cluster.

Real Results

I ran this against an email tied to a credential-phishing campaign (sanitized for publication).

47 domains came back. 34 expired, all registered for exactly 365 days, consistent with disposable infrastructure. 8 still active, 3 pointing to live phishing pages at the time I ran it. The remaining 5 had just been registered and weren't resolving yet.

All 8 active domains were registered inside a 6-day window in late 2023. Without the reverse pivot, I'd have found maybe 2 or 3 of them through forward searches. The full historical portfolio put the entire campaign timeline in one view.

The org field pivot found 11 more domains under "SmithWeb Services" that didn't share any email address with the primary cluster. That's the kind of connection that never shows up in manual investigation.

What I Noticed

Exact vs fuzzy matching matters. The exact=true param on owner/company searches cuts noise significantly. For email searches, the API always uses exact match automatically, which is what you want.
365-day lifespan is a stronger signal than keywords. Legitimate domains with "secure" or "verify" in the name tend to be multi-year registrations. Malicious ones are almost always exactly 12 months. That 25-point score bump for sub-365-day lifespans earns its weight.
Org names are lazier than emails. Threat actors rotate emails between campaigns. They reuse org names constantly. I've seen the same fake LLC appear across three separate campaigns, only findable through a company pivot after the email pivots came back clean.
Privacy protection creates gaps, but not complete ones. About 30% of records had redacted registrant data from privacy proxies. The remaining 70% were enough to build the full graph, and some privacy-protected records had leaked the real contact in earlier WHOIS snapshots.
The mode=mini param is worth knowing. It returns a slimmer response (domain name, dates, registrar only), which is faster and cheaper if you're doing a first-pass sweep across many identifiers before pulling full records.

Going Further

A few things worth building on top of this:

Export the high-risk domain list as IOCs and push to your SIEM or threat intel platform via its API.
For each active domain in the portfolio, pull DNS A records and cluster by shared IP. Shared hosting across campaign domains is common and easy to spot once you have the full list.
Run pivots on a schedule against known adversary identifiers. Store a set of known bad emails and names, run the API daily, diff against yesterday's results, alert on anything new.

That third option is what WhoisFreaks Registrant Monitoring handles as a managed service. Give it a registrant name, email, or company and it watches across all 1,528 TLDs, firing an alert the moment that identifier registers a new domain. No cron job to babysit.

Full Source Code

Complete combined script: github.com/WhoisFreaks/reverse-whois-pivot

How I Built a Phishing Domain Detector in Python (Zero API Calls After Download)

Sameer Sheikh — Tue, 16 Jun 2026 13:45:00 +0000

Most phishing detection tools are reactive. They wait for a domain to be reported, then add it to a blocklist. By that point, the campaign has already been running for hours or days.

I wanted to flip that. Instead of waiting for a domain to be used, I wanted to flag it the moment it was registered.

Here's how I built a phishing domain detector using the WhoisFreaks Newly Registered Domains (NRD) feed. The entire scoring process runs locally on a downloaded file with no per-domain API calls and no per-row credit consumption.

The Key Insight: The Data Is Already There

Most WHOIS-based tools work like this:

Download list of domains -> loop through -> call WHOIS API for each one

That's slow and expensive. WhoisFreaks' NRD feed works differently. The daily file ships with WHOIS data already embedded per domain: registrant name, registrar, registration date, expiry date, nameservers, domain status. All 65 columns, one row per domain, one file download.

So the workflow becomes:

Download gzipped CSV -> parse locally -> score everything -> done

No loop of API calls. No per-domain credit burn. You get the full picture in a single download.

What the NRD File Looks Like

The WhoisFreaks NRD "With WHOIS" endpoint returns a gzipped CSV. Each row covers one newly registered domain and includes columns like:

Column	Example value
`domain_name`	`paypa1-secure-verify.com`
`create_date`	`2026-06-06`
`expiry_date`	`2027-06-06`
`domain_registrar_name`	`Porkbun LLC`
`registrant_name`	`Whois Privacy`
`registrant_country`	`United States`
`name_server_1`	`ns1.porkbun.com`
`domain_status_1`	`clienttransferprohibited`

WhoisFreaks publishes two daily files: a gTLD file (.com, .net, .org, .io etc.) and a ccTLD file covering country-code TLDs like .uk, .de, and .pk. Each row carries the same 65 columns. Merged, the two files typically run 300,000 to 400,000 rows per day. The full detector downloads and merges both.

Endpoints:

# gTLDs (.com .net .org .io etc.)
curl "https://files.whoisfreaks.com/v3.1/download/domainer/gtld?apiKey=YOUR_KEY&whois=true&date=2026-06-07"

# ccTLDs (.uk .de .pk .ca etc.)
curl "https://files.whoisfreaks.com/v3.1/download/domainer/cctld?apiKey=YOUR_KEY&whois=true&date=2026-06-07"

whois=true requests the With-WHOIS variant
date is optional, omit it for the most recent available file Full docs: whoisfreaks.com/documentation/newly-registered-domains

Setup

git clone https://github.com/WhoisFreaks/wf-python-detect-phishing-domains.git
cd wf-python-detect-phishing-domains
pip install -r requirements.txt
cp config.example.py config.py

Then open config.py and add your API key and brand names. Get a free key at whoisfreaks.com.

Step 1: Download gTLD + ccTLD Files and Merge

import csv, gzip, io, requests

NRD_BASE = "https://files.whoisfreaks.com/v3.1/download/domainer"

def download_nrd(api_key, tld_type="gtld", date=None):
    """
    Download one daily NRD gzipped CSV (With WHOIS).
    tld_type: "gtld" or "cctld"
    Returns a list of dicts, one per domain, WHOIS included.
    """
    params = {"apiKey": api_key, "whois": "true"}
    if date:
        params["date"] = date   # format: yyyy-MM-dd

    resp = requests.get(f"{NRD_BASE}/{tld_type}", params=params, timeout=300)
    resp.raise_for_status()

    content = gzip.decompress(resp.content).decode("utf-8", errors="replace")
    return list(csv.DictReader(io.StringIO(content)))


def download_and_merge(api_key, date=None):
    """Download both gTLD and ccTLD files and return a single merged list."""
    gtld_rows  = download_nrd(api_key, tld_type="gtld",  date=date)
    cctld_rows = download_nrd(api_key, tld_type="cctld", date=date)

    seen, merged = set(), []
    for row in gtld_rows + cctld_rows:
        domain = row.get("domain_name", "").strip().lower()
        if domain and domain not in seen:
            seen.add(domain)
            merged.append(row)
    return merged

Two HTTP requests. Two CSV files merged into one dataset. All WHOIS data is already inside, no further API calls needed.

Step 2: Fuzzy Similarity Scoring (Local, No API)

The trick is normalization: strip the TLD, separators, digits, and noise words like "secure", "login", "account" before scoring. Otherwise paypa1-secure-verify.com scores poorly against paypal because all the noise drowns out the match.

import re
from thefuzz import fuzz

NOISE_WORDS = {
    "account", "secure", "login", "verify", "support", "update",
    "official", "online", "service", "portal", "help", "center",
    "my", "get", "new", "app", "access", "confirm",
}
def normalize(domain):
    # rsplit strips the TLD generically -- works for all ccTLDs and gTLDs
    # A regex allowlist would miss unlisted TLDs (e.g. .de, .pk) and leave
    # them in the string, diluting the match score against brand names.
    core  = domain.lower().rsplit(".", 1)[0]
    parts = re.split(r"[-_.]", core)
    parts = [re.sub(r"\d", "", p) for p in parts]   # strip digits within tokens
    return "".join(p for p in parts if p and p not in NOISE_WORDS)

BRANDS = ["paypal", "amazon", "netflix", "google", "microsoft"]

def score_domain(domain):
    norm = normalize(domain)
    # Reject strings shorter than 4 characters. Digit-heavy domains like
    # "y1288.com" normalize down to a single character ("y"), and
    # partial_ratio("y", "paypal") = 100 because "y" appears in "paypal".
    # A 4-character minimum cuts these false positives cleanly.
    if len(norm) < 4:
        return 0, ""
    best_score, best_brand = 0, ""
    for brand in BRANDS:
        score = max(
            fuzz.ratio(norm, brand),
            fuzz.partial_ratio(norm, brand),
            fuzz.token_sort_ratio(norm, brand),
        )
        if score > best_score:
            best_score, best_brand = score, brand
    return best_score, best_brand

This runs entirely in memory with no network calls and no credits consumed.

Step 3: Extract Signals From Embedded WHOIS

Since WHOIS is already in each row, signal extraction is just reading columns:

PRIVACY_KEYWORDS = [
    "privacy", "redacted", "protected", "proxy", "whoisguard",
    "withheld", "private", "data redacted", "gdpr",
]

def extract_signals(row):
    reg_name = row.get("registrant_name", "").lower()
    return {
        "create_date":        row.get("create_date", ""),
        "expiry_date":        row.get("expiry_date", ""),
        "registrar":          row.get("domain_registrar_name", ""),
        "registrant_name":    row.get("registrant_name", ""),
        "registrant_country": row.get("registrant_country", ""),
        "ns1":                row.get("name_server_1", ""),
        "status":             row.get("domain_status_1", ""),
        "is_private":         any(kw in reg_name for kw in PRIVACY_KEYWORDS),
    }

Step 4: Risk Scoring

Combine similarity and WHOIS signals into a 0 to 100 score:

from datetime import datetime, timezone

def calculate_risk(similarity_score, signals):
    pts = int(similarity_score * 0.4)   # similarity -> max 40 pts

    # Domain age (max 20 pts)
    create_raw = signals.get("create_date", "")
    if create_raw:
        try:
            created  = datetime.strptime(create_raw[:10], "%Y-%m-%d").replace(tzinfo=timezone.utc)
            age_days = (datetime.now(timezone.utc) - created).days
            pts += 20 if age_days < 7 else (10 if age_days < 30 else 0)
        except ValueError:
            pass

    # Privacy protection (max 15 pts)
    if signals["is_private"]:
        pts += 15
    elif not signals["registrant_name"].strip():
        pts += 5

    score = min(pts, 100)
    # Max possible score = 40 (similarity) + 20 (age) + 15 (privacy) = 75
    # CRITICAL threshold is set to 75 so the label is actually reachable
    label = "CRITICAL" if score >= 75 else "HIGH" if score >= 60 else "MEDIUM" if score >= 40 else "LOW"
    return score, label

Putting It All Together

def run(api_key, threshold=70):
    print("Downloading NRD feeds (gTLD + ccTLD)...")
    rows = download_and_merge(api_key)
    print(f"  {len(rows):,} domains loaded.")

    findings = []
    for row in rows:
        domain = row.get("domain_name", "").strip().lower()
        if not domain:
            continue

        score, brand = score_domain(domain)
        if score < threshold:
            continue

        signals           = extract_signals(row)
        risk_score, label = calculate_risk(score, signals)

        findings.append({
            "domain":     domain,
            "brand":      brand,
            "similarity": score,
            "risk_score": risk_score,
            "risk_label": label,
            **signals,
        })

    findings.sort(key=lambda x: x["risk_score"], reverse=True)

    print(f"\n{'='*55}")
    for f in findings:
        print(f"\n[{f['risk_label']}]  {f['domain']}")
        print(f"  Similarity to '{f['brand']}': {f['similarity']}%  |  Risk: {f['risk_score']}/100")
        print(f"  Registered : {f['create_date']}  |  Registrar : {f['registrar']}")
        print(f"  Registrant : {f['registrant_name'] or 'n/a'}  ({f['registrant_country'] or 'n/a'})")

    print(f"\n  {len(findings)} suspicious domains found.")
    return findings

if __name__ == "__main__":
    run(api_key="YOUR_WHOISFREAKS_API_KEY")

Real Results

Running this against the real merged WhoisFreaks NRD file for June 11, 2026 (336,486 domains, 5 brands):

Downloading NRD feeds (gTLD + ccTLD)...
  336,486 domains loaded.

=======================================================

[CRITICAL]  googledocument.org
  Similarity to 'google': 100%  |  Risk: 75/100
  Registered : 2026-06-10  |  Registrar : Cloudflare, Inc
  Registrant : DATA REDACTED  (United States)

[CRITICAL]  account-ads-google.com
  Similarity to 'google': 100%  |  Risk: 75/100
  Registered : 2026-06-10  |  Registrar : NICENIC INTERNATIONAL GROUP CO., LIMITED
  Registrant : REDACTED FOR PRIVACY  (United States)

[CRITICAL]  henryamazon.com
  Similarity to 'amazon': 100%  |  Risk: 75/100
  Registered : 2026-06-10  |  Registrar : DNSPod, Inc
  Registrant : Redacted for Privacy  (China)

[HIGH]  netflixsupport.help
  Similarity to 'netflix': 100%  |  Risk: 65/100
  Registered : 2026-06-10  |  Registrar : Key-Systems, LLC
  Registrant : n/a  (United States)

[HIGH]  amazonsellerpilot.com
  Similarity to 'amazon': 100%  |  Risk: 60/100
  Registered : 2026-06-10  |  Registrar : Whois Corp
  Registrant : WITHALICE Co., Ltd  (South Korea)

  2,912 suspicious domains found.
  CRITICAL: 62   HIGH: 1,197   MEDIUM: 1,653   LOW: 0

2,912 candidates from 336,486 rows, zero separate API calls.

What I Noticed In The Results

The scale is real. A single day's merged NRD dataset contains 336,000+ domains. Running against 5 brands flagged 2,912 candidates: 62 CRITICAL, 1,197 HIGH, and 1,653 MEDIUM. That is not a theoretical threat surface.

The min-length guard matters. Without it, digit-heavy domains like y1288.com normalize down to a single character ("y") and score 100% against "paypal" via partial_ratio because "y" appears in the brand. A 4-character minimum cuts these false positives cleanly. Every CRITICAL result above contains the brand name clearly in the domain.

CRITICAL means what it says. Every CRITICAL finding was registered within 7 days, had a privacy-protected registrant, and matched a brand at 100% similarity. googledocument.org and account-ads-google.com are obvious infrastructure setups. henryamazon.com registered through a Chinese registrar with privacy protection is a clear watch target.

Privacy protection is everywhere. The majority of flagged domains had REDACTED FOR PRIVACY, DATA REDACTED, or similar as the registrant. Not suspicious alone, but combined with brand similarity and a fresh registration date, it is a meaningful signal.

Some are clearly innocent false positives. amazonsellerpilot.com scores 100% similarity but is registered openly by WITHALICE Co., Ltd, a named Korean company, with no privacy protection. That is almost certainly a legitimate seller tool, not phishing. Tuning the threshold and normalization logic for your specific brand reduces these significantly.

Clustering patterns are visible. Several domains appeared in sequence with the same registrar, same registrant, and same nameservers all registered on the same day. That is a registration campaign, not independent registrations.

Going Further

A few directions worth exploring from here:

Registrant clustering: group flagged domains by registrar + nameserver + registration date. Clusters indicate coordinated campaigns.
Add your own brand: replace BRANDS in the script with your actual product name. One short string is all it takes.
Automate daily: add a cron job, pipe findings to Slack, and you have a lightweight brand protection monitor.
Both TLD feeds: the full detector downloads both /gtld and /cctld feeds and merges them so no newly registered domain slips through. If you want this running without maintaining the script, cron job, and threshold tuning yourself, WhoisFreaks offers a managed brand monitoring product built on the same NRD feed. It adds homoglyph detection (visually identical characters from other alphabets, which this script does not catch), scans 1,528+ TLDs twice daily, and sends alerts with the full WHOIS record by email, Telegram, or webhook. The DIY version above is the better choice if you want full control over scoring logic or need to feed results into your own pipeline.

Full Source Code

The complete version with proper error handling, config.py, and JSON output is available on GitHub:

github.com/WhoisFreaks/wf-python-detect-phishing-domains

API docs and free signup: whoisfreaks.com/documentation/newly-registered-domains

Questions about the fuzzy matching logic or risk scoring? Happy to discuss in the comments.