I open sourced a small tool recently for flagging suspicious domains, and a few people asked how it actually works under the hood rather than just what it does. So here's the technical side: the design decisions, the tradeoffs, and why I built it this way instead of a few obvious alternatives.
Repo is here if you want to skip ahead: https://github.com/Furqan-Ashraf/Typosquat-detector
The problem with checking domains one at a time
If you've ever tried to script anything around WHOIS data, you've probably hit the same wall I did. Raw WHOIS responses aren't structured. They're plain text, and the format changes depending on the registrar and the TLD. Some responses have a Creation Date field. Others have created. Others bury it three lines deep in a block of legal text you have to skip past first.
Writing a parser that handles even a handful of TLDs reliably takes real effort, and it breaks the moment you hit a registry you didn't test against. This is the main reason I used a WHOIS API (WhoisFreaks, specifically) instead of building a raw WHOIS parser from scratch. It returns normalized JSON, so the code doesn't care whether it's looking at a .com, a .io, or a .dev domain. Same fields, same structure, every time.
Two independent checks, combined
The core logic is intentionally simple: two checks, each cheap to compute, combined into one risk flag.
Check one: registration age. This one's a straightforward date diff once you have the creation date:
def parse_age_days(create_date_str):
for fmt in ("%Y-%m-%d", "%Y-%m-%dT%H:%M:%SZ", "%Y-%m-%d %H:%M:%S"):
try:
created = datetime.strptime(create_date_str, fmt).replace(tzinfo=timezone.utc)
return (datetime.now(timezone.utc) - created).days
except ValueError:
continue
return None
Multiple date formats are handled because different registries return dates differently even through a normalized API. Better to try a few formats than crash on the one registry that's slightly different.
Check two: string similarity against a watchlist. This is Levenshtein distance, implemented without any external dependency since it's a small enough algorithm not to warrant pulling in a library:
def levenshtein(a, b):
if a == b: return 0
if not a: return len(b)
if not b: return len(a)
prev = list(range(len(b) + 1))
for i, ca in enumerate(a):
cur = [i + 1]
for j, cb in enumerate(b):
cur.append(min(prev[j+1]+1, cur[j]+1, prev[j]+(ca != cb)))
prev = cur
return prev[-1]
This is the classic dynamic programming approach, O(n*m) where n and m are the string lengths. For domain names, which are short strings, this runs instantly even across a large watchlist.
One bug worth mentioning, since it's the kind of thing that only shows up once you test against real-world typosquat patterns: comparing the full base string against the watch domain misses cases where the attacker adds extra text, like paypa1-secure.com against paypal.com. The full-string edit distance there is large enough to fall outside any sane threshold, even though it's an obvious typosquat to a human. Fixed it by also sliding a same-length window across the candidate string and taking whichever comparison, full string or windowed, gives the smaller distance:
def closest_watch_match(domain, watch_domains):
base = strip_tld(domain)
best_domain, best_distance = None, None
for watch in watch_domains:
watch_base = strip_tld(watch)
full_dist = levenshtein(base, watch_base)
window_dist = full_dist
w = len(watch_base)
if len(base) > w:
window_dist = min(
levenshtein(base[i:i + w], watch_base)
for i in range(len(base) - w + 1)
)
dist = min(full_dist, window_dist)
if best_distance is None or dist < best_distance:
best_domain, best_distance = watch, dist
return best_domain, best_distance
Small fix, but it's the difference between catching paypa1-secure.com and missing it entirely, which matters a lot more than the character-swap case since real phishing domains almost always pad the name with something like -secure, -support, or -login.
Why combine them instead of scoring separately
An earlier version of this scored age and similarity separately and ranked domains by a weighted total. I dropped that pretty quickly. Weighted scoring sounds more sophisticated, but in practice it just makes the threshold harder to reason about, and harder to explain to anyone else looking at the flagged list. A boolean AND, new domain and close lookalike, both true, is easy to tune (two thresholds, both adjustable as plain variables at the top of the script) and easy to explain to someone reviewing the output.
result.risk_flag = result.is_new and result.is_lookalike
That's the entire decision. Everything before it in the script is just gathering the two inputs.
Handling the API layer
A few things mattered here beyond just making the request:
Rate limiting courtesy. There's a small sleep between requests (REQUEST_DELAY_SECONDS = 0.3). Nothing fancy, just enough to avoid hammering the API on a large batch.
Graceful degradation. If a lookup fails for one domain (timeout, malformed response, whatever), that shouldn't kill the whole batch. Errors get caught, logged into the result object, and the script moves on to the next domain:
try:
data = query_whoisfreaks(domain, api_key)
# ...
except requests.exceptions.RequestException as e:
result.error = f"Request failed: {e}"
Dataclasses for structured results. Each domain check produces a DomainResult dataclass rather than a loose dictionary. Makes the CSV export trivial (asdict() handles the conversion) and keeps the fields self-documenting.
What I'd add next
The repo's open for contributions, and a few extensions are on my list:
DNS and nameserver cross-referencing would add a third signal, since a lot of phishing infrastructure reuses the same nameservers across multiple campaigns even when the domain names differ. SSL certificate issuance date is another one worth adding, mainly because a domain with no cert, or one issued the same day as registration, tends to correlate with the same short-lived infrastructure pattern.
Async requests would help too. Right now it's sequential with a delay, which is fine for a few hundred domains but slow if you're trying to batch-check thousands.
Where to get it
The full script, README, and usage instructions are on GitHub: https://github.com/Furqan-Ashraf/Typosquat-detector. If you're running something similar, or you've layered in other signals (SSL, DNS, ASN data) that work well for this kind of detection, I'd be curious to hear about it.
If you want the fuller story behind why I built this in the first place, I wrote that up separately here: I Built a Script to Catch Phishing Domains Before They Hit My Inbox
Top comments (0)