I Pointed My Domain Search Engine at Cybercrime — Here's What It Found

cl0q — Wed, 30 Sep 2026 02:55:51 +0000

I Pointed My Domain Search Engine at Cybercrime — Here's What It Found

I build cl0q, a domain search engine (38.5M domains scanned, free API, no tracking). Last night I got curious: what happens when you run threat-intel queries through a general domain index instead of a specialized feed?

Turns out: quite a lot. Here's what three searches returned.

1. "crypto drainer" → 13 results, including live drainer kits

Top hits included infernodrainer.xyz ("Inferno Drainer — best crypto drainer since 2022," advertising "0-day exploits, 120+ wallet support"), exogator.com ("Liquidate all cryptocurrencies and NFTs from any wallet"), purpledrainer.club, and cryptodrainerscript.net selling "multichain crypto drainer scripts."

Each result came with resolved IPs and MX records — immediate pivot points. exogator.com sits on 5.182.209.88; infernodrainer.xyz on 216.198.79.65. That's infrastructure you can feed straight into a blocklist.

2. "stealer log" → a notorious carding market, indexed

russianmarket.work showed up: "The most trusted underground marketplace. Buy CVV, Stealer Logs, RDP, dumps..." — sitting in a general web index, with its IPs (172.67.149.100) and metadata attached. Alongside it: breach-intel platforms, stealer-log search services, and a Turkish hacking forum (spyhackerz.org) dealing in RATs and fresh leaks.

3. "phishing kit" → an active scampage shop

lurez.online — "Spamming Tools, Office 365 Scampages, Bybit & Binance Scampages For Sale," complete with SMTP servers, cracking utilities, and a "complete spamming course." Plus spams.tools ("Spam Tools | Scampages | Checkers | Letters | Email Senders | SMTPs | cPanels") with its MX records exposed.

Why this matters

Threat intel usually means expensive feeds and APIs. But a huge amount of attacker infrastructure lives on the clear web, on cheap domains, behind Cloudflare — indexed like anything else. A domain-level search engine with DNS enrichment (IPs, MX, CNAMEs) turns a hunch into pivotable infrastructure in one query.

The searches are free, the API is free (30 req/min, 1,000/day), and nothing is logged. Try it: cl0q.com

All domains above were live in the index at time of writing. If you're a defender, the IPs and MX records are yours to use.

AI assistance was used in drafting this article.

38.5 million domains on a 2 GB droplet: the honest story of building an open search engine

cl0q — Sun, 27 Sep 2026 21:25:38 +0000

Disclosure: I used AI assistance to help draft this post. The story, the mistakes, and every number below are mine.

In 2025, I started crawling the web.

Not metaphorically. I pointed a crawler at the open internet from a 2 GB VPS and decided to build a search engine — a free, open-source one that doesn't log your searches, doesn't need an account, and that anyone can self-host. The first crawl records date to April 2025. Then things went quiet for a while. Then this June, cl0q took off: 8.5 million domains in a single month.

Today, cl0q knows 38.5 million domains and has indexed 24.8 million pages. The goal is 100 million.

This is the honest version of how that went.

The first million nearly killed the project

The naive approach to crawling is: start with a list of sites, follow every link, repeat. What nobody tells you is that the web has traps. Point a crawler at the open web and it finds Blogspot — about a million subdomains hanging off a single domain — and happily spends a week indexing them while the rest of the internet waits.

Getting to the first million was the hardest part, because I didn't know the right crawl path. Crawl too narrow and you re-walk the same neighborhood forever. Crawl too wide and you fall down a subdomain hole and your index fills up with a million pages from the same blog host.

The fix was treating the known web as a rotating frontier instead of a finished list: keep re-sampling domains you already know and mine their homepages for fresh outbound links. And Certificate Transparency logs turned out to be a goldmine — at peak, they were surfacing around 1,700 new domains a minute. Discovery never ends; it just keeps moving.

Everything broke, mostly because of disk

For most of this project, cl0q ran on that 2 GB VPS. A web crawl is, at its core, a fight against disk space. Every page is a row, every favicon is bytes, every DNS record is overhead, and the crawl never stops growing.

Lots broke. Things crashed, crawls stalled, I ran out of room at the worst moments. These days the storage lives on a remote host connected over Tailscale, which helped enormously — and I'm still fighting storage issues today. If there's one thing I'd tell anyone starting a crawl project: budget ten times the disk you think you need, then double it.

The database has one core, and everything fights over it

Here's something the docs won't tell you: search and the crawler share a single database, a single core, and the same table. When a burst of background jobs stampedes the database, search times out. When the crawler picks its next targets, it eats a big share of DB time doing it.

There's no elegant fix yet — just triage, scheduling, and the occasional 2 a.m. realization that the thing slowing down search is my own crawler. The one part that never touches the database: the stats and drill-down pages, which are served from precomputed files. If the site feels fast while the crawler is hammering away, that's why.

What actually worked

A few decisions paid off:

The pipeline is four small stages — discovery, page crawl, enrichment, serving — each incremental and restartable. Nothing depends on a full rebuild. (That's the simplified version. The real version involves the single-core database above.)
Refusing to index garbage. When a site serves an anti-bot challenge page, lazy crawlers index the text "Checking your browser…" as if it were content. cl0q detects interstitials and throws them out. A search engine is only as good as what it refuses to index.
Favicons turned out to be a superpower. Every icon gets hashed with SHA-256: 16.3 million sites in the index have a favicon, collapsing down to about 7 million distinct icons. Sites sharing an icon are usually related — same operator, same framework, same phishing kit. What started as a storage deduplication trick became one of the most useful OSINT pivots in the index.
Privacy as architecture, not policy. Signed out, cl0q sets no cookies and stores nothing about your searches — there's no per-request log to leak, subpoena, or sell. Accounts are optional and passkey-based, and search history exists only if you turn it on. The code is open source, though I'll be honest: the public mirror is a couple months behind right now, and refreshing it is on my list.

Where things stand

The numbers, measured from the actual database rather than the dashboard: cl0q knows 38.5 million domains — 97% have been attempted, 92% resolve in DNS. It's indexed 24.8 million pages, 24.4 million of them live, across 1,027 TLDs. About 250,000 to 300,000 pages get crawled every day, and re-crawls start October 9 so pages stay fresh instead of going stale.

The road to 100 million

The plan is simple: hit 100 million domains, then open up the API properly. There's a free beta API today and paid tiers are coming — and once the project makes money, the next frontier is depth. Right now cl0q fetches homepages; with real revenue behind it, the crawl goes deeper into sites instead of just wider across them.

If you want to help, you can run a crawler and contribute pages — that's by invitation today, so get in touch — or just try a search at cl0q.com.

A year and a half of crawling, 38.5 million domains, one very tired VPS. On to 100.

DEV Community: cl0q

I Pointed My Domain Search Engine at Cybercrime — Here's What It Found

I Pointed My Domain Search Engine at Cybercrime — Here's What It Found

1. "crypto drainer" → 13 results, including live drainer kits

2. "stealer log" → a notorious carding market, indexed

3. "phishing kit" → an active scampage shop

Why this matters

38.5 million domains on a 2 GB droplet: the honest story of building an open search engine

The first million nearly killed the project

Everything broke, mostly because of disk

The database has one core, and everything fights over it

What actually worked

Where things stand

The road to 100 million