I measured how many websites actually publish a contact address

Mehmet — Mon, 31 Aug 2026 12:41:40 +0000

Every contact scraper on the market says roughly the same thing: it finds contact details on "almost any webpage." I went through six of them last week looking for a number — any number — and found exactly one. A tool called Website Email Extractor mentions "60-80% email hit rate" for business websites, buried in a tips section. Everyone else just says "almost any."

So I measured mine and I'm publishing the number, including the part where I got it badly wrong.

The short version
Two samples, both scraped from a datacenter IP with no proxy, no browser, no login.

On 100 real business websites — restaurants, law firms, dental practices, accountants, estate agents — pulled from OpenStreetMap entries that list a website:

84% of them responded at all
Of those, 82% gave up an email address or a phone number
Of those, 71% gave up an email specifically
On 200 domains sampled systematically from the Tranco top-1M, which is a very different animal:

64% responded
Of those, 64% had contact details
41% of the whole list
That gap between 82% and 64% is the entire story, and I spent two days not understanding it.

The part where I was an idiot
My first measurement came in at 28%. That felt low. I assumed the scraper was bad, so I started fixing the scraper.

I got it to 41% — I'll go through what worked in a minute — and it still felt low. Then I looked at what was actually in my sample. Parked domains. CDN hostnames. Seven gambling sites, five porn sites, four file-sharing sites. A domain that returned a 200 with an empty body.

A random slice of the web is not a lead list. Nobody's CRM export looks like that. The competitor who published a number was careful to say "business websites," and I'd been benchmarking against a completely different population.

So I built a second sample from OpenStreetMap — every node in central London and Berlin tagged as a restaurant, lawyer, dentist, accountant, estate agent or ad agency that also has a website tag. Real businesses that publish a website. 84% responded. 82% had contact details.

The scraper was mostly fine. The sample was the problem.

The part where I published a wrong number
This one is worse and it's the reason I'm writing this post.

For each site the scraper couldn't reach, I wanted to know whether it was dead or just blocking me. Easy enough: retry from a second network. I retried from my own machine, compared the two, and concluded that 64% of unreachable domains were genuinely gone.

I put that in the docs. I used it to compute a "live sites" denominator, which made my coverage look better.

Here's what I hadn't accounted for: my second vantage point resolved DNS through a filtering resolver. When a resolver refuses to answer for a domain, the failure is indistinguishable from the domain not existing. Same timeout, same empty result. Plenty of setups do this — a Pi-hole, a corporate resolver with a blocklist, an ISP with malware filtering switched on. Mine was one of them, and I'd never had a reason to notice.

I re-ran the check against Cloudflare's DNS-over-HTTPS resolver, which bypasses the local resolver entirely:

Domain resolves, host is alive 66%
Registered, no web server at all 29%
Actually gone 5%
Not 64% dead. 5% dead. Two-thirds of what I'd written off as dead domains were alive and simply not serving me.

I'd inflated my own number and hadn't noticed, because the error pointed in the flattering direction — the errors that survive are usually the ones you like.

The general lesson: a machine cannot tell you whether a domain is dead if its resolver is allowed to lie. If you're classifying unreachable hosts, do the check over DoH against a resolver you chose deliberately, not whatever your network hands you.

What actually moved the number
I tested each change separately against the same 200 sites. Baseline: 56 sites with contact details.

Trusting the site's own mailto: links (56 → 60). This was the biggest single bug and it was inherited. The email-extraction code came from a different tool of mine that scrapes YouTube channels, where the page you're looking at belongs to a third party — so an address on a different domain is suspicious and gets filtered.

That rule is exactly wrong here. If I hand the scraper vincheck.by and the page has , that is the contact address. The site is declaring it. My filter was throwing it away because the domain didn't match. Same for megatv.com → megatv@alteregomedia.org (a Greek media group's corporate domain) and abcactionnews.com → webteam@wfts.com (the station's parent company).

Structural markup — mailto:, JSON-LD — now gets accepted regardless of domain. Free text still goes through the strict filter, because an address in body copy really might belong to a commenter.

Following the site's own navigation (56 → 62). I was trying a fixed list of paths: /contact, /contact-us, /about, /impressum. The leader in this category doesn't do that — I noticed its output includes scrapedUrls: ["https://apify.com/contact-sales"], which no fixed list would ever contain. It reads the page's links.

Once I did the same, 19 of 67 sites with an email published it on a path no list would have guessed: /pages/contact, /outreach, /company/contact, /en/contacts, /information/contact-us/, and — this one is embarrassing — /contact/ with a trailing slash. My list had /contact.

Decoding Cloudflare's email protection (email hits 37 → 47). If a site is behind Cloudflare with email obfuscation on, the address is XOR-encoded in a data-cfemail attribute and the page shows [email protected]. First byte is the key, XOR the rest. Ten lines of code, biggest single gain in email hits of anything I tried. There's also a second form where the payload sits in the href hash of a /cdn-cgi/l/email-protection link — I found a site using both at once.

Trying www. and http (reachable 113 → 126). One site in ten was being written off purely over a spelling difference.

Not counting 404s against the page budget (77 → 81). I capped each site at N requests. Turns out most sites were burning that cap on 404s — one site spent seven of eight requests on paths that didn't exist and read exactly one real page before I declared the budget spent. A 404 has no body. It costs almost nothing.

I switched the budget to count pages that actually loaded, with a separate ceiling on total requests. Requests went up 96%. Cost went up 13%. That gap is the whole argument.

What didn't work
I think this list is more useful than the one above, and nobody ever publishes it.

Adding more fixed paths — zero. I tried 27 more: language variants (/contacto, /contatti, /iletisim), trailing-slash versions, /contact.php, /get-in-touch. Across 34 sites and 986 requests: nothing. I tested this twice, under two different budget rules, because the first test wasn't fair — the old budget rule meant extra paths ate the request cap. Still zero.

Reading sitemap.xml to find the contact page — zero. Sound idea. Of 42 sites, only 3 had a contact URL in their sitemap, only 1 had one that wasn't already linked from the homepage, and that page had no address on it.

Retrying blocked sites with realistic browser headers — zero. Recovered 3 sites in reachability and zero in contact details. The blocks aren't header-based.

Datacenter proxy — basically zero. I'd assumed a proxy would fix the ~26% of unreachable sites that were reachable from an ordinary connection. Same 200 sites, same build, only the proxy flag changed: it recovered 2. Of course it did. Apify's datacenter proxy pool is datacenter IP space, and the WAF rules that block me are matching on IP range. I had this written in the docs as a confident prediction before I bothered to test it.

Loosening the free-text filter — zero. Two extra addresses across 200 sites, no extra sites, and a real noise risk. The strict gate earns its keep.

Some numbers on what's actually out there
Falling out of all this, on the 200-site random sample:

5 sites disallowed crawling in robots.txt. On the 100 business sites: zero.
7 sites had a contact form and no address anywhere. Nothing to extract; I now report the form's URL instead of an empty record.
Of the phone numbers found, 79% came from tel: links rather than body text — structural markup is much more reliable than digit-hunting in prose, which is full of dates, prices and SKUs.
Two sites published a phone number only inside a wa.me WhatsApp link.
One measurement note: I ran the 200-site sample four times while working on this. Three consecutive builds scored 76, 77 and 79 sites. That spread is run-to-run noise, not progress. If you're benchmarking a scraper against the live web, run it more than once before you believe a two-point improvement.

The thing I still can't answer
I don't know how any of this compares to the competition, and I tried to find out.

Every tool in this category has its source hidden. None publishes a hit rate. There's no public review API. The only way to get a real comparison is to run their actor on my sample and diff the output — and my Apify plan doesn't allow running other people's public actors, so that experiment is blocked for now.

So: 82% of responding business websites, 71% for email specifically. That's my number. I have no idea whether it's good relative to the field, and I'd rather say that than imply I'm ahead.

The tool is Website Contact Scraper on Apify. HTTP only, no browser, and every address in the output says whether it came from structural markup or body text — plus, where relevant, that it wasn't found on the site at all and is a naming-convention guess. All the measurements above are in its README, which is the closest thing I have to a warranty.

DEV Community: Mehmet

I measured how many websites actually publish a contact address