If you are building a pipeline that crawls a company's public website and pulls contact and business signals out of it, the first question is how far that actually gets you. Here are the numbers from a benchmark pass the Mantis team ran on 2,250 sites, plus two rules the pipeline follows.
The headline numbers
On 2,250 sites the pipeline reached:
- Fetch 87%: the homepage HTML came back.
- Email 73%: at least one usable email on the fetched pages.
- Company LinkedIn 54%: a company page URL, not a person.
- Phone ~52%: at least one phone number.
On classification, the business-model guess was right 82% of the time, and 96% of the quotes pulled into the site profile were actually supported by the fetched text.
The 13% that did not fetch is the floor. In general, sites fail to fetch because of dead domains, bot blocks, redirect chains or JavaScript-only landing pages. Smarter parsing does not fix that layer: you either add a headless browser (slower and more expensive per domain) or you accept the floor.
Throughput on a real box
On the production machine, the bulk enricher ran roughly 400 domains per 30–50 seconds, about 35–50k domains per hour, with box load under 1.1. About 25% of domains came back as site_down, and of the sites that did read, about 50% yielded an email.
Those production figures are not a controlled benchmark, but they point the same way as the 2,250-site pass: a real share of any domain list is simply unreachable, and email is the signal you can count on most often.
Rule 1: keep a do-not-enrich list
One rule in the pipeline is explicit: do not enrich the scrape_domains registry in general, because it is mostly agencies who sell the same services. If your input pool is full of companies that all describe themselves the same way, your classifier output will look the same too, and you pay crawl cost for rows that do not help anyone.
In code, that is a cheap gate in front of the expensive step. A minimal sketch (illustrative, not our production code):
DO_NOT_ENRICH_SOURCES = {"scrape_domains"}
def should_enrich(row: dict) -> bool:
# Skip whole source pools you have decided are noise,
# before you spend a fetch on them.
if row.get("first_source") in DO_NOT_ENRICH_SOURCES:
return False
return bool(row.get("domain"))
Rule 2: never trust a site-extracted person alone
Pulling a named person off a website is where naive pipelines go wrong. The example in our notes: a YC company page mentions Jensen Huang, and a simple extractor will happily return "Jensen Huang" as the contact for that company. The rule: never trust a site-extracted person alone.
In practice:
- Treat a site-pulled name as a candidate, not a result.
- Cross-check it against a second source (company LinkedIn, a press mention, a GitHub profile) before it goes into a CRM row.
- If you cannot cross-check, return a company-level contact (the generic inbox or contact form) and leave the last step to a human.
def accept_person(name: str, sources: set[str]) -> bool:
# "site" alone is never enough; require one independent source.
independent = sources - {"site"}
return bool(name) and len(independent) >= 1
Classification held up
Business-model classification at 82% right and quote grounding at 96% supported matter more than they look. Even when phone or named-person extraction fails, the pipeline can still return a segment tag and a short, grounded "what they do" line. That is enough to route a lead to the right template, which is usually worth more than a messy phone number.
What these numbers do not tell you
Read them with these limits in mind:
- They say nothing about how much a headless browser would recover from the fetch floor.
- The 73% email figure is reported as one number; it does not tell you how many are role addresses (info@, sales@) versus personal ones.
- The 54% company LinkedIn figure counts a found company URL; it does not tell you whether that page is still active.
- The ~50% email yield and ~25%
site_downrate come from the production box, not a controlled benchmark.
If you are building something similar
- Instrument the metric that routes decisions. Fetch rate is a system-health signal; "business model right" is the one that decides what happens to a lead.
- Keep a strict do-not-enrich list for source pools that only add noise.
- Never ship a single-source named person.
The next pass is Phase 2: 182,041 Apollo companies loaded as a campaign called mantis-enrich with first_source set to apollo. We plan to share how its numbers compare with this one.
I work on Mantis (mantisai.in), an AI tool for finding and enriching B2B leads. New accounts get 500 free credits once at signup, no card.

Top comments (0)