Building a Manifest V3 Lead Extractor: Architecture of the Lead Extraction Engine

#chromeextension #javascript #opensource #webscraping

Lead Extraction Engine is an open-source Chrome extension (Manifest V3, v1.0.0) that pulls B2B/B2C leads from ~20 directory sites — Google Maps, LinkedIn, Justdial, Yelp, Zillow, IndiaMART — plus a generic extractor for any directory-style page. Leads land in IndexedDB, export to CSV/JSON. I read the source. Here's the architecture.

Manifest: minimum viable permissions

{
  "manifest_version": 3,
  "permissions": ["storage", "scripting", "activeTab", "notifications"],
  "host_permissions": ["<all_urls>"],
  "background": { "service_worker": "background/service-worker.js", "type": "module" }
}

<all_urls> is the honest-but-aggressive choice — the README justifies it: the generic extractor's whole job is reading contact info on arbitrary directory pages. unlimitedStorage was deliberately removed. No remote code, no eval, no external scripts — important for Web Store review.

The extractor pattern

content/injector.js runs on every page, detects the platform, and dispatches to one of ~20 extractor classes in content/extractors/. Each class follows the same interface: detect → parse → dedupe → message. The LinkedIn extractor shows the pattern well:

class LinkedInExtractor {
  constructor() {
    this.extractedHashes = new Set();
    this.isAutoScrolling = false;
  }
  async extractLeads() {
    const searchCards = document.querySelectorAll(
      '.reusable-search__result-container, .entity-result, li.reusable-search__result-container'
    );
    for (const card of searchCards) {
      const lead = this.parseSearchCard(card);
      if (lead && !this.extractedHashes.has(lead.hash)) { /* push, highlight */ }
    }
    // ... Sales Navigator cards, company pages ...
    chrome.runtime.sendMessage({ type: 'SAVE_LEADS', leads });
  }
}

Parsing is site-specific DOM work (.entity-result__title-text, [data-anonymize="person-name"] for Sales Navigator), and dedup is a content hash — btoa of name+title+URL truncated to 32 chars — so re-scrolling never double-counts. Extracted cards get highlighted in-page.

The clever bit is the generic extractor: no site knowledge, just directory-shaped patterns (repeated cards with names + phone-like strings + mailto: links). Recall-first by design; the popup shows captures before export.

Pipeline: worker → storage → export

background/service-worker.js routes SAVE_LEADS messages and validates records. common/db.js persists to IndexedDB (no cloud, no account). common/intelligence.js handles phone/email parsing — emails come from visible mailto:/text only; guessing is heuristic, verification explicitly not implemented. The popup (popup.js, ES module) drives sessions: auto-detect → Start (auto-scrolls, keeps extracting) → Stop → CSV/JSON export.

Licensing as code

The monetization is the most carefully designed part. Free = 50 leads, fully offline. Paid (Starter 500 / Pro 2,000 / Agency 10,000) validates server-side via backend/server.js (Express, POST /validate behind X-API-Key):

The extension never grants paid plans from built-in keys.
Activation fails closed — no license server configured → paid activation just doesn't work; free tier unaffected.
The server refuses to boot without LEE_API_KEY; .env.example ships dummies only.
npm test runs a smoke test: JS syntax + manifest validation + hardcoded-secret scan.

Run it

Load src/ unpacked at chrome://extensions (Developer mode) — no build step. npm run package zips src/ into the Web Store artifact. PRIVACY_POLICY.md and permission justifications ship in-repo, ready for listing submission.

Status: working v1.0.0, not yet on the Web Store. If you build browser extensions, the extractor-class pattern + fail-closed licensing is worth stealing.

DEV Community