DEV Community: Tùng Nguyễn

Automating Excel Reporting & Data Cleaning: 10 Essential VBA Macros + Python Pipeline

Tùng Nguyễn — Sat, 31 Oct 2026 06:37:11 +0000

Even in an era of cloud data warehouses and Python notebooks, Microsoft Excel remains the final delivery format for 90% of finance, operations, e-commerce, and sales teams.

When your team spends hours every week manually deduplicating rows, trimming invisible non-breaking spaces (CHAR(160)), merging multi-tab CSV exports, and exporting filtered PDF reports, combining Python (openpyxl / pandas) with 1-Click Excel VBA Macros (.bas) turns a 3-hour Friday chore into a 5-second workflow.

⚡ Ready-to-Use Excel VBA & Python Automation Packs

📊 Ultimate Excel VBA Automation Pack — 10 One-Click Business Macros ($9.99):
- Payhip (PayPal / Card Instant Download): payhip.com/b/239q5
- Gumroad: tungnguyen84.gumroad.com/l/wybjyn
🧲 Python B2B Lead Scraper + Excel VBA Automation Suite Pro ($14.99):
- Payhip: payhip.com/b/Vm3DR | Gumroad: tungnguyen84.gumroad.com/l/pinmv
🛍️ Shopify Competitor Price Spy Pro + Excel Dashboard Exporter ($12.99):
- Payhip: payhip.com/b/P9mGA | Gumroad: tungnguyen84.gumroad.com/l/lxqssu
🤝 Custom Excel VBA, Google Sheets & Python Automation: Hire me on Fiverr (@tungnguyen84)

🛠️ 3 Production-Grade VBA Macros Every Analyst Should Have

1. Deep-Clean Whitespace, Non-Breaking Spaces (`CHAR(160)`) & Case Normalization

Standard WorksheetFunction.Trim in Excel does not remove HTML non-breaking spaces (Chr(160)) copied from web pages or CRM exports, which silently breaks VLOOKUP and XLOOKUP.

Sub CleanSelectedRangeForXLOOKUP()
    Dim rng As Range, cell As Range
    On Error Resume Next
    Set rng = Intersect(Selection, ActiveSheet.UsedRange)
    On Error GoTo 0
    If rng Is Nothing Then Exit Sub

    Application.ScreenUpdating = False
    Application.Calculation = xlCalculationManual

    For Each cell In rng.Cells
        If Not cell.HasFormula And Not IsEmpty(cell.Value) Then
            Dim cleanVal As String
            cleanVal = CStr(cell.Value)
            cleanVal = Replace(cleanVal, Chr(160), " ")
            cleanVal = Replace(cleanVal, vbCr, " ")
            cleanVal = Replace(cleanVal, vbLf, " ")
            cleanVal = Application.WorksheetFunction.Trim(cleanVal)
            cell.Value = cleanVal
        End If
    Next cell

    Application.Calculation = xlCalculationAutomatic
    Application.ScreenUpdating = True
    MsgBox "Cleaned " & rng.Cells.Count & " cells!", vbInformation, "Clean Complete"
End Sub

2. Merge All CSV / XLSX Files in a Folder into One Master Sheet

Instead of opening 30 daily sales exports one by one, this macro loops through a chosen folder, copies the data rows below the header from each file, and tags every row with the source filename:

Sub MergeFolderCSVsIntoMaster()
    Dim folderPath As String, fileName As String
    Dim wbSource As Workbook, wsMaster As Worksheet
    Dim lastRow As Long, srcRows As Long

    With Application.FileDialog(msoFileDialogFolderPicker)
        .Title = "Select Folder Containing CSV/XLSX Reports"
        If .Show <> -1 Then Exit Sub
        folderPath = .SelectedItems(1) & Application.PathSeparator
    End With

    Application.ScreenUpdating = False
    Set wsMaster = ThisWorkbook.Sheets.Add
    wsMaster.Name = "Merged_" & Format(Now, "hhmmss")

    fileName = Dir(folderPath & "*.csv")
    Dim isFirst As Boolean: isFirst = True

    Do While fileName <> ""
        Set wbSource = Workbooks.Open(folderPath & fileName, ReadOnly:=True)
        With wbSource.Sheets(1).UsedRange
            If isFirst Then
                .Copy wsMaster.Range("A1")
                isFirst = False
            ElseIf .Rows.Count > 1 Then
                lastRow = wsMaster.Cells(wsMaster.Rows.Count, 1).End(xlUp).Row + 1
                .Offset(1, 0).Resize(.Rows.Count - 1, .Columns.Count).Copy wsMaster.Cells(lastRow, 1)
            End If
        End With
        wbSource.Close SaveChanges:=False
        fileName = Dir()
    Loop

    wsMaster.Columns.AutoFit
    Application.ScreenUpdating = True
    MsgBox "All files merged into " & wsMaster.Name, vbInformation
End Sub

3. Auto-Generate Formatted Excel Workbooks from Python (`openpyxl`)

When pairing scraped web data (from our RapidAPI Hub REST APIs) with Excel delivery, openpyxl lets you apply frozen headers, auto-filters, and conditional fills automatically:

from openpyxl import Workbook
from openpyxl.styles import Font, PatternFill

def export_leads_to_excel(rows: list[dict], out_path: str = "b2b_leads.xlsx") -> None:
    wb = Workbook()
    ws = wb.active
    ws.title = "Verified Leads"

    headers = ["Company URL", "Title", "Primary Email", "Phone", "LinkedIn"]
    ws.append(headers)

    header_fill = PatternFill(start_color="1F4E78", end_color="1F4E78", fill_type="solid")
    header_font = Font(color="FFFFFF", bold=True)
    for col_idx in range(1, len(headers) + 1):
        cell = ws.cell(row=1, column=col_idx)
        cell.fill = header_fill
        cell.font = header_font

    for r in rows:
        ws.append([
            r.get("url", ""),
            r.get("title", ""),
            (r.get("emails") or [""])[0],
            (r.get("phones") or [""])[0],
            ((r.get("socials") or {}).get("linkedin") or [""])[0],
        ])

    ws.freeze_panes = "A2"
    ws.auto_filter.ref = ws.dimensions
    wb.save(out_path)

📦 Grab the Complete 10-Macro Pack + Python Suites

Ultimate Excel VBA Automation Pack ($9.99) — Payhip | Gumroad
Python B2B Lead Scraper & Excel VBA Suite Pro ($14.99) — Payhip | Gumroad
Shopify Competitor Price & Product Spy Pro ($12.99) — Payhip | Gumroad

5 Production-Ready Web Scraping & AI Data Extraction REST APIs on RapidAPI Hub (Python, Node.js & cURL)

Tùng Nguyễn — Sat, 31 Oct 2026 06:30:59 +0000

Maintaining headless browsers, residential proxy rotation, anti-bot retries, and HTML parsers across dozens of client projects takes time away from building your core product.

To make web data extraction plug-and-play for Python, Node.js, Go, and no-code (Make / Zapier / n8n) workflows, I published 5 production-ready Web Scraping & AI Data Extraction REST APIs on RapidAPI Hub — each with a Free Tier (BASIC $0.00/mo) so you can test immediately in your browser or terminal.

🚀 The 5 Live REST APIs on RapidAPI Hub

1. B2B Website Email, Phone & Social Media Scraper API

Extracts verified emails (mailto: + de-obfuscated text), phone numbers, LinkedIn, Twitter/X, Facebook, Instagram, YouTube, and TikTok profiles across company homepages and /contact or /about pages.

🔗 RapidAPI Hub: rapidapi.com/tungnguyen84/api/b2b-website-email-phone-social-scraper
☁️ Apify Store: apify.com/tungnguyen84/b2b-website-email-phone-social-scraper
🎮 Live Demo: Hugging Face Space

2. Website to Clean Markdown AI, LLM & RAG Converter API

Strips navigation menus, ads, cookie banners, inline scripts, and layout clutter from any webpage and returns structured Markdown with YAML frontmatter, word count, and token estimates for LangChain, LlamaIndex, and OpenAI / Claude RAG pipelines.

🔗 RapidAPI Hub: rapidapi.com/tungnguyen84/api/website-to-clean-markdown-ai-llm-rag-converter
☁️ Apify Store: apify.com/tungnguyen84/website-to-markdown-ai-rag-llm-scraper
🎮 Live Demo: Hugging Face Space

3. Shopify Store Product, Price & Contact Scraper API

Extracts complete product catalogs (title, handle, vendor, product_type, price, compare_at_price, discount_pct, in_stock, SKU, images) plus store owner contact details from any Shopify storefront.

🔗 RapidAPI Hub: rapidapi.com/tungnguyen84/api/shopify-store-product-price-contact-scraper
☁️ Apify Store: apify.com/tungnguyen84/shopify-store-product-contact-scraper
🎮 Live Demo: Hugging Face Space

4. WooCommerce & WordPress Product, Price & Contact Spy API

Extracts product listings, sale prices, stock status, categories, and merchant contact info from WooCommerce / WordPress e-commerce stores via Store API + ld+json Product schema fallback.

🔗 RapidAPI Hub: rapidapi.com/tungnguyen84/api/woocommerce-wordpress-product-price-contact-spy
☁️ Apify Store: apify.com/tungnguyen84/woocommerce-wordpress-store-product-contact-scraper

5. Website Sitemap, SEO Audit & Contact Scraper API

Audits on-page technical SEO (<title>, <meta name="description">, canonical URLs, H1–H3 structure, missing image alt attributes, robots.txt, sitemap.xml) while simultaneously extracting lead emails and social links.

🔗 RapidAPI Hub: rapidapi.com/tungnguyen84/api/website-sitemap-seo-audit-contact-scraper
☁️ Apify Store: apify.com/tungnguyen84/website-sitemap-seo-audit-email-scraper

💻 Quickstart Code Snippets (Python & Node.js)

Python (`requests`) — Convert Any URL to LLM-Ready Markdown

import requests

url = "https://website-to-clean-markdown-ai-llm-rag-converter.p.rapidapi.com/v2/acts/tungnguyen84~website-to-markdown-ai-rag-llm-scraper/run-sync-get-dataset-items"
headers = {
    "x-rapidapi-host": "website-to-clean-markdown-ai-llm-rag-converter.p.rapidapi.com",
    "x-rapidapi-key": "YOUR_RAPIDAPI_KEY",
    "Content-Type": "application/json"
}
payload = {
    "startUrls": [{"url": "https://docs.python.org/3/tutorial/index.html"}]
}

resp = requests.post(url, json=payload, headers=headers, timeout=60)
for item in resp.json():
    print(f"Title: {item.get('title')} ({item.get('estimatedTokens')} tokens)")
    print(item.get("markdown")[:500])

Node.js (`fetch`) — Scrape B2B Emails & Social Profiles

const response = await fetch(
  "https://b2b-website-email-phone-social-scraper.p.rapidapi.com/v2/acts/tungnguyen84~b2b-website-email-phone-social-scraper/run-sync-get-dataset-items",
  {
    method: "POST",
    headers: {
      "x-rapidapi-host": "b2b-website-email-phone-social-scraper.p.rapidapi.com",
      "x-rapidapi-key": process.env.RAPIDAPI_KEY,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      startUrls: [{ url: "https://apify.com" }],
      maxContactPages: 2,
    }),
  }
);

const leads = await response.json();
console.log(leads);

📦 Want the Full Standalone Source Code & Excel VBA Dashboards?

If you prefer owning the full offline Python CLI tools + 1-click Excel VBA macro suites with zero recurring API fees:

Python B2B Lead Scraper, Email Extractor & Excel VBA Automation Suite Pro ($14.99)
- 👉 Download on Payhip (PayPal / Card) | Gumroad
Shopify Competitor Price & Product Spy Pro — Python CLI & Excel Exporter ($12.99)
- 👉 Download on Payhip (PayPal / Card) | Gumroad
Ultimate Excel VBA Automation Pack — 10 One-Click Business Macros ($9.99)
- 👉 Download on Payhip (PayPal / Card) | Gumroad

Need custom web scraping, API integration, or automated reporting? Message me on Fiverr (@tungnguyen84)!

How to Build an Automated Technical SEO, Sitemap & Lead Audit Pipeline with Python & REST API

Tùng Nguyễn — Sat, 31 Oct 2026 06:30:26 +0000

Technical SEO audits and B2B lead discovery often live in separate tools: one crawler checks <title>, <meta name="description">, H1 tags, canonical URLs, OpenGraph tags, and sitemap.xml / robots.txt health, while another script scrapes decision-maker emails and social profiles.

In this tutorial, we will combine both into a single high-speed Python Technical SEO, Sitemap & Lead Audit Pipeline — and show how you can call it directly via our RapidAPI Hub REST API or Apify Cloud Actor without managing servers or proxies.

⚡ Instant Cloud APIs, Interactive Demo & Pro Suite

🌐 RapidAPI Hub REST API (Freemium BASIC $0.00/mo -> PRO $9.99/mo): Website Sitemap SEO Audit Contact Scraper on RapidAPI
☁️ Apify Cloud Actor ($0.002/page + $5 Free Credit): Website Sitemap, SEO Audit & Lead Extractor on Apify Store
🎮 Live Interactive Web App (Hugging Face Space): B2B Lead, Email & SEO Audit Pro
📦 Download Full Standalone Python + Excel VBA Pro Suite ($14.99):
- Payhip Store (PayPal / Card): payhip.com/b/Vm3DR
- Gumroad Store: tungnguyen84.gumroad.com/l/pinmv
💻 GitHub Open-Source Repository: tungnguyen84/b2b-lead-scraper-automation-suite

🔍 What a Complete Technical SEO + Contact Audit Checks

When auditing prospect websites for an SEO agency, SaaS outreach campaign, or web development pitch, extracting the exact technical gaps alongside verified contact details lets you personalize cold outreach automatically (e.g., "Hi team, I noticed your homepage is missing a meta description and has 3 images without alt text..."):

On-Page SEO Signals:
- Title tag presence and character length (30–60 chars ideal)
- Meta description presence and character length (70–160 chars ideal)
- Canonical URL (<link rel="canonical">), Robots meta tag, and OpenGraph (og:title, og:image)
- Heading hierarchy (H1, H2, H3 counts) and missing alt attributes on <img> tags
Infrastructure & Discovery:
- robots.txt status (200 OK vs missing)
- sitemap.xml discovery and total indexed URL count
Decision-Maker Contact Enrichment:
- De-obfuscated emails (mailto: and body text) + phone numbers + LinkedIn / X / Facebook / Instagram links.

🐍 Python Implementation: Single-Pass SEO & Contact Auditor

import re
from urllib.parse import urljoin
import requests
from bs4 import BeautifulSoup

EMAIL_RE = re.compile(r'[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,}')

def audit_website_seo_and_contacts(url: str) -> dict:
    base = url.rstrip('/')
    headers = {'User-Agent': 'Mozilla/5.0 (compatible; SEOAuditBot/1.0)'}
    resp = requests.get(base, timeout=15, headers=headers)
    soup = BeautifulSoup(resp.text, 'html.parser')

    title = soup.title.string.strip() if soup.title and soup.title.string else ''
    meta_desc_tag = soup.find('meta', attrs={'name': re.compile(r'^description$', re.I)})
    meta_desc = (meta_desc_tag.get('content') or '').strip() if meta_desc_tag else ''
    canonical_tag = soup.find('link', rel=lambda v: v and 'canonical' in v.lower())
    canonical = canonical_tag.get('href') if canonical_tag else None

    h1_tags = [h.get_text(" ", strip=True) for h in soup.find_all('h1')]
    imgs = soup.find_all('img')
    imgs_missing_alt = sum(1 for img in imgs if not (img.get('alt') or '').strip())

    # Check robots.txt and sitemap.xml
    has_robots = False
    has_sitemap = False
    try:
        r_rob = requests.get(urljoin(base + '/', 'robots.txt'), timeout=6, headers=headers)
        has_robots = (r_rob.status_code == 200 and len(r_rob.text) > 10)
    except Exception:
        pass
    try:
        r_site = requests.get(urljoin(base + '/', 'sitemap.xml'), timeout=6, headers=headers)
        has_sitemap = (r_site.status_code == 200 and '<urlset' in r_site.text[:500])
    except Exception:
        pass

    emails = sorted({
        m.lower() for m in EMAIL_RE.findall(resp.text)
        if not m.lower().endswith(('.png', '.jpg', '.svg', '.webp', '.js', '.css'))
    })

    issues = []
    if not title:
        issues.append('Missing <title> tag')
    if not meta_desc:
        issues.append('Missing <meta name="description">')
    if len(h1_tags) == 0:
        issues.append('Missing H1 heading')
    elif len(h1_tags) > 1:
        issues.append(f'Multiple H1 headings ({len(h1_tags)})')
    if imgs_missing_alt > 0:
        issues.append(f'{imgs_missing_alt} images missing alt text')

    return {
        'url': base,
        'statusCode': resp.status_code,
        'title': title,
        'titleLength': len(title),
        'metaDescription': meta_desc,
        'metaDescriptionLength': len(meta_desc),
        'canonical': canonical,
        'h1Count': len(h1_tags),
        'imagesTotal': len(imgs),
        'imagesMissingAlt': imgs_missing_alt,
        'hasRobotsTxt': has_robots,
        'hasSitemapXml': has_sitemap,
        'seoIssues': issues,
        'emails': emails,
    }

🌐 Calling the Live REST API on RapidAPI Hub

Prefer a managed REST endpoint with a free monthly tier? You can call our Website Sitemap SEO Audit Contact Scraper API on RapidAPI Hub:

import requests

url = "https://website-sitemap-seo-audit-contact-scraper.p.rapidapi.com/v2/acts/tungnguyen84~website-sitemap-seo-audit-email-scraper/run-sync-get-dataset-items"
headers = {
    "x-rapidapi-host": "website-sitemap-seo-audit-contact-scraper.p.rapidapi.com",
    "x-rapidapi-key": "YOUR_RAPIDAPI_KEY",
    "Content-Type": "application/json"
}
payload = {
    "startUrls": [{"url": "https://example.com"}],
    "maxPagesPerSite": 3
}

response = requests.post(url, json=payload, headers=headers)
print(response.json())

🚀 Explore the Full Developer Toolkit

🌐 All 5 RapidAPI Hub REST APIs:
🛒 Standalone Python & Excel VBA Pro Bundles:
- Payhip Store: payhip.com/tungnguyen84 ($9.99 – $14.99)
- Gumroad Store: tungnguyen84.gumroad.com
🤝 Custom Scraping & Automation Projects: Hire me on Fiverr (@tungnguyen84)

Automating Shopify & WooCommerce Competitor Price Monitoring & Contact Extraction with Python

Tùng Nguyễn — Sat, 31 Oct 2026 03:41:10 +0000

Whether you run an e-commerce brand, a dropshipping research agency, or a B2B wholesale supply business, tracking competitor product catalogs, stock availability, price drops, and store contact details across Shopify and WooCommerce stores is a massive competitive advantage.

In this tutorial, we'll cover how to extract full product catalogs (title, SKU, price, compare_at_price, in_stock, variants, images) plus store owner emails & social links using Python.

âš¡ Ready-to-Use Cloud Scrapers & Standalone Tools

ðŸ›ï¸ Shopify Store Product, Price & Contact Scraper (Apify Cloud API): apify.com/tungnguyen84/shopify-store-product-contact-scraper
ðŸ›’ WooCommerce & WordPress Store Product & Contact Scraper (Apify Cloud API): apify.com/tungnguyen84/woocommerce-wordpress-store-product-contact-scraper
ðŸ’» Download Shopify Competitor Spy Pro (Python Desktop CLI + Excel Exporter â€” $12.99): tungnguyen84.gumroad.com/l/lxqssu
ðŸ“Š Ultimate Excel VBA Automation Pack: 10 One-Click Macros ($9.99): tungnguyen84.gumroad.com/l/wybjyn
â GitHub Repo: tungnguyen84/shopify-woocommerce-product-price-scraper

ðŸ” How Shopify & WooCommerce Expose Structured Product JSON

Shopify Stores: Almost all Shopify storefronts expose a paginated JSON feed at /products.json?limit=250&page=1 containing every product, variant price, compare-at price, SKU, vendor, and availability flag.
WooCommerce Stores: Modern WooCommerce stores expose the Store API at /wp-json/wc/store/v1/products?per_page=100, with fallback to ld+json Product schema and OpenGraph price tags.

Python Example: Fetching Shopify Products + Discount Calculation

import requests

def fetch_shopify_catalog(store_url: str, max_items: int = 100) -> list[dict]:
    base = store_url.rstrip('/')
    resp = requests.get(f"{base}/products.json?limit=250&page=1", timeout=15)
    resp.raise_for_status()
    products = resp.json().get('products', [])[:max_items]

    rows = []
    for p in products:
        v = (p.get('variants') or [{}])[0]
        price = float(v.get('price') or 0)
        compare_at = float(v.get('compare_at_price') or 0)
        discount_pct = round((compare_at - price) / compare_at * 100, 1) if compare_at > price else 0.0
        rows.append({
            'title': p.get('title'),
            'vendor': p.get('vendor'),
            'product_type': p.get('product_type'),
            'price': price,
            'compare_at_price': compare_at or None,
            'discount_pct': discount_pct,
            'in_stock': bool(v.get('available')),
            'sku': v.get('sku'),
            'url': f"{base}/products/{p.get('handle')}",
        })
    return rows

ðŸš€ Need Custom E-Commerce Monitoring or Excel Dashboards?

Try our 5 Cloud Actors on Apify Store (@tungnguyen84)
Grab the standalone tools on Gumroad (tungnguyen84.gumroad.com)
Or message me directly on Fiverr (@tungnguyen84) for custom scrapers and automated Excel/Google Sheets dashboards!

🌐 Live REST APIs on RapidAPI Hub & Instant Download Store

RapidAPI Hub (Free Tier BASIC $0.00/mo -> PRO $9.99/mo):
Payhip Instant Download Store (PayPal / Card):

Turning Any Website into Clean LLM-Ready Markdown for AI RAG Pipelines in Python

Tùng Nguyễn — Sat, 31 Oct 2026 03:31:40 +0000

When building Retrieval-Augmented Generation (RAG) pipelines or AI agents with LangChain, LlamaIndex, or OpenAI / Claude, feeding raw HTML directly into an embedding model wastes up to 78% of your context window on <nav>, <script>, SVG icons, cookie banners, and inline styles.

To get high-precision vector embeddings and clean chunking, you need a deterministic Website-to-Markdown Crawler that strips layout boilerplate while preserving heading hierarchies (H1â€“H6), code blocks, tables, and YAML metadata.

âš¡ Try the Live Cloud Actor or Open-Source Repo

â˜ï¸ Apify Cloud Actor ($0.002 / page â€” Free $5 Monthly Credit): Website to Markdown & Clean Text Scraper for AI, RAG & LLMs
ðŸ’» GitHub Repository: tungnguyen84/website-to-markdown-ai-rag-crawler
ðŸ¤ Custom AI RAG Pipelines & Web Crawlers: Fiverr (@tungnguyen84)

ðŸ› ï¸ Why Standard HTML-to-Text Breaks RAG Chunking

If you simply call soup.get_text(), your headings, bullet lists, and code blocks flatten into an unstructured wall of text. Semantic chunkers (MarkdownHeaderTextSplitter in LangChain) rely on clean #, ##, and ### boundaries to keep related paragraphs together.

Python Implementation: Boilerplate Removal + Markdown Conversion

import re
import requests
from bs4 import BeautifulSoup

NOISE_SELECTORS = [
    'script', 'style', 'noscript', 'svg', 'iframe',
    'nav', 'footer', 'header', 'aside',
    '.cookie-banner', '.newsletter', '.sidebar', '#comments'
]

def html_to_rag_markdown(url: str) -> dict:
    resp = requests.get(url, timeout=15, headers={'User-Agent': 'Mozilla/5.0'})
    soup = BeautifulSoup(resp.text, 'html.parser')

    title = soup.title.string.strip() if soup.title and soup.title.string else url
    for sel in NOISE_SELECTORS:
        for tag in soup.select(sel):
            tag.decompose()

    main = soup.find('main') or soup.find('article') or soup.body or soup
    lines = [
        "---",
        f'title: "{title}"',
        f'source_url: "{url}"',
        "---\n"
    ]

    for el in main.find_all(['h1', 'h2', 'h3', 'h4', 'p', 'pre', 'li']):
        text = el.get_text(" ", strip=True)
        if not text:
            continue
        if el.name.startswith('h'):
            level = int(el.name[1])
            lines.append(f"\n{'#' * level} {text}\n")
        elif el.name == 'pre':
            lines.append(f"\n```
{% endraw %}
\n{el.get_text()}\n
{% raw %}
```\n")
        elif el.name == 'li':
            lines.append(f"- {text}")
        else:
            lines.append(text + "\n")

    markdown = re.sub(r'\n{3,}', '\n\n', "\n".join(lines)).strip()
    word_count = len(markdown.split())
    return {
        "url": url,
        "title": title,
        "wordCount": word_count,
        "estimatedTokens": int(word_count * 1.33),
        "markdown": markdown,
    }

â˜ï¸ Calling the Cloud API Directly in Python

With our Apify Cloud Actor, you can convert hundreds of documentation pages or knowledge bases into LLM-ready Markdown via a single REST call and pipe the JSON dataset directly into LangChain or Pinecone:

ðŸ‘‰ Launch Website to Markdown & Clean Text Scraper for AI, RAG & LLMs on Apify Store

🌐 Live REST APIs on RapidAPI Hub & Instant Download Store

RapidAPI Hub (Free Tier BASIC $0.00/mo -> PRO $9.99/mo):
Payhip Instant Download Store (PayPal / Card):

How to Build a High-Speed B2B Lead, Email & Social Profile Scraper in Python (Open-Source + Cloud API)

Tùng Nguyễn — Sat, 31 Oct 2026 03:31:07 +0000

Building a reliable B2B lead enrichment pipeline in 2026 is much harder than simply running requests.get() on a list of URLs. Most modern company websites split their contact details across /contact, /about-us, footer obfuscation (info [at] company.com), and mailto: / tel: attributes.

In this guide, I will walk through how to build a high-concurrency Python B2B Website Email, Phone & Social Media Scraper that extracts decision-maker contact info and exports directly to a formatted Excel / CSV dashboard.

âš¡ Quick Links (Cloud API, Open-Source & Pro Suite)

If you want to run this immediately in the cloud or download the full multi-threaded CLI + Excel VBA reporting suite:

â˜ï¸ Run on Apify Cloud (No setup, Pay-Per-Result $0.002/lead + $5 Free Credit): B2B Website Email, Phone & Social Media Scraper on Apify Store
ðŸ“¦ Download Full Standalone Pro Suite ($14.99 on Gumroad): Python B2B Lead Scraper, Email Extractor & Excel VBA Automation Suite Pro
ðŸ’» GitHub Open-Source Repository: tungnguyen84/b2b-lead-scraper-automation-suite
ðŸ¤ Need a Custom Scraper or B2B Lead List?: Hire me on Fiverr (@tungnguyen84 â€” 5.0â˜… Rated)

ðŸ§ Architecture of a High-Accuracy B2B Contact Crawler

A production B2B contact extractor needs 4 stages:

Homepage Fetch & Metadata Extraction: Grab <title>, <meta name="description">, and visible text from the landing page.
Smart Contact Page Discovery: Automatically detect internal links matching /contact, /about, /team, or /support and crawl them concurrently.
Regex + De-obfuscation + False-Positive Filtering: Strip out junk asset matches (logo@2x.png, sentry@domain.com, example@domain.com) while capturing mailto: and visible emails.
Social Profile Normalization: Match canonical company profiles on LinkedIn (/company/), Twitter/X, Facebook, Instagram, YouTube, and TikTok.

Core Python Implementation

import re
from urllib.parse import urljoin, urlparse
import requests
from bs4 import BeautifulSoup

EMAIL_REGEX = re.compile(r'[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,}')
IGNORE_DOMAINS = {'sentry.io', 'example.com', 'domain.com', 'wixpress.com', 'cloudflare.com'}
IGNORE_EXTS = ('.png', '.jpg', '.jpeg', '.gif', '.svg', '.webp', '.css', '.js')

SOCIAL_PATTERNS = {
    'linkedin': re.compile(r'https?://(?:www\.)?linkedin\.com/(?:company|in)/[a-zA-Z0-9_\-]+', re.I),
    'twitter': re.compile(r'https?://(?:www\.)?(?:twitter\.com|x\.com)/[a-zA-Z0-9_]+', re.I),
    'facebook': re.compile(r'https?://(?:www\.)?facebook\.com/[a-zA-Z0-9.\-_]+', re.I),
    'instagram': re.compile(r'https?://(?:www\.)?instagram\.com/[a-zA-Z0-9._]+', re.I),
}

def clean_emails(raw_text: str) -> list[str]:
    found = set()
    for match in EMAIL_REGEX.findall(raw_text):
        email = match.lower().strip('.')
        if email.endswith(IGNORE_EXTS):
            continue
        domain = email.split('@')[-1]
        if domain in IGNORE_DOMAINS:
            continue
        found.add(email)
    return sorted(found)

def scrape_b2b_domain(url: str) -> dict:
    resp = requests.get(url, timeout=15, headers={'User-Agent': 'Mozilla/5.0'})
    soup = BeautifulSoup(resp.text, 'html.parser')
    combined_html = resp.text

    # Follow /contact or /about link if present
    for a in soup.find_all('a', href=True):
        href = a['href'].lower()
        if any(k in href for k in ('/contact', '/about', '/impressum')):
            sub_url = urljoin(url, a['href'])
            if urlparse(sub_url).netloc == urlparse(url).netloc:
                try:
                    combined_html += "\n" + requests.get(sub_url, timeout=10).text
                except Exception:
                    pass
                break

    emails = clean_emails(combined_html)
    socials = {
        name: sorted(set(pattern.findall(combined_html)))
        for name, pattern in SOCIAL_PATTERNS.items()
    }
    return {
        'url': url,
        'title': soup.title.string.strip() if soup.title and soup.title.string else '',
        'emails': emails,
        'socials': socials,
    }

ðŸš€ Scaling to 10,000+ Domains via Apify Cloud API

If you don't want to manage proxies, concurrency, or headless browsers locally, I published a ready-to-run Cloud Actor on the Apify Store that handles batch URLs, automatic contact page traversal, and structured JSON/CSV/Excel dataset export:

ðŸ”— Apify Actor: tungnguyen84/b2b-website-email-phone-social-scraper
ðŸ” Bonus SEO + Sitemap Lead Auditor: tungnguyen84/website-sitemap-seo-audit-email-scraper

Or grab the Standalone Python + Excel VBA Pro Suite on Gumroad:
ðŸ‘‰ Download Python B2B Lead Scraper & Excel VBA Suite Pro ($14.99)

🌐 Live REST APIs on RapidAPI Hub & Instant Download Store

RapidAPI Hub (Free Tier BASIC $0.00/mo -> PRO $9.99/mo):
Payhip Instant Download Store (PayPal / Card):

DEV Community: Tùng Nguyễn

Automating Excel Reporting & Data Cleaning: 10 Essential VBA Macros + Python Pipeline

⚡ Ready-to-Use Excel VBA & Python Automation Packs

🛠️ 3 Production-Grade VBA Macros Every Analyst Should Have

1. Deep-Clean Whitespace, Non-Breaking Spaces (CHAR(160)) & Case Normalization

2. Merge All CSV / XLSX Files in a Folder into One Master Sheet

3. Auto-Generate Formatted Excel Workbooks from Python (openpyxl)

📦 Grab the Complete 10-Macro Pack + Python Suites

5 Production-Ready Web Scraping & AI Data Extraction REST APIs on RapidAPI Hub (Python, Node.js & cURL)

🚀 The 5 Live REST APIs on RapidAPI Hub

1. B2B Website Email, Phone & Social Media Scraper API

2. Website to Clean Markdown AI, LLM & RAG Converter API

3. Shopify Store Product, Price & Contact Scraper API

4. WooCommerce & WordPress Product, Price & Contact Spy API

5. Website Sitemap, SEO Audit & Contact Scraper API

💻 Quickstart Code Snippets (Python & Node.js)

Python (requests) — Convert Any URL to LLM-Ready Markdown

Node.js (fetch) — Scrape B2B Emails & Social Profiles

📦 Want the Full Standalone Source Code & Excel VBA Dashboards?

How to Build an Automated Technical SEO, Sitemap & Lead Audit Pipeline with Python & REST API

⚡ Instant Cloud APIs, Interactive Demo & Pro Suite

🔍 What a Complete Technical SEO + Contact Audit Checks

🐍 Python Implementation: Single-Pass SEO & Contact Auditor

🌐 Calling the Live REST API on RapidAPI Hub

🚀 Explore the Full Developer Toolkit

Automating Shopify & WooCommerce Competitor Price Monitoring & Contact Extraction with Python

âš¡ Ready-to-Use Cloud Scrapers & Standalone Tools

ðŸ” How Shopify & WooCommerce Expose Structured Product JSON

Python Example: Fetching Shopify Products + Discount Calculation

ðŸš€ Need Custom E-Commerce Monitoring or Excel Dashboards?

🌐 Live REST APIs on RapidAPI Hub & Instant Download Store

Turning Any Website into Clean LLM-Ready Markdown for AI RAG Pipelines in Python

âš¡ Try the Live Cloud Actor or Open-Source Repo

ðŸ› ï¸ Why Standard HTML-to-Text Breaks RAG Chunking

Python Implementation: Boilerplate Removal + Markdown Conversion

â˜ï¸ Calling the Cloud API Directly in Python

🌐 Live REST APIs on RapidAPI Hub & Instant Download Store

How to Build a High-Speed B2B Lead, Email & Social Profile Scraper in Python (Open-Source + Cloud API)

âš¡ Quick Links (Cloud API, Open-Source & Pro Suite)

ðŸ§ Architecture of a High-Accuracy B2B Contact Crawler

Core Python Implementation

ðŸš€ Scaling to 10,000+ Domains via Apify Cloud API

🌐 Live REST APIs on RapidAPI Hub & Instant Download Store

1. Deep-Clean Whitespace, Non-Breaking Spaces (`CHAR(160)`) & Case Normalization

3. Auto-Generate Formatted Excel Workbooks from Python (`openpyxl`)

Python (`requests`) — Convert Any URL to LLM-Ready Markdown

Node.js (`fetch`) — Scrape B2B Emails & Social Profiles

ðŸ” How Shopify & WooCommerce Expose Structured Product JSON

ðŸ› ï¸ Why Standard HTML-to-Text Breaks RAG Chunking

â˜ï¸ Calling the Cloud API Directly in Python