DEV Community

Henry Knight
Henry Knight

Posted on

I Automated My Entire Lead Pipeline with Claude (Python + Google Maps Scraper)

Lead generation is one of those tasks that sounds simple until you're 3 hours deep in a Google Maps tab, copying business names and phone numbers into a spreadsheet like it's 2010.

I used to do this manually. Now a Python script does it for me — scrapes Google Maps, enriches the data, writes personalized outreach emails, and logs everything to a database. Claude handles the brain work. I just review the output.

Here's exactly how I built it.

The Problem

I run a small AI automation agency (solo, bootstrapped). Every week I need 50-100 fresh leads — local businesses who might pay for automation work. The manual loop:

  1. Search Google Maps for "restaurants in [city]"
  2. Click through results, copy name/phone/website
  3. Google the website, find a contact email
  4. Write a cold email
  5. Repeat 50x

That's 3-4 hours. Every week. On a task a Python script should handle.

The Architecture

scraper.py       → Google Maps     → raw lead data
enricher.py      → website scraper → emails, social links
claude_writer.py → Claude API      → personalized outreach per lead
pipeline.py      → orchestrates all 3, logs to SQLite
Enter fullscreen mode Exit fullscreen mode

Each module is independent. Swap out the scraper, use a different LLM, or add new enrichment steps without touching the rest.

Step 1: Scraping Google Maps with Playwright

Google Maps doesn't have a free public API for scraping, so I use Playwright for browser automation:

from playwright.sync_api import sync_playwright
import time

def scrape_google_maps(query: str, max_results: int = 50) -> list[dict]:
    leads = []

    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()

        page.goto(f"https://www.google.com/maps/search/{query.replace(' ', '+')}")
        page.wait_for_selector('[role="feed"]', timeout=10000)

        last_count = 0

        while len(leads) < max_results:
            items = page.locator('[role="feed"] > div').all()

            for item in items[last_count:]:
                try:
                    name = item.locator('a[aria-label]').get_attribute('aria-label')
                    href = item.locator('a[aria-label]').get_attribute('href')

                    item.click()
                    page.wait_for_timeout(1500)

                    phone_el = page.locator('[data-tooltip="Copy phone number"]')
                    phone = phone_el.inner_text() if phone_el.count() else None

                    website_el = page.locator('a[data-item-id="authority"]')
                    website = website_el.get_attribute('href') if website_el.count() else None

                    leads.append({
                        "name": name,
                        "phone": phone,
                        "website": website,
                        "maps_url": href
                    })
                except Exception:
                    continue

            last_count = len(items)
            page.evaluate("document.querySelector('[role="feed"]').scrollBy(0, 2000)")
            time.sleep(1.5)

            if len(items) == last_count:
                break

        browser.close()

    return leads[:max_results]
Enter fullscreen mode Exit fullscreen mode

Key thing: the [role="feed"] selector is stable across Google Maps updates. I've been using it for 4 months without breaking.

Step 2: Enriching with Contact Info

Raw Maps data usually has a phone number and website, but rarely an email. A lightweight scraper visits each website and hunts for contact addresses:

import re, httpx
from bs4 import BeautifulSoup

EMAIL_REGEX = re.compile(r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+.[a-zA-Z]{2,}')
SKIP_DOMAINS = {'sentry.io', 'wix.com', 'wordpress.com', 'example.com'}

def extract_emails(website_url: str) -> list[str]:
    if not website_url:
        return []

    try:
        resp = httpx.get(website_url, timeout=8, follow_redirects=True,
                         headers={"User-Agent": "Mozilla/5.0"})
        soup = BeautifulSoup(resp.text, 'html.parser')

        for tag in soup(['script', 'style']):
            tag.decompose()

        text = soup.get_text()
        emails = EMAIL_REGEX.findall(text)

        clean = [e for e in set(emails)
                 if not any(skip in e for skip in SKIP_DOMAINS)
                 and len(e) < 80]

        return clean[:3]

    except Exception:
        return []
Enter fullscreen mode Exit fullscreen mode

This gets an email for about 60-70% of leads. For the rest, fall back to the contact form or LinkedIn.

Step 3: Claude Writes the Outreach

Instead of a template, Claude generates a personalized email per lead based on their business name, website content, and niche:

import anthropic, json

client = anthropic.Anthropic()

def write_outreach_email(lead: dict, agency_context: str) -> dict:
    prompt = f"""You are writing a cold outreach email for an AI automation agency.

Lead info:
- Business: {lead['name']}
- Website: {lead['website']}
- Niche: {lead['niche']}
- Website snippet: {lead.get('website_text', 'N/A')[:500]}

Agency context: {agency_context}

Write a short (3-4 paragraph), non-salesy cold email that:
1. Opens with something specific to their business
2. Identifies one automation opportunity relevant to their niche
3. Proposes a free 15-min call
4. Has a clear subject line

Return JSON: {{"subject": "...", "body": "..."}}"""

    message = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=600,
        messages=[{"role": "user", "content": prompt}]
    )

    return json.loads(message.content[0].text)
Enter fullscreen mode Exit fullscreen mode

The niche-specific framing is the unlock. A restaurant gets an email about reservation automation. A real estate agent gets one about lead follow-up sequences. Generic templates don't convert — this does.

Step 4: The Pipeline Orchestrator

Everything ties together in pipeline.py:

import sqlite3
from scraper import scrape_google_maps
from enricher import extract_emails
from claude_writer import write_outreach_email

DB_PATH = "leads.db"
AGENCY_CONTEXT = "We build AI automation for local businesses — chatbots, workflow bots, lead follow-up systems."

def init_db():
    conn = sqlite3.connect(DB_PATH)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS leads (
            id INTEGER PRIMARY KEY,
            name TEXT, phone TEXT, website TEXT,
            email TEXT, outreach_subject TEXT,
            outreach_body TEXT, status TEXT DEFAULT 'new',
            created_at DATETIME DEFAULT CURRENT_TIMESTAMP
        )
    """)
    conn.commit()
    return conn

def run_pipeline(query: str, max_leads: int = 50):
    conn = init_db()

    print(f"Scraping: {query}")
    raw_leads = scrape_google_maps(query, max_leads)

    for lead in raw_leads:
        emails = extract_emails(lead['website'])
        lead['email'] = emails[0] if emails else None
        lead['niche'] = query.split(' in ')[0]

        outreach = write_outreach_email(lead, AGENCY_CONTEXT)

        conn.execute("""
            INSERT INTO leads (name, phone, website, email, outreach_subject, outreach_body)
            VALUES (?, ?, ?, ?, ?, ?)
        """, (lead['name'], lead['phone'], lead['website'],
              lead['email'], outreach['subject'], outreach['body']))
        conn.commit()

        print(f"{lead['name']}{lead['email'] or 'no email found'}")

if __name__ == "__main__":
    run_pipeline("restaurants in Austin TX", max_leads=50)
Enter fullscreen mode Exit fullscreen mode

Run it once and you have 50 leads with personalized emails waiting in a SQLite database.

Results

I run this every Monday morning on 3 different queries. By the time I finish coffee, I have 150 fresh leads with outreach emails ready to review and send.

Conversion rates improved versus my old templates — Claude's personalization references real details from each website so emails read as human.

Time saved: ~3.5 hours per week. At a $100/hr rate, that's $350/week recovered from one script.

What's Next

A few improvements I'm testing:

  • Auto-send via Gmail API after a human review flag in the DB
  • Follow-up sequencing — Claude writes 3-email drip sequences, not just the opener
  • LinkedIn enrichment — scrape the owner's LinkedIn for even more personalization signal

Want the Full Starter Kit?

I packaged everything — the full pipeline, browser automation patterns, Claude API integration snippets, and a Playwright setup guide — into the Claude Browser Agent Starter Kit.

It's $7. Grab it here → payhip.com/b/Gu

It's what I wish I had when I started building these automations. Skip 20 hours of debugging and get straight to shipping.


Questions? Drop them in the comments — I read everything.

Top comments (0)