Yellow Pages Scraper GitHub: How to Extract Local Business Leads with Python

If you need to build a local business lead list and do not want to pay for a commercial database, an open-source Yellow Pages scraper gives you a repeatable way to collect public business names, addresses, phone numbers, and categories from directory listings. The data-scrape/yellow-pages-scraper repository on GitHub provides a Python workflow that turns search results into structured JSON or CSV records you can import into a CRM, enrich, or analyze.

This article is for sales teams, marketing agencies, and developers who want to understand how a repository-based Yellow Pages workflow works, what fields it returns, and how to run it safely without violating terms of service or privacy rules.

TL;DR

• Yellow Pages directories publish public business listings that can be collected with a structured scraper.
• The data-scrape/yellow-pages-scraper repository offers a Python workflow with request pacing, field extraction, and JSON/CSV output.
• A typical run returns business name, address, phone, category, rating, and listing URL.
• You should pace requests, respect robots directives, and verify current directory terms before production use.

Why Manual Lead Research Does Not Scale

Building a local lead list by hand is straightforward until you need more than a few dozen records. Copying business names, addresses, and phone numbers from a directory into a spreadsheet is slow, error-prone, and becomes unmaintainable when:

• You target multiple cities or zip codes.
• You need fresh data every month for outreach campaigns.
• You want to filter by category, rating, or geographic radius.

Commercial lead databases exist, but they charge per record or require annual contracts. A self-hosted scraper lets you control scope, freshness, and cost structure, as long as you accept the maintenance burden and compliance responsibility that comes with it.

What the Yellow Pages Scraper Repository Provides

The data-scrape/yellow-pages-scraper repository is an open-source Python project designed to extract public business listing data from Yellow Pages-style directories. It is not a managed API, and it does not include proxy infrastructure or anti-detection guarantees. What it does provide is:

• A structured request builder that targets search result pages.
• Selectors for common Yellow Pages fields: business name, address, phone, website, category, rating, review count, and listing URL.
• Output normalization into JSON lines or CSV.
• Configurable request delays and retry logic.

Because directory layouts change over time, the repository is maintained with updated selectors. You should verify the current README for the latest supported domains and field coverage before running a production job.

Verified Repository Links

• Yellow Pages Scraper for Python — open-source local business listing extraction.
• Google Maps Scraper for Local Business Data — complementary public business data from Google Maps.
• data-scrape GitHub Profile — additional open-source scrapers for sales and market research workflows.

Step-by-Step Workflow

1. Clone and Install

Start by cloning the repository and installing the documented dependencies. Most workflows require requests, beautifulsoup4, and lxml.

git clone https://github.com/data-scrape/yellow-pages-scraper.git
cd yellow-pages-scraper
pip install -r requirements.txt

If the repository does not include a requirements.txt, install the common dependencies listed in the README:

pip install requests beautifulsoup4 lxml pandas

2. Configure Search Parameters

Create a configuration file or set environment variables for the search you want to run. Typical parameters include:

• Search query (e.g., "plumbers", "dentists", "marketing agencies")
• Location (e.g., "New York, NY", "90210")
• Maximum pages to scrape
• Output format (JSON or CSV)

3. Run the Scraper

The repository usually includes a CLI entry point or a main script. Run it with your configured parameters and review the output file before any automated processing.

Runnable Python Example

Below is a representative Python workflow that shows how to structure a Yellow Pages scraping job using environment-based configuration. This example assumes you have reviewed the target directory's terms and robots directives.

import os
import csv
import time
import json
import requests
from bs4 import BeautifulSoup
from urllib.parse import urlencode

# Configuration via environment variables
SEARCH_QUERY = os.environ.get("YP_SEARCH", "plumbers")
LOCATION = os.environ.get("YP_LOCATION", "Chicago, IL")
MAX_PAGES = int(os.environ.get("YP_MAX_PAGES", "3"))
OUTPUT_FILE = os.environ.get("YP_OUTPUT", "yellow_pages_leads.jsonl")
REQUEST_DELAY = float(os.environ.get("YP_DELAY", "2.0"))

HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/120.0.0.0 Safari/537.36"
    ),
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Language": "en-US,en;q=0.5",
}


def parse_listing(card):
    """Extract fields from a single listing card element."""
    name = card.select_one(".business-name")
    phone = card.select_one(".phone")
    address = card.select_one(".street-address")
    locality = card.select_one(".locality")
    category = card.select_one(".categories")
    rating = card.select_one(".rating")
    review_count = card.select_one(".count")
    website = card.select_one(".links a[href]")

    return {
        "business_name": name.get_text(strip=True) if name else None,
        "phone": phone.get_text(strip=True) if phone else None,
        "address": " ".join(
            filter(
                None,
                [
                    address.get_text(strip=True) if address else None,
                    locality.get_text(strip=True) if locality else None,
                ],
            )
        ) or None,
        "category": category.get_text(strip=True) if category else None,
        "rating": rating.get_text(strip=True) if rating else None,
        "review_count": review_count.get_text(strip=True) if review_count else None,
        "website": website.get("href") if website else None,
        "source_url": None,  # populated per-page below
    }


def scrape_yellow_pages(query, location, max_pages=3):
    results = []
    for page in range(1, max_pages + 1):
        # Note: actual URL structure depends on the target directory domain.
        # Verify the current pattern in the repository README.
        params = {"search_terms": query, "geo_location_terms": location, "page": page}
        url = f"https://www.yellowpages.com/search?{urlencode(params)}"

        try:
            resp = requests.get(url, headers=HEADERS, timeout=30)
            resp.raise_for_status()
        except requests.RequestException as e:
            print(f"Request failed for page {page}: {e}")
            continue

        soup = BeautifulSoup(resp.text, "lxml")
        cards = soup.select(".result")  # selector varies by site; verify in repo

        if not cards:
            print(f"No listings found on page {page}; stopping.")
            break

        for card in cards:
            record = parse_listing(card)
            record["source_url"] = url
            results.append(record)

        print(f"Page {page}: extracted {len(cards)} listings.")
        time.sleep(REQUEST_DELAY)

    return results


def save_results(records, filepath):
    ext = os.path.splitext(filepath)[1].lower()
    if ext == ".csv":
        if not records:
            print("No records to save.")
            return
        keys = records[0].keys()
        with open(filepath, "w", newline="", encoding="utf-8") as f:
            writer = csv.DictWriter(f, fieldnames=keys)
            writer.writeheader()
            writer.writerows(records)
    else:
        with open(filepath, "w", encoding="utf-8") as f:
            for r in records:
                f.write(json.dumps(r, ensure_ascii=False) + "\n")
    print(f"Saved {len(records)} records to {filepath}")


if __name__ == "__main__":
    print(f"Searching: {SEARCH_QUERY} in {LOCATION}")
    data = scrape_yellow_pages(SEARCH_QUERY, LOCATION, MAX_PAGES)
    save_results(data, OUTPUT_FILE)

Important Notes on the Code

• The CSS selectors (.business-name, .phone, .result) are examples. The actual selectors depend on the current directory HTML structure and are maintained in the repository.
• The target URL pattern should be verified against the README; directory domains and path structures change.
• REQUEST_DELAY defaults to 2 seconds. Increase it if you encounter rate limiting.
• The script does not bypass CAPTCHAs, login walls, or anti-bot systems. If those appear, the repository may suggest additional headers or session handling.

Representative Output

A successful extraction produces records like this:

{
  "business_name": "Ace Plumbing Services",
  "phone": "[phone]",
  "address": "123 Main St, Chicago, IL",
  "category": "Plumbers",
  "rating": "4.5",
  "review_count": "27",
  "website": "https://aceplumbingexample.com",
  "source_url": "https://www.yellowpages.com/search?search_terms=plumbers&geo_location_terms=Chicago%2C+IL&page=1"
}

When saved as CSV, the same fields appear as columns. You can open the file in Excel, Google Sheets, or import it directly into a CRM like HubSpot or Salesforce using their standard contact/company import wizards.

Comparison: Build vs. Buy for Local Lead Data

Dimension	Self-Hosted Scraper (Repository)	Commercial Lead Database
Best for	Developers, agencies with custom filtering needs	Teams that want ready-made lists immediately
Setup model	Clone, configure, run locally or on a VPS	Web dashboard or API subscription
Data coverage	Public directory listings only; depends on target site	Often combines multiple sources, may include emails
Output format	JSON, CSV, or custom pipeline	Varies by provider; usually CSV or API JSON
Maintenance burden	High: selectors break when layouts change	Low: provider handles extraction infrastructure
Integration path	Direct file import or custom ETL pipeline	Native CRM integrations or REST API
Compliance responsibility	You must verify terms and robots directives	Provider typically manages legal framework
Pricing verification	Free (open source), but compute and proxy costs apply	Check current provider pricing page for tiers

Business Use Cases

Local Sales Prospecting

A B2B service company can scrape plumbers, electricians, or contractors in a target metro area and load the results into an outreach tool. The phone and address fields support direct mail or cold-call campaigns.

Competitive Analysis

A marketing agency can collect all restaurants or retail stores in a neighborhood to analyze category density, average ratings, and review volume. This informs pitch decks for local SEO services.

Data Enrichment

A CRM with partial company records can be enriched with public phone numbers, addresses, and website URLs from directory listings. This is especially useful for small businesses that do not have a LinkedIn presence.

Market Research

Researchers studying local economic activity can aggregate business counts by category and zip code to measure commercial vitality over time. For additional public-web-data research methodology, see the CoreClaw market research guide.

Limits, Compliance, and Maintenance

Rate Limiting and IP Blocks

Directory sites monitor traffic patterns. Running a scraper without delays from a single IP can trigger blocks. Use the built-in delay settings, and consider rotating IPs through a legitimate proxy provider if you need higher throughput.

Layout Fragility

HTML selectors break when a directory redesigns its result pages. The repository maintainers update selectors when possible, but you should expect to inspect the DOM and adjust CSS paths yourself between updates.

Terms of Service

Yellow Pages and similar directories publish terms that restrict automated access. Before running any scraper at scale, read the current terms and robots.txt of the target domain. This article discusses public data extraction; it does not encourage violating access controls or terms of service.

Data Quality

Public listings contain user-generated or self-reported information. Phone numbers may be outdated, addresses may reflect old locations, and categories may be inconsistently labeled. Always validate a sample before launching a campaign.

No Private Data

The repository extracts only public directory fields. Do not attempt to scrape private contact information, internal dashboards, or password-protected pages.

FAQ

Is there an official Yellow Pages API?
Some directories offer partner APIs or data feeds, but they are typically restricted to large partners or require application approval. The open-source scraper is an alternative for teams that cannot access an official API.

What data fields are returned?
Typically: business name, phone number, address, category, rating, review count, website URL, and the source listing URL. Exact field availability depends on the target directory page structure.

How often should I run the workflow?
For outreach lists, monthly refresh cycles are common. For competitive monitoring, weekly may be appropriate. Always respect the target site's load and do not scrape more frequently than necessary.

What happens when the page layout changes?
Extraction may return empty fields or fail entirely. Monitor output quality with validation checks (e.g., assert that at least 80% of records contain a business name). Update selectors and open an issue or PR on the repository if a fix is needed.

Can I connect this to a CRM or automation tool?
Yes. The JSON or CSV output can be imported into HubSpot, Salesforce, Pipedrive, or n8n. For automated pipelines, schedule the script with cron or a task runner, then push the output file through the CRM import API.

Should I use a proxy?
If you scrape more than a few hundred listings per day, a rotating residential or datacenter proxy reduces the chance of IP blocks. The repository does not include proxy configuration by default; add it through environment variables or a requests session adapter.

How does this compare to Google Maps data?
Yellow Pages and Google Maps overlap for many local businesses, but each source has unique listings and field coverage. The data-scrape/scrape-google-maps repository offers a complementary workflow if you want to merge both datasets for higher match rates.

Next Steps

If you are building a local lead generation pipeline, start with the data-scrape/yellow-pages-scraper repository. Review the README for current selectors, clone the project, configure your search parameters through environment variables, and run a small test batch before scaling.

For complementary local business data, also explore the Google Maps scraper. Both repositories are part of the open-source data-scrape collection maintained for developers and agencies who need structured public business data without managing proprietary APIs.