DEV Community

Gio Rich
Gio Rich

Posted on

Pull local business leads from YellowPages with Python

#webscraping #python #javascript #api

I built this actor; it's a paid tool on Apify with a free trial credit.

If you do web design, local marketing or B2B sales for small businesses, YellowPages.com is still one of the better public directories of US local businesses: plumbers, dentists, roofers, restaurants, with phone numbers and addresses. Copying them out by hand is slow, and the site sits behind Cloudflare, which blocks a lot of simple scripts.

I'm an 18-year-old engineering student, and I built YellowPages Scraper to turn a search into a CRM-ready list. This post covers the Python and Node calls, the output, and a project I think is the most useful one: finding local businesses that don't have a website.

How it works

Each search page is fetched over plain HTTP with a desktop browser fingerprint (no headless browser) and parsed with Cheerio. The result cards give name, phone, address, categories and ratings; the page's schema.org JSON-LD adds structured address parts and opening hours. Page 1's total count is used to fetch the remaining pages in parallel. When Cloudflare challenges a request, the actor retires that proxy session and retries on a fresh IP.

Python example 

pip install apify-client
Enter fullscreen mode Exit fullscreen mode 
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("rel8ble/yellowpages-scraper").call(run_input={
    "searchTerms": ["plumber", "hvac contractor"],
    "locations": ["Charlotte, NC", "Austin, TX"],
    "maxResults": 100,
})

for b in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(b["name"], b["phone"], b.get("city"), b.get("website"))
Enter fullscreen mode Exit fullscreen mode

Every search term is searched in every location, so the input above is 4 searches.

Node example 

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: "<YOUR_APIFY_TOKEN>" });

const run = await client.actor("rel8ble/yellowpages-scraper").call({
  searchTerms: ["dentist"],
  locations: ["28202"],
  maxResults: 50,
  includeDetails: true, // adds email, hours, social links, coordinates
  proxyConfiguration: { useApifyProxy: true, apifyProxyGroups: ["RESIDENTIAL"] },
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.filter((b) => b.email).length, "listings with an email");
Enter fullscreen mode Exit fullscreen mode

What comes back

A real row from a cloud run, "roofing contractors" in Charlotte, NC with includeDetails: true, trimmed:

{
  "searchTerm": "roofing contractors",
  "location": "Charlotte, NC",
  "name": "Roofing Contractors of America",
  "ypid": "520015737",
  "categories": ["Roofing Contractors", "General Contractors", "Fire & Water Damage Restoration"],
  "rating": 5,
  "reviewCount": 4,
  "phone": "(844) 251-9995",
  "street": "10130 Perimeter Pkwy",
  "city": "Charlotte",
  "state": "NC",
  "zip": "28216",
  "website": "http://roofingcontractorsofamerica.localsearch.com",
  "openStatus": "open 24 hours",
  "isServiceAreaOnly": false,
  "isClaimed": true,
  "email": "admin@rcacarolina.com",
  "paymentMethods": ["check", "financing available", "insurance", "visa", "mastercard"],
  "latitude": 35.347534,
  "longitude": -80.85175,
  "socialLinks": ["https://www.facebook.com/RCACarolina/", "https://www.instagram.com/rcacarolina/"]
}
Enter fullscreen mode Exit fullscreen mode

The address is split into street, city, state and zip so it imports cleanly into a CRM. Businesses that only list a service area ("Serving the Charlotte Area") are flagged with isServiceAreaOnly: true.

Use case: find local businesses with no website

For a web design or local SEO freelancer, a business with a phone number but no website is the obvious prospect. The website field is filled for about 94% of listings in my test run, so the remaining ~6% is exactly the list you want.

import csv
from apify_client import ApifyClient

TRADES = ["plumber", "electrician", "roofing contractors", "landscaping", "auto repair"]

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("rel8ble/yellowpages-scraper").call(run_input={
    "searchTerms": TRADES,
    "locations": ["Charlotte, NC"],
    "maxResults": 300,
})

seen = set()
with open("no-website-charlotte.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.writer(f)
    w.writerow(["name", "phone", "street", "city", "zip", "categories", "yearsInBusiness", "claimed", "ypUrl"])
    for b in client.dataset(run["defaultDatasetId"]).iterate_items():
        if b.get("website") or b.get("isSponsored") or b["ypid"] in seen:
            continue
        seen.add(b["ypid"])
        w.writerow([b["name"], b["phone"], b.get("street"), b.get("city"), b.get("zip"),
                    "; ".join(b.get("categories") or []), b.get("yearsInBusiness"),
                    b.get("isClaimed"), b["yellowPagesUrl"]])
Enter fullscreen mode Exit fullscreen mode

yearsInBusiness helps you prioritise: a plumber who's been around 20 years without a website is a different conversation than one who started last month. Unclaimed listings (isClaimed: false) are another signal that nobody is managing their online presence.

One honest note: if you contact these businesses, you're responsible for following anti-spam and privacy laws (CAN-SPAM, TCPA for calls and texts, and so on). Phone calls to a business line and a genuinely useful offer go a lot further than a blast.

What it costs

$1.50 per 1,000 results, one result = one unique business saved. Duplicates are removed before saving, so you only pay for unique businesses.

  • The 5-trade Charlotte job above (up to 1,500 listings) ≈ $2.25
  • 10,000 leads = $15
  • Apify's free plan gives $5 of monthly credit, about 3,300 businesses

Limits

  • US only. YellowPages.com covers US businesses; other countries' Yellow Pages sites aren't covered.
  • Up to ~3,000 results per search (about 100 pages of 30). Split big metros by suburb or ZIP.
  • Emails need includeDetails and only exist when the business published one. In my test, 57% of detail pages had an email.
  • Cloudflare. Search pages always got through in testing, usually after 1-4 retries. For includeDetails or big jobs, use the RESIDENTIAL proxy group: it loaded 100% of pages in testing vs ~75% of detail pages on datacenter proxies. A detail page that still fails keeps the listing and is marked with detailsError.
  • No full review texts, only the snippet from the results page.

Issues tab on the actor page if you hit something odd.

YellowPages Scraper on Apify

This article was drafted with AI and published by me.

Top comments (0)