Disclaimer: This guide covers extracting publicly accessible data. Always review a site's robots.txt and Terms of Service before scraping.
TL;DR
To get structured Yellow Pages data via API, use the AlterLab Extract API to send a POST request containing the target URL and a JSON schema defining your desired fields. The API handles browser rendering and anti-bot bypass, returning a validated JSON object containing the business name, category, and contact details without requiring manual HTML parsing.
Why use Yellow Pages data?
Directory data is the foundation for several high-value engineering projects. Instead of treating the web as a series of HTML documents, treat it as a database where you can query specific business attributes.
Practical use cases include:
- LLM Training & RAG: Feeding local business directories into a Retrieval-Augmented Generation (RAG) pipeline to power AI agents that provide localized business recommendations.
- Market Analytics: Monitoring the density of specific business categories (e.g., "Plumbers in Austin") to identify underserved markets or competitive clusters.
- Lead Pipeline Automation: Automatically populating CRM systems with publicly listed business contact information to streamline B2B outreach.
What data can you extract?
When building a directory data pipeline, you should target fields that are publicly listed and consistent across the platform. By using a structured data API, you ensure these fields are typed correctly (e.g., strings for names, arrays for categories) rather than dealing with messy raw text.
Key extractable fields include:
- Business Name: The primary trading name of the entity.
- Description: The "About" section or short business bio.
- Category: The industry classification (e.g., "HVAC", "Legal Services").
- URL: The direct link to the business's own website.
- Contact Information: Publicly listed phone numbers and addresses.
The extraction approach
Traditional web scraping involves writing fragile CSS selectors or XPath queries. If the website changes a single <div> class or updates its layout, your entire pipeline breaks. This "brittle" approach requires constant maintenance and manual updates to your parsing logic.
A data API approach is fundamentally different. Instead of telling the system where the data is (selectors), you tell the system what the data is (schema). The API uses AI-powered extraction to locate the data regardless of the underlying HTML structure. This abstracts away the browser management, proxy rotation, and CAPTCHA solving, allowing you to focus on the data pipeline rather than the infrastructure.
Quick start with AlterLab Extract API
To begin, you will need an API key. If you are new to the platform, follow the Getting started guide to configure your environment. For detailed technical specifications, refer to the Extract API docs.
Python Implementation
The Python SDK allows you to define a schema and receive a typed response. The following example demonstrates how to extract business details from a specific listing.
```python title="extract_yellowpages-com.py" {5-12}
client = alterlab.Client("YOUR_API_KEY")
schema = {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "The name field"
},
"description": {
"type": "string",
"description": "The description field"
},
"category": {
"type": "string",
"description": "The category field"
},
"url": {
"type": "string",
"description": "The url field"
},
"contact": {
"type": "string",
"description": "The contact field"
}
}
}
result = client.extract(
url="https://yellowpages.com/example-page",
schema=schema,
)
print(result.data)
### cURL Implementation
For those integrating via a shell script or a different language, the REST API is the most direct route.
```bash title="Terminal"
curl -X POST https://api.alterlab.io/v1/extract \
-H "X-API-Key: YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://yellowpages.com/example-page",
"schema": {"properties": {"name": {"type": "string"}, "description": {"type": "string"}, "category": {"type": "string"}}}
}'
Define your schema
The schema is the most critical part of the request. AlterLab uses this schema to validate the output. If the AI cannot find a field, it returns null rather than guessing or returning irrelevant HTML.
When defining your schema, be specific in the description field. Instead of saying "the name", say "The primary trading name of the business as listed in the H1 header." This guidance improves extraction accuracy for complex directory pages.
Example JSON Output:
The API returns a clean JSON object that can be piped directly into a database or an AI model:
```json title="response.json"
{
"data": {
"name": "Elite Plumbing Services",
"description": "Specializing in emergency leak repair and pipe installation since 1998.",
"category": "Plumbers",
"url": "https://eliteplumbing.example.com",
"contact": "555-0123"
},
"cost": 1200
}
## Handle pagination and scale
Scaling from one page to ten thousand requires a shift from synchronous calls to asynchronous batch processing. When scraping directory listings, you will typically encounter pagination.
To scale efficiently:
1. **Collect URLs**: Use a initial pass to gather all listing URLs from the search results pages.
2. **Async Batching**: Send these URLs to the API in parallel.
3. **Cost Management**: Use the `/v1/estimate` endpoint to calculate costs before committing to large batches.
Costs are clamped between $0.001 and $0.50 per request. If you use your own LLM key (BYOK), you pay a flat orchestration fee of 300 µ¢; otherwise, the platform rate of 1000 µ¢ applies. See [AlterLab pricing](/pricing) for full details.
### Asynchronous Batch Example
For high-volume pipelines, use an asynchronous pattern to avoid blocking your main thread.
```python title="batch_extract.py" {8-15}
async def extract_batch(urls):
client = alterlab.Client("YOUR_API_KEY")
tasks = []
for url in urls:
# Assume schema is defined as in the previous example
tasks.append(client.extract_async(url=url, schema=schema))
results = await asyncio.gather(*tasks)
return [r.data for r in results]
urls = ["https://yellowpages.com/url1", "https://yellowpages.com/url2"]
# Run the batch
# data = asyncio.run(extract_batch(urls))
Key takeaways
- Stop parsing HTML: Use a data API to define the what, not the how.
- Schema Validation: Use JSON schemas to ensure your data pipeline receives typed, predictable output.
- Async for Scale: Use asynchronous requests and the cost estimation endpoint to manage high-volume directory extractions.
-
Compliance: Always respect
robots.txtand only extract publicly available data.
Top comments (1)
I appreciate how the article highlights the benefits of using a structured data API, such as AlterLab Extract API, to extract Yellow Pages data, particularly in handling browser rendering and anti-bot bypass. The example use cases, like LLM Training & RAG and Market Analytics, demonstrate the potential applications of this approach. I've worked on similar projects where traditional web scraping methods proved brittle and prone to breaking, so I can see the value in defining a schema to extract data regardless of the underlying HTML structure. Have you explored any techniques for handling variations in data quality or inconsistencies in the extracted fields, such as missing or duplicate values?