New business license leads from city open data in Python

#python #opendata #sales #automation

A business that opened last week needs everything at once: a POS system, payroll, insurance, a bank account, a website, signage, a bookkeeper. Most cities record that moment, because the business has to get a licence or register for city tax before it trades. Chicago, New York City, Los Angeles, San Francisco, Seattle and several other cities publish those records as open data, updated daily or weekly.

The catch is that every city publishes a different dataset: different field names, different date meanings, renewals mixed in with new licences, and plenty of sole proprietors whose "business name" is just their own name and home address. The New Business Licenses Scraper reads 11 of these city datasets and returns one clean lead per new business. This post builds a daily new-business feed in Python.

Disclosure: I'm Aurelix, and I built the New Business Licenses Scraper tool. It's a paid tool (prices below).

What each record holds

business_name and dba_name (trade name), license_number, license_type as the city publishes it.
category: one of 15 normalised categories (Food and drink, Retail, Construction and trades, Beauty and personal care...), so one filter works in every city.
naics_code and naics_description where the city publishes them, and ownership_type in Seattle and Baton Rouge.
issue_date, expiration_date and status.
address, city, state, zip_code, country, neighborhood, coordinates, and phone where the city publishes a business phone (New York City, Seattle, New Orleans).
lead_key, a stable ID per business for dedupe and CRM joins.

Cities: Chicago, New York City, Los Angeles, San Francisco, Seattle, Denver, New Orleans, Baton Rouge, Norfolk, Edmonton and Calgary.

New restaurants and salons from the last 30 days

pip install apify-client

import csv
import os
from collections import Counter

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("aurelix/new-business-licenses").call(
    run_input={
        "cities": ["chicago", "san-francisco", "seattle"],
        "daysBack": 30,
        "categories": ["Food and drink", "Beauty and personal care"],
    }
)
rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())

with open("new_businesses.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.writer(f)
    w.writerow(["issued", "city", "business", "trade_name", "category", "licence", "address", "zip", "phone"])
    for r in rows:
        w.writerow([r["issue_date"], r["jurisdiction"], r["business_name"], r["dba_name"], r["category"],
                    r["license_type"], r["address"], r["zip_code"], r["phone"]])

print(len(rows), "new businesses")
print(Counter(r["jurisdiction"] for r in rows).most_common())

Each city has its own idea of "new": Chicago's date issued (new issues only, renewals left out), Los Angeles' and San Francisco's location start date, Seattle's licence start date, Calgary's and Edmonton's first issue date. All of them land in issue_date as YYYY-MM-DD.

A daily feed that never repeats a business

run = client.actor("aurelix/new-business-licenses").call(
    run_input={
        "cities": ["chicago", "nyc", "los-angeles", "san-francisco", "seattle"],
        "daysBack": 30,
        "incremental": True,
        "stateKey": "daily-new-businesses",
    }
)

With incremental on, the first run returns what matches in daysBack and quietly remembers the rest of the last 90 days. Each later run with the same stateKey reads the whole 90-day window again, and for Los Angeles, Denver and Baton Rouge the newest 5,000 accounts by account number whatever their date, and returns only businesses it has not returned before. That matters because several cities publish late: New York City runs about two weeks behind, and Los Angeles lists many new accounts with a start date months back. A plain date-range run misses those; the only-new feed picks them up the day they appear. Records a city publishes more than 90 days after their date are the exception (outside Los Angeles, Denver and Baton Rouge). A city whose portal is down is named in the run status and is not marked as read, so the next run picks it up.

Schedule it in Apify (or call it from cron), and send the new rows to Google Sheets, a CRM, Slack or an n8n workflow. To find each business's website and rating, pass business_name and the city to Google Maps Businesses.

What it leaves out on purpose

Many new licences belong to sole proprietors trading under their own name from home, so the tool is default-deny. Street address, phone and coordinates are returned only when the record is clearly a business: a legal name with a company form like LLC or Inc or a business word like Cleaning or Cafe, or an owner column that names a company. A sole proprietor whose legal name is their own name gets no address, even under a trade name like "Example Cleaning Service". Never for records the city marks as a sole proprietor, individual or home-based business, or for apartment addresses. Names that read as a person's name, family trusts and names holding an email or phone number are removed (a name check that errs on the side of removing), and owner-name, contact-person and mailing-address columns are never returned. Residential and short-term rental, adult, individual practitioner and liquor-only licences are left out. You get business leads, not a list of people.

Why this tool

Eleven US and Canadian cities in one run and one schema.
15 normalised categories on top of each city's own licence type.
Only-new mode for a daily feed, with late-publishing cities handled.
The run status names any city that hasn't published the days you asked for.
$2.50 per 1,000 leads ($1.75 on paid plans), cheaper than the leading tools.
What it doesn't cover: state company filings (LLC and corporation registrations) and email addresses.

Limits

Eleven cities. Other cities are added on request.
City licences, not state filings. A business shows up when the city licenses or registers it, which can be before or after it opens.
Publishing lag. Use the only-new mode: a plain date-range run misses records published after its look-back. Denver yields only a handful of new business licences a month (most of its data is rental licences).
Phones in three cities only. New York City, Seattle and New Orleans publish a business phone, and it is returned only for records that are clearly businesses.

Cost

You pay per lead returned plus a small start fee. Cities with nothing new, and businesses an earlier run already returned, are free. The current prices are on the Store page; Apify's free plan includes $5 of credit a month, about 1,900 leads.

DEV Community