DEV Community

David Black
David Black

Posted on

Seeding a local directory with real listings on day one

Nobody adds their business to an empty directory, and nobody searches one twice. So a local directory has to open with listings already in it. Here is how to fill one with real businesses before launch, without inventing anything.

1. Count before you build

Pick the unit first: one trade across a set of cities, or every trade in one town. Then count how many real businesses exist in that unit. If "tree surgeons in ten cities" comes to 150 businesses, you have a directory. If it comes to twelve, you have a blog post.

A rough count takes an hour and saves you from building pages for cities that will hold one listing each.

2. Choose a source you are allowed to use

There are three honest ones.

  • Google Places API. The widest coverage, with phone numbers, websites and map positions. It is paid per request, and its terms limit what you may store and for how long, so read them before you design around it.
  • OpenStreetMap. Free and openly licensed (ODbL). You must credit it, and coverage is uneven: good for shops and clinics with premises, thin for trades that work from a van.
  • Public registers. Licensing boards, trade associations and council lists. Slow to collect, but often the only complete source for a regulated trade.

Copying another directory is not on the list. Their descriptions are their copyright, and their mistakes become yours.

3. Make one CSV the hand-off between every step

Whatever the source, land everything in one plain file with fixed column names. A workable minimum:

business_name,city,country,website,phone,street_address,postal_code,latitude,longitude,category_1,description
Enter fullscreen mode Exit fullscreen mode

Only the name and the city are essential. The point of the file is that you can open it in a spreadsheet between steps, fix what is wrong and carry on. A pipeline you cannot inspect will import its errors at full speed.

4. Clean it before it goes anywhere

  • Remove duplicates. Match on the website's domain first, then on name plus city. The same business often appears under two spellings.
  • Decide about chains. A national chain with forty branches can swamp a niche directory. Keep them or drop them on purpose.
  • Drop the dead. A listing whose website no longer loads is usually a business that has closed.
  • Assign categories by rule. "Found by the search phrase tree removal" is a reliable reason to file a business under tree removal. A guess from the business name is not.

5. Write your own descriptions

Do not paste the text from each business's own site. It is theirs, and a page that repeats what is already on the web gives a search engine no reason to show yours. Write two or three plain sentences per listing from the facts you hold: what they do, where, and anything that sets them apart. If you use a language model for this, give it only those facts and check a sample by hand.

6. Fill in missing map positions

Rows from a places API arrive with latitude and longitude. Rows from a register do not. OpenStreetMap's Nominatim service will look addresses up for free, at no more than one request a second and with a user agent that says who you are.

7. Import as pending, then approve

Load the file with every listing held back, read a sample of fifty, and only then make them public. Skip duplicates on import so that fixing ten bad rows and loading the file again adds ten listings, not a second copy of everything.

Once they are live, give owners a way to claim and correct their listing. They know their opening hours better than any source you used.

What not to do

  • Do not publish a city page with nothing on it. Leave the city out until it has listings.
  • Do not invent businesses, reviews or ratings to make the place look busy.
  • Do not email the businesses you imported unless the law where they are allows it.

Where this comes from

I build DirectoryLab, a set of self-hosted directory kits. Its local kit runs these steps from the admin screens, and the page that explains how the data pipeline works, including which steps have to run on your own computer, is public whether or not you use the kit.

Top comments (0)