The Challenge of Sourcing Prospects
Every business grapples with the challenge of sourcing prospects. While there are tons of solutions on the market currently, each comes with its limitations. Learning curves, tool-specific constraints, and sometimes unmet expectations are common pitfalls.
Cost vs. Quality of Leads
Investing money in a business is obvious and all the costs can quickly add up, especially at the beginning. But it's the quality of leads you get in return that really counts. Outsourcing your customer sources raises questions: Is the quality of your leads a priority for them? Is the data fresh and relevant ?
From my perspective, the prospecting process should be transparent and hands-on. It's essential for entrepreneurs to trace back every customer's origin.
The Power of the Described System
Despite the challenges, the system I've elaborated below is able to yield over 10,000 quality contacts within an hour,but specifically tailored for the gym niche across several US States. And the good part is that it can be done to many niches that can be found on Google (as for the moment).
BEWARE
This tool can generate very high amount of prospects very fast. But the API I described uses Google API under the hood.
YOU NEED TO fire it up in portions, and set up a budget in your Google Cloud account, where you use your API. This is just a reminder for a future reference, in case you build this project yourself from the ground up.
Who Can Benefit?
Software developers making the shift to entrepreneurship or business-savvy individuals with a developer ally will find this guide invaluable. Not only will you acquire a leads-filled excel/csv file, but also a step-by-step guide on utilizing ChatGPT for efficient lead sourcing.
Seeking Ready-Made Prospects for Your Venture?
Exploring a new niche or aiming to expand your current one? I've put together some ready to use files for the gym and plumbing/foundation repair sectors on my website. In case you seek outside of these niches, fill out a straightforward form detailing your needs, and I'll promptly generate new prospects for you – completely free. Your growth is important to me, let's navigate it together.
Currently: The system gleans leads from Google, but I'm on the path to incorporating LinkedIn, Instagram, and more.
Essential Skills for Developers:
Proficiency In:
Docker
RabbitMQ
Microservice patterns: Understand their operations, interactions via message queues, and HTTP calls.
APIs
Familiarity With:
PHP & Symfony Framework
Python
Bash scripting
ChatGPT prompting
A Glimpse into the System's Architecture
The system functions through three primary stages.
Here is just a miniature, so we are on the same page.
For an intricate look, here's the full-resolution Miro board:
https://miro.com/app/board/uXjVMiSun6Y=/?share_link_id=994343205773
We'll methodically dissect each segment, beginning with the ChatGPT stage. Here, you'll master the art of prompting ChatGPT for diverse formatted outputs, extending to website content scraping. For the ensuing stages, I harness an API that is in the very core of the system and is responsible for getting business details from Google Maps.
Location Nuances: Many places often share identical names, a phenomenon evident in countries like the US.
API Endpoint Breakdown:
For clarity, let's decode the following API endpoint:
http://localhost:8080/leads/England/Aley+Green(Bedfordshire)?keyword=gym&keywords=fitness%20center
Where we have 3 moving parts (parameters) we need to provide:
- Country: England
- Place name & Administrative Area: Aley Green, Bedfordshire
- Search Criteria: Gyms and fitness centers
The result? A JSON array of venues fitting our specified criteria.
Automating the Process: How Far Have We Come?
Though we're yet to achieve full automation with this solution, the current system exemplifies rapid, formatted data extraction via ChatGPT.
Tapping into ChatGPT for Swift Data Formatting
I targeted the potent markets of the UK and US due to their immense opportunities for digital services.
So, the first prompt to chatGPT is:
as I didn't care about googling it, when I can have everything in one tool. As I was sure that counties is what England is build of, I asked:
And this gave me a good list of 48 counties.
Now. We obviously could ask now one by one:
But ChatGPT is not that good with extracting that long lists in structured format from it's "internals".
The task we gave - to generate a list of all cities will be tedious and will require a lot of asking to "generate more", it often freezes and crashes.
I tried to give this task to BARD, to see if it can generate such a comprehensive dataset from the get-go.
For this prompt:
BARD generated only a handful of places.
When nurtured, it again generated just a few examples, just take a look:
Maybe these responses could be made longer with a bit of prompt-engineering and generating more "few-shot" examples, but we won't eventually know whether or not AI did not skip some important places, which are filled with juicy prospects.
So, we need to move on.
Let me show you what approach worked out well in this situation and can be a good foundation for your experiments.
We will make ChatGPT just a part of data processing engine.
At this point, I would recommend ChatGPT Plus, the paid version. It's the only way ChatGPT can access the internet through plugins. While Google's BARD is an alternative, I won't delve into it in this piece. If you're not already using ChatGPT Plus, give it a whirl. For just $20/month, with some acclimatization, it can drastically reduce your workload.
Here I have my ChatGPT instance switched on with Plugins enabled.
The plugin we actually need to use is Webpilot. It allows ChatGPT to process data from websites(html), but as well text files, such as .pdf's.
In case you need a guide on how to set this up, pause reading and head off to:
https://devopscube.com/chatgpt-plugins/
What I did is I found a source of reputable information to give ChatGPT as a reference.
I used google, and for England, I found out that Wikipedia has a website for each of the counties, listing all the places worth interest in that particular county. As we know, Wikipedia is tried and tested tool, so I have no problem in trusting there are all the places. No need to worry that ChatGPT can screw us over. Nevermind,
For the first on our list, it's :
https://en.wikipedia.org/wiki/List_of_places_in_Bedfordshire
As we see, the template for these urls is:
https://en.wikipedia.org/wiki/List_of_places_in_<county>
When you actually visit this url, you'll see that the page is neatly structured. It lists every place in Bedfordshire county in alphabetical order.
There we go back to the ChatGPT and put this prompt there:
Any advanced, fancy prompt engineering is not needed here.
However it always pays off to describe the requirements in detail, even writing them down from 2 different angles and put at least one example of the output we want.
When AI understands us clearly enough, it will generate actual data we need in desired format at blazing speed.
.
Now, when ChatGPT machinery is properly set up for a job, we can easily provide the links to next counties and generate .csv files within minutes.
I provided a second link, for Berkshire this time and shortly after, I have a list od 369 places to scrape the prospects from.
Once you’ve done the task once, you can easily scale it, saving time and accelerating results.
INTERESTED IN MORE DETAILS ?
If you want the series to be continued, put the hand up and I will prepare the next parts.
Specifically, how to tie together this architecture between components written in two different programming languages - Python & PHP. I can dive into details and give you ready reciepes on how to connect 2 different systems via message queue.
Let me know in the comments.
Thank you.
Top comments (0)