Building a B2B Prospecting Agent in 15 Minutes with CrewAI and a Single API Call

#python #crewai #ai #agents

The problem

A B2B prospecting workflow usually looks the same regardless of industry: you have a list of target company domains, and for each one you need three things before you can write a useful outreach email — a way to contact them, a read on what they're running (so you can tailor the pitch), and a quick judgment call on whether they're even worth reaching out to.

Doing this by hand doesn't scale past a handful of leads. Doing it with a scraper you wrote yourself means dealing with SSRF-safe fetching, HTML parsing, and rate limits before you've written a single line of actual agent logic.

This walks through building a two-agent CrewAI crew that takes a list of company URLs and produces a prioritized, reasoned shortlist — using crewai-webmetadata-extractor to handle the extraction side, so the agents can focus on judgment instead of parsing.

Setup

pip install crewai crewai-webmetadata-extractor

You'll need two keys:

A free RapidAPI key for the Web Metadata & Contact Extractor API (1,000 requests/month, no card required)
An API key for whatever LLM provider you're running CrewAI's agents on (OpenAI, Anthropic, etc. — CrewAI handles the provider abstraction, not covered here)

export WEBMETADATA_API_KEY=your-rapidapi-key

The tools

crewai-webmetadata-extractor ships four ready-made BaseTool subclasses. For a prospecting crew, two matter most:

Tool	What it returns
`WebContactsTool`	Emails, phone numbers, and social links found on the page
`WebMetadataExtractTool`	Full picture: SEO/OpenGraph data, detected tech stack, security-headers grade, Schema.org data, links

Every tool returns a JSON string, and API errors come back as {"error": true, ...} instead of raising — so a bad URL in your lead list doesn't crash the whole crew run.

Building the crew

Two agents: one that gathers raw signal per company, one that turns that signal into a ranked, reasoned shortlist.

from crewai import Agent, Task, Crew, Process
from crewai_webmetadata_extractor import WebContactsTool, WebMetadataExtractTool

researcher = Agent(
    role="B2B Lead Researcher",
    goal="Extract contact information and technical footprint for a list of company websites",
    backstory=(
        "You investigate company websites to surface contact points and technology "
        "signals that a sales team can act on. You report facts, not conclusions."
    ),
    tools=[WebContactsTool(), WebMetadataExtractTool()],
    verbose=True,
)

qualifier = Agent(
    role="Sales Development Rep",
    goal="Turn raw research into a prioritized, reasoned outreach shortlist",
    backstory=(
        "You review research on prospective companies and decide who's worth "
        "contacting first, based on how good a fit their tech stack and public "
        "presence make them for our product."
    ),
    verbose=True,
)

target_urls = [
    "https://example-company-one.com",
    "https://example-company-two.com",
    "https://example-company-three.com",
]

research_task = Task(
    description=(
        f"For each of these URLs, extract public contact info and the detected tech "
        f"stack: {', '.join(target_urls)}. List what you found per company, including "
        f"any URL that returned no usable contact info."
    ),
    expected_output="A per-company breakdown of contacts found and tech stack detected.",
    agent=researcher,
)

qualify_task = Task(
    description=(
        "Using the research above, rank the companies from most to least worth "
        "contacting. Justify each ranking with a specific signal from the research "
        "(a detected technology, a missing security header, presence or absence of "
        "a direct contact channel) — not a generic guess."
    ),
    expected_output="A ranked list of companies with a one-line justification each.",
    agent=qualifier,
    context=[research_task],
)

crew = Crew(
    agents=[researcher, qualifier],
    tasks=[research_task, qualify_task],
    process=Process.sequential,
)

result = crew.kickoff()
print(result)

What the qualifier actually has to work with

The researcher's tool calls return structured JSON, not free text — which is what lets the qualifier reason about specific fields instead of vibes. A WebMetadataExtractTool call includes a tech_stack array (CMS, analytics, frameworks detected) and a graded security_headers object, so "rank by fit" can turn into something like "runs WordPress with no CMS-specific caching layer detected and a missing CSP header — a plausible fit for a dev/security retainer" instead of a made-up reason.

WebContactsTool's output is deliberately scoped to what's actually on the page (emails, phones, social links) — it's a raw signal for the qualifier to weigh, not a claim about verified company or people data.

Where this goes from here

The same two tools compose into other shapes: a WebSEOAuditTool pass turns this into an SEO-focused sales angle ("your homepage is missing H1 structure, here's a report"); dropping in WebMarkdownTool on the same URL list turns the researcher into the ingestion step for a RAG pipeline over your leads' own public content, instead of a prospecting crew.

The point of separating "extraction" from "judgment" into two agents (rather than one agent doing both) is that the researcher's output stays inspectable — you can log or cache the raw JSON independent of whatever the qualifier concludes from it, and swap the qualifier's prompt without re-running extraction.

DEV Community