How to Build a Company Data Crawler in Python

“Data-driven decision making.”

Cool.

Now go find the data. 😐

That was one of the reasons I started building Company Data Crawler, an open-source Python library for collecting company information from public sources.

Give it a company name or stock ticker and get structured company data such as:

• Company details and industry
• Employees and key executives
• Funding and financial information
• Locations
• Social profiles
• Similar companies
• Operating metrics and more

The data is returned as a Pydantic model, so you get validated, typed data instead of another giant dictionary to deal with.

A simple example

from company_data_crawler import CompanyDataCrawler

crawler = CompanyDataCrawler(cache_dir="./cache")

# 1. Find a company by name
results = crawler.search_company("stripe", source="craft")
print(results[0].company_name)   # Stripe
print(results[0].source_url)     # https://craft.co/stripe

# 2. Scrape its public company page into a validated model
company = crawler.get_company_data(results[0].source_url, source="craft")

print(company.company_name)            # Stripe
print(company.company_domain)          # stripe.com
print(company.company_founded_year)    # 2010
print(company.company_funding_info)    # [CompanyFundingInfo(funding_amount=..., ...)]
print(company.company_locations)       # [CompanyLocation(city=..., is_headquarter=True), ...]
print(company.key_executives)          # [KeyExecutive(name=..., title=...), ...]

You can also search using a stock ticker:

company = crawler.get_company_data_by_symbol(
    "MSFT",
    source="craft"
)

Currently, it supports Craft and Owler, with more sources planned.

The collected data can also be:

• Stored in PostgreSQL or MongoDB
• Exported to JSON, CSV, Excel or Parquet
• Cached to avoid unnecessary requests
• Used for data pipelines, RAG, company research or AI agents

The idea is simple: make company data easier to collect and actually use.

Open source and available on PyPI.

GitHub: https://github.com/shaikhsajid1111/company-data-crawler

PyPI: https://pypi.org/project/company-data-crawler/