Extracting B2B Software Metadata via Shared Catalog Records

Building B2B competitive intelligence pipelines requires extracting consistent software catalog metrics across pricing models, platform dependencies, and core features. Bypassing front-end scraper protection on product aggregators directly often results in ip blocks or inconsistent DOM parsing. The Capterra Software Scraper handles this by querying the underlying Gartner Digital Markets infrastructure behind Capterra and GetApp, returning structured JSON metadata across product catalogs.

Resolving Capterra Products Without Direct HTML Scraping

Extracting metadata directly from Capterra product pages presents scraping challenges due to aggressive bot mitigation controls. The catalog data powering Capterra is shared across the Gartner Digital Markets network, including GetApp. Instead of parsing heavily protected Capterra HTML directly, this actor targets the shared catalog backend through the GetApp stack while returning URL attributes pointing back to original Capterra listings.

When executing queries, the actor uses a Google SERP proxy to resolve public product pages. For individual product detail extraction, it routes traffic through residential proxy rotation to ensure high retrieval rates without triggering rate limits.

The extracted output includes high-value fields such as startingPrice, pricingPlans, licensingModel, features, integrations, pros, cons, and vendor background data like vendorFoundedYear and vendorWebsite. Empty fields are omitted entirely rather than returned as null placeholders.

Configuring Modes for Direct URLs vs Search Queries

The actor operates in two distinct operational modes determined by the mode parameter: direct URL ingestion (byProductUrls) and discovery search (bySearchQueries).

Direct Extraction with byProductUrls

When specific software entities are already identified, pass explicit Capterra product URLs. This approach bypasses search resolution entirely, moving directly to catalog data extraction.

{
  "mode": "byProductUrls",
  "productUrls": [
    "https://www.capterra.com/p/135003/Slack/",
    "https://www.capterra.com/p/228385/Notion/"
  ],
  "maxItems": 2
}

Keyword Resolution with bySearchQueries

For exploratory collection or dynamic catalog building, supply targeted search strings. The actor resolves these broad queries against public product pages before pulling catalog details.

{
  "mode": "bySearchQueries",
  "searchQueries": [
    "Slack",
    "project management software"
  ],
  "maxItems": 10
}

The maxItems integer restricts the total number of processed products across the run execution, preventing unexpected dataset inflation.

Executing the Scraper Programmatically

You can invoke the scraper using the Apify Python SDK. Install the client library first:

pip install apify-client

This Python script initializes the client, configures search queries to capture project management tools, and extracts the results into a pandas DataFrame for analysis:

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_KEY")

run_input = {
    "mode": "bySearchQueries",
    "searchQueries": ["project management software"],
    "maxItems": 50,
}

# Run the actor and wait for completion
run = client.actor("crawlerbros/capterra-software-scraper").call(
    run_input=run_input
)

# Fetch results from the run's default dataset
dataset_items = client.dataset(run["defaultDatasetId"]).list_items().items

for item in dataset_items:
    name = item.get("name")
    rating = item.get("rating")
    starting_price = item.get("startingPrice")
    features = item.get("features", [])

    print(
        f"Product: {name} | Rating: {rating} | Starts at: {starting_price}"
    )
    print(f"Features count: {len(features)}\n")

Structure of the Extracted Catalog Schema

A successfully processed product record returns structured JSON containing company, technical, and commercial parameters:

{
  "productId": "135003",
  "slug": "Slack",
  "name": "Slack",
  "url": "https://www.capterra.com/p/135003/Slack/",
  "tagline": "Where work happens",
  "rating": 4.7,
  "reviewCount": 23000,
  "startingPrice": "8.75",
  "pricingModel": "Per User",
  "priceCurrency": "USD",
  "freeTrial": true,
  "freeVersion": true,
  "primaryCategory": "Collaboration Software",
  "categories": ["Team Communication", "Collaboration"],
  "features": ["Chat/Messaging", "File Sharing", "Video Conferencing"],
  "featureCount": 42,
  "integrationsCount": 2400,
  "pros": ["Ease of use", "Extensive integration library"],
  "cons": ["Search indexing limits on free plan"],
  "vendorName": "Slack Technologies",
  "vendorCountry": "United States"
}

If a product fails to resolve during the search phase, the actor skips emitting the record entirely. It does not generate synthetic or fallback placeholder items.

Cost Structure Analysis

Pricing for this actor operates exclusively under a Pay-Per-Event billing model. You pay strictly for runtime initialization and successfully dataset-emitted items.

• Actor Start Event (apify-actor-start): Flat $0.005 per GB of memory allocated to the run.
• Dataset Result Event (apify-default-dataset-item): $0.005 per emitted item at the base Free tier.

Volume discounts apply to result events for higher platform usage tiers:

• BRONZE: $0.00433 per item
• SILVER: $0.00367 per item
• GOLD / PLATINUM / DIAMOND: $0.003 per item

For example, a execution run configured with 1 GB of memory ($0.005) that extracts 100 resolved software products on the base tier ($0.005 x 100 = $0.50) incurs a total run cost of $0.505.

It is important to note that this actor does not scrape individual end-user review text strings or paginated category listing pages; it focuses exclusively on product-level catalog metadata.

---

Capterra Software Scraper is the Actor behind these examples. If a selector in your own version breaks, compare your output against the fields listed in its README first.