DEV Community

Gio Rich
Gio Rich

Posted on

Mining Capterra reviews with Python for competitor research

#webscraping #python #javascript #api

I built this actor; it's a paid tool on Apify with a free trial credit.

Capterra reviews are one of the most honest sources of "what do people actually hate about this product". Every review has separate pros and cons fields, sub-ratings for ease of use, support, features and value, and sometimes a "reasons for switching" answer that tells you exactly why someone left a competitor. The catch is that capterra.com sits behind an aggressive Cloudflare challenge, so scraping it directly needs a real browser and fails a lot.

I'm an 18-year-old engineering student, and I built Capterra Reviews Scraper around a simpler route. This post shows how to call it, what you get back, and a small competitor-research script.

How it works

Capterra, GetApp and Software Advice are all owned by Gartner and share one review database. Software Advice serves those same reviews without the Cloudflare challenge, so the actor reads them there, over plain HTTP with no headless browser. Reviews keep their original Capterra IDs (like Capterra___7209621), so you can match them against what you see on capterra.com. It's about 25 reviews per request; 600 reviews took under 5 seconds in my tests.

Python example 

pip install apify-client
Enter fullscreen mode Exit fullscreen mode 
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("rel8ble/capterra-reviews-scraper").call(run_input={
    "products": ["https://www.capterra.com/p/135003/Slack/", "Asana"],
    "maxReviewsPerProduct": 200,
    "sortBy": "newest",
})

for r in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(r["productName"], r["rating"], r["title"])
Enter fullscreen mode Exit fullscreen mode

You can pass Capterra, GetApp or Software Advice URLs, or a product name. Names are matched exactly, never guessed, so URLs are the safest input.

Node example 

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: "<YOUR_APIFY_TOKEN>" });

const run = await client.actor("rel8ble/capterra-reviews-scraper").call({
  products: ["https://www.capterra.com/p/19319/Jira/"],
  maxReviewsPerProduct: 0, // every review matching the filters
  starRatings: ["1", "2"],
  companySizes: ["D", "E", "F"], // 51-1,000 employees
  reviewedAfter: "2025-01-01",
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.length, "negative reviews from mid-size companies");
Enter fullscreen mode Exit fullscreen mode

What comes back

A real row from a Slack run (newest first), trimmed:

{
  "productName": "Slack",
  "reviewId": "Capterra___7209621",
  "title": "Can’t work without",
  "rating": 4,
  "pros": "Its ease of use and flexibility. The way you can open a new chat and a channel to share with others is the best",
  "cons": "Just as a personal point of view, the UI is not as modern as other tools are made. Some changes could help improve.",
  "reasonsForSwitching": "Many features were under expensive plans",
  "easeOfUseRating": 5,
  "customerServiceRating": 3,
  "featuresRating": 5,
  "valueForMoneyRating": 4,
  "date": "2026-09-05",
  "reviewerName": "Lorenzo F.",
  "reviewerIndustry": "Computer Software",
  "reviewerCompanySize": "11-50 employees",
  "timeUsed": "More than 2 years",
  "usageFrequency": "Daily",
  "incentive": "Nominal gift (e.g. gift card)",
  "productOverallRating": 4.66,
  "productTotalReviews": 24209,
  "productRatingBreakdown": { "1star": 48, "2star": 127, "3star": 912, "4star": 5731, "5star": 17391 }
}
Enter fullscreen mode Exit fullscreen mode

The product summary (overall rating, total reviews, star breakdown, average sub-ratings) is repeated on every row, so you don't need a second table.

Use case: turn a competitor's bad reviews into a feature-gap list

Say you're building a project management tool and want to know what Jira and Asana users complain about. Pull the 1- and 2-star reviews, then count the words that show up most in the cons field.

import re
from collections import Counter
from apify_client import ApifyClient

STOP = set("the a an and or to of in is it for on with that this be are was not but you i my we they have has its at as can very too".split())

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("rel8ble/capterra-reviews-scraper").call(run_input={
    "products": ["https://www.capterra.com/p/19319/Jira/", "Asana"],
    "maxReviewsPerProduct": 0,
    "starRatings": ["1", "2"],
    "reviewedAfter": "2025-01-01",
})

words = Counter()
switch_reasons = []
for r in client.dataset(run["defaultDatasetId"]).iterate_items():
    text = (r.get("cons") or "").lower()
    words.update(w for w in re.findall(r"[a-z']+", text) if w not in STOP and len(w) > 2)
    if r.get("reasonsForSwitching"):
        switch_reasons.append((r["productName"], r["reasonsForSwitching"]))

print("Most common words in cons:")
for word, n in words.most_common(25):
    print(f"{n:>4}  {word}")

print("\nWhy people switched:")
for product, reason in switch_reasons[:20]:
    print(f"- [{product}] {reason}")
Enter fullscreen mode Exit fullscreen mode

Word counts are crude, but they point you at the clusters ("slow", "pricing", "notifications", "learning curve") worth reading in full. For something better, feed the cons text into an LLM or a topic model. The reviewerCompanySize and reviewerIndustry fields let you split complaints by who's making them.

For ongoing monitoring of your own product, schedule a daily run with sortBy: "newest" and reviewedAfter set to yesterday. With newest-first sorting the scraper stops as soon as it reaches older reviews.

What it costs

$1.50 per 1,000 reviews. No start fee. Reviews skipped by your filters, duplicates, and products that aren't found cost nothing.

  • The Jira + Asana negative-review pull is usually a few hundred reviews: well under $1
  • 10,000 reviews = $15
  • Apify's free plan gives $5 of monthly credit, about 3,300 reviews

Limits

  • Coverage is the Software Advice catalog. Most Capterra products are there (Slack, Asana, Jira, monday.com, Notion, QuickBooks Online, Docusign, Canva, Shopify all worked in testing). A few big vendors aren't, for example Salesforce Sales Cloud, HubSpot CRM, Zendesk and Mailchimp. Those return "product not found" with close matches, and you're not charged.
  • No reviewer job titles. The source doesn't publish them; you get industry and company size.
  • Reviewer names are first name + last initial, the way Capterra shows them.
  • A few older reviews come from GetApp or Software Advice rather than Capterra; reviewSource tells you which.

Issues tab on the actor page for bugs.

Capterra Reviews Scraper on Apify

This article was drafted with AI and published by me.

Top comments (0)