Zenkdom

Posted on Oct 30 Edited on Oct 30

How to Scrape G2 & Capterra Reviews in Python Bypassing DataDome (2026 Guide)

#python #webscraping #datadome #tutorial

If you have ever tried extracting B2B SaaS reviews, product comparisons, or pricing tiers from G2.com or Capterra, you have almost certainly run headfirst into a wall of 403 Forbidden errors or an interactive DataDome slider puzzle.

In this guide, we will analyze:

What triggers DataDome's anti-bot detection on G2.
Why standard libraries like requests, BeautifulSoup, and vanilla Playwright fail.
How to extract clean review data and star ratings using Python.

1. What Makes DataDome So Difficult?

Unlike basic web scrapers that only check your User-Agent string, DataDome operates on multiple layers:

A. TLS / JA3 / JA4 Fingerprinting

Before your script even sends the HTTP GET request, your Python client performs a TLS handshake with G2's edge servers.

Python's default urllib3 (used by requests and httpx) uses system OpenSSL cipher suites in an order that screams "I am an automated Python script".
DataDome matches this against its signature database and drops the connection before returning any HTML.

B. Device & Mouse Telemetry

If you render with headless Chrome, DataDome inspects:

navigator.webdriver flags
chrome.runtime object bindings
Mouse coordinate curves (linear robotic movements vs authentic human Bezier curves)
Missing speech synthesis and WebGL vendor mismatches

2. The Traditional Approach (And Why It Costs So Much)

To solve this locally, developers typically stitch together:

patchright (stealth Playwright fork) to remove Chrome DevTools Protocol (CDP) leakages.
Residential proxy pools ($10-$15 per GB) because datacenter IPs (AWS, Hetzner, DO) are blacklisted by default.
CAPTCHA solving services ($2 per 1,000 solves) that add 15-20 seconds of latency.

Managing this infrastructure for high-volume extraction quickly turns into a full-time DevOps nightmare.

3. The Modern Solution: Automated Bypass via Parse Everything

Instead of hosting heavy headless browsers that consume 1.5 GB of RAM per worker, you can use Parse Everything. It handles JA3/JA4 TLS impersonation, Turnstile/DataDome challenge clearing, and residential IP routing behind a single API call.

Prerequisites:

Python 3.8+
A free API key from parse-everything.com (includes 1,000 free requests on sign-up).

pip install requests

4. The Complete Extraction Script

Here is a working Python script to extract product titles, review ratings, and review snippets from G2:

import requests
import json

API_KEY = "pe_live_YOUR_API_KEY"
TARGET_URL = "https://www.g2.com/products/slack/reviews"

payload = {
    "target_url": TARGET_URL,
    "render_js": True,           # Renders dynamic reviews DOM
    "auto_detect": True,         # Automatically solves DataDome challenges
    "country_code": "US",
    "selectors": {
        "product_name": "h1",
        "review_cards": ".nested-ajax-container",
        "review_titles": "h3.review-title",
        "review_body": ".review-body"
    }
}

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

print(f"[*] Extracting G2 reviews from {TARGET_URL}...")
response = requests.post(
    "https://parse-everything.com/api/v1/extract",
    headers=headers,
    json=payload,
    timeout=60
)

data = response.json()

if response.status_code == 200 and data.get("status") == "success":
    extracted = data["data"]
    print(f"[+] HTTP Status: {extracted['http_status']}")
    print(f"[+] Page Title: {extracted['page_title']}")
    print(f"[+] Protection Bypassed: {extracted['protection_bypassed']}")
    print(f"[+] Execution Time: {extracted['execution_time_seconds']}s")

    print("\n--- Structured Selectors Data ---")
    print(json.dumps(extracted.get("selectors_data", {}), indent=2))
else:
    print(f"[-] Failed: {data}")

5. Output Preview

{
  "http_status": 200,
  "page_title": "Slack Reviews 2026: Details, Pricing, & Features | G2",
  "protection_bypassed": "DataDome Device Telemetry & WAF",
  "execution_time_seconds": 4.82,
  "selectors_data": {
    "product_name": ["Slack Reviews and Pricing"],
    "review_titles": [
      "Indispensable tool for real-time remote collaboration",
      "Great communication app for agile engineering teams",
      "Essential daily messaging app with incredible integrations"
    ]
  }
}

Summary

Never send raw requests.get() to DataDome targets — you burn your IP reputation.
If running locally: use curl_cffi for TLS impersonation or patchright with residential proxies.
If running in production at scale: save developer hours by offloading anti-bot evasion to parse-everything.com.

DEV Community