DEV Community

Cover image for How to Benchmark CAPTCHA Solving Services (With Working Python Scripts)
HasData profile image Valentina Skakun
Valentina Skakun for HasData

Posted on

How to Benchmark CAPTCHA Solving Services (With Working Python Scripts)

#programming #python #hasdata #scraping

We tested 5 CAPTCHA solving APIs to measure actual performance. Here's how to run the same benchmarks yourself, with complete working code.

Why This Matters

CAPTCHA latency directly impacts automation throughput. A service that takes 20 seconds versus 6 seconds to solve Cloudflare Turnstile means 140 seconds of idle time per 10 requests. Success rates matter even more, a 70% rate means 30% of your budget goes to retries.

We measured both metrics across 5 services:

  • 2Captcha
  • CapMonster Cloud
  • Anti-Captcha
  • DeathByCaptcha
  • SolveCaptcha

Quick Setup

Install dependencies:

pip install 2captcha-python anticaptchaofficial capmonstercloudclient deathbycaptcha requests playwright
playwright install
Enter fullscreen mode Exit fullscreen mode

All examples below run 50 iterations and save timing data to text files. Adjust the range for faster testing:

for i in range(1, 50):  # Change to 10 instead of 50 for quick tests
Enter fullscreen mode Exit fullscreen mode

Example 1: Image CAPTCHA (2Captcha)

The simplest benchmark. Upload an image (save it to images subfolder in the project folder), get text back, measure time and accuracy.

import time
from twocaptcha import TwoCaptcha
from twocaptcha.api import ApiException

API_KEY = "YOUR-API-KEY"
IMAGE_FILE = "images/captcha.jpg"
CORRECT_ANSWER = "hwjrc"

solver = TwoCaptcha(API_KEY)
correct_count = 0

with open("image_results.txt", "w", encoding="utf-8") as file:
    for i in range(1, 50):
        start_time = time.perf_counter()

        try:
            result = solver.normal(IMAGE_FILE, numeric=0)
            end_time = time.perf_counter()
            elapsed = end_time - start_time

            answer = result["code"].strip()

            if answer.lower() == CORRECT_ANSWER.lower():
                correct_count += 1
                file.write(f"{i}. OK ({answer}) — {elapsed:.2f} sec\n")
            else:
                file.write(f"{i}. WRONG ({answer}) — {elapsed:.2f} sec\n")

            file.flush()

        except ApiException as e:
            end_time = time.perf_counter()
            elapsed = end_time - start_time
            file.write(f"{i}. API Error: {e} — {elapsed:.2f} sec\n")
            file.flush()
            continue

        time.sleep(2)  # Rate limiting

accuracy = (correct_count / 50) * 100
print(f"Accuracy: {accuracy:.2f}%")
Enter fullscreen mode Exit fullscreen mode

Output example:

1. OK (hwjrc) — 2.33 sec
2. OK (hwjrc) — 2.41 sec
3. WRONG (hwjrz) — 2.18 sec
...
Accuracy: 58.00%
Enter fullscreen mode Exit fullscreen mode

This pattern works for any CAPTCHA solving service with minimal changes. Just swap the SDK and adjust the API call.

Example 2: Cloudflare Turnstile with Browser Automation (Anti-Captcha)

More complex. We use Playwright to load the page, solve the CAPTCHA via API, inject the token, and submit.

import time
from anticaptchaofficial.turnstileproxyless import turnstileProxyless
from playwright.sync_api import sync_playwright

API_KEY = "YOUR-API-KEY"
SITE_KEY = "0x4AAAAAAAGlwMzq_9z6S9Mh"
URL = "https://clifford.io/demo/cloudflare-turnstile"

solver = turnstileProxyless()
solver.set_verbose(0)
solver.set_key(API_KEY)

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()

    with open("turnstile_results.txt", "w", encoding="utf-8") as file:
        for i in range(1, 50):
            page.goto(URL)
            start_time = time.perf_counter()

            try:
                solver.set_website_url(URL)
                solver.set_website_key(SITE_KEY)

                # Solve CAPTCHA via API
                token = solver.solve_and_return_solution()

                if token != 0:
                    end_time = time.perf_counter()
                    elapsed = end_time - start_time
                    file.write(f"{i}. Success — {elapsed:.2f} sec\n")
                    file.flush()

                    # Inject token into page
                    page.evaluate(
                        """
                        (token) => {
                            const input = document.querySelector('input[name="cf-turnstile-response"]');
                            if (input) input.value = token;
                        }
                        """,
                        token,
                    )

                    page.click("button[type='submit']")
                    page.wait_for_timeout(2000)
                else:
                    end_time = time.perf_counter()
                    elapsed = end_time - start_time
                    error_message = solver.get_error_description()
                    file.write(f"{i}. Solve error: {error_message} — {elapsed:.2f} sec\n")
                    file.flush()

            except Exception as e:
                end_time = time.perf_counter()
                elapsed = end_time - start_time
                file.write(f"{i}. Error: {str(e)} — {elapsed:.2f} sec\n")
                file.flush()

    browser.close()
Enter fullscreen mode Exit fullscreen mode

Key steps:

  1. Navigate to target page
  2. Extract site key (hardcoded here, but can be scraped)
  3. Send to CAPTCHA service API
  4. Wait for token (polling happens inside SDK)
  5. Inject token into DOM
  6. Submit form
  7. Log timing

This pattern applies to reCAPTCHA v2, Invisible, and other challenge types that require browser validation.

Example 3: Building Your Own Integration (SolveCaptcha)

Some services don't provide SDKs. Here's how to build a custom integration with manual polling:

import time
import requests
from playwright.sync_api import sync_playwright

API_KEY = "YOUR-API-KEY"
SITE_KEY = "6Le-wvkSAAAAAPBMRTvw0Q4Muexq9bi0DJwx_mJ-"
URL = "https://www.google.com/recaptcha/api2/demo"

def solve_captcha():
    # Step 1: Submit CAPTCHA task
    submit_url = "https://api.solvecaptcha.com/in.php"
    payload = {
        "key": API_KEY,
        "method": "userrecaptcha",
        "googlekey": SITE_KEY,
        "pageurl": URL,
        "json": 1
    }

    start = time.perf_counter()
    r = requests.post(submit_url, data=payload).json()

    if r.get("status") != 1:
        raise RuntimeError(f"Submit error: {r}")

    captcha_id = r["request"]

    # Step 2: Poll for solution
    result_url = "https://api.solvecaptcha.com/res.php"
    while True:
        time.sleep(5)  # Wait before polling

        res = requests.get(result_url, params={
            "key": API_KEY,
            "action": "get",
            "id": captcha_id,
            "json": 1
        }).json()

        if res.get("status") == 1:
            end = time.perf_counter()
            return res["request"], end - start

        if res.get("request") != "CAPCHA_NOT_READY":
            raise RuntimeError(f"Solve error: {res}")

# Use with Playwright
with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()

    page.goto(URL)
    token, elapsed = solve_captcha()

    print(f"Solved in {elapsed:.2f} seconds")

    # Inject token
    page.evaluate(f"""
        document.getElementById('g-recaptcha-response').value = '{token}';
    """)

    browser.close()
Enter fullscreen mode Exit fullscreen mode

This is more work than using an SDK, but gives you full control over:

  • Polling intervals
  • Timeout handling
  • Error retry logic
  • Cost tracking

Performance Results

Here's what we measured across 50 attempts per CAPTCHA type:

Service reCAPTCHA v2 Turnstile Image
CapMonster 32.23s 6.24s 3.28s
2Captcha 50.71s 16.96s 2.33s
Anti-Captcha 92.86s 20.33s 13.82s
SolveCaptcha 84.80s 17.24s 8.46s
DeathByCaptcha 34.54s 13.07s 2.13s

Key findings:

  • CapMonster is 3x faster on Turnstile (6.24s vs 20.33s for Anti-Captcha)
  • 2Captcha and DeathByCaptcha have the best Image CAPTCHA accuracies (58% and 64% vs 14% for CapMonster)
  • Success rates vary significantly - Anti-Captcha only achieves 86% on reCAPTCHA v2

Practical Tips

1. Python 3.13 Compatibility

DeathByCaptcha breaks on Python 3.13 due to removed imghdr module:

# Error you'll see:
ModuleNotFoundError: No module named 'imghdr'
Enter fullscreen mode Exit fullscreen mode

Fix: Use Python 3.11 or manually patch:

py -3.11 your_script.py
Enter fullscreen mode Exit fullscreen mode

2. Proper Error Handling

Always wrap API calls in try/except and log failures:

try:
    result = solver.solve()
    elapsed = time.perf_counter() - start
    file.write(f"{i}. Success — {elapsed:.2f}s\n")
except ApiException as e:
    file.write(f"{i}. API Error: {e}\n")
except Exception as e:
    file.write(f"{i}. Error: {e}\n")
Enter fullscreen mode Exit fullscreen mode

This prevents one failed solve from crashing your entire benchmark.

3. Rate Limiting

Add delays between requests to avoid hitting API rate limits:

time.sleep(2)  # 2 seconds between solves
Enter fullscreen mode Exit fullscreen mode

Most services allow 60-120 requests per minute. Check your provider's limits.

4. Audio CAPTCHA Support

Only 2Captcha handles Audio CAPTCHAs:

from twocaptcha import TwoCaptcha

solver = TwoCaptcha(API_KEY)
result = solver.audio("audio_file.mp3", "en")
answer = result["code"]
Enter fullscreen mode Exit fullscreen mode

Success rate: 27%. Not great, but better than 0% from all other services.

5. Cost Optimization

Track cost per successful solve:

total_cost = attempts * price_per_solve
successful_solves = sum(1 for r in results if r["success"])
cost_per_success = total_cost / successful_solves
Enter fullscreen mode Exit fullscreen mode

A cheaper service with 70% success rate may cost more than an expensive one with 100% success rate.

Decision Matrix

Choose based on your priority:

Speed? CapMonster (6.24s Turnstile, 32.23s reCAPTCHA v2)

Reliability? 2Captcha (100% success on major types, only Audio support)

Cost? SolveCaptcha ($0.55/1K reCAPTCHA v2, but no SDK)

Image accuracy? DeathByCaptcha (64% success rate)

Complete Benchmark

The full analysis includes:

  • Performance tables for all 6 CAPTCHA types
  • Cost per mille (CPM) calculations
  • Integration complexity breakdown
  • All test scripts on GitHub

Read the full benchmark with detailed results

Run It Yourself

Clone the benchmark repository. Install dependencies for your chosen service. Add your API key and run:

cd 2captcha
python image.py          # Test image recognition
python turnstile.py      # Test Cloudflare Turnstile
python recaptcha2.py     # Test reCAPTCHA v2
python geetest.py        # Test GeeTest v4
Enter fullscreen mode Exit fullscreen mode

Results save to .txt files.

Notes

Testing environment: Python 3.13.7, Playwright 1.40+, official SDKs where available.
Full benchmark data: Complete performance tables, cost breakdowns, and all test scripts available in our full analysis.
GitHub repository: All scripts are open source and ready to run, captcha-solver-benchmark
Join our web scraping community to discuss automation strategies, CAPTCHA solving, proxy management, and more:
HasData Discord

Have you benchmarked CAPTCHA services? What metrics matter most for your use case? Drop your experience in the comments 👇

Top comments (0)