DEV Community: Bob

why your reCAPTCHA v3 score is low

Bob — Sun, 14 Jun 2026 21:02:07 +0000

If your reCAPTCHA v3 score comes back at 0.1 or 0.3 no matter what solver you throw at it, the solver usually isn't the problem - your IP is. The v3 score is a risk verdict Google assigns mostly from network reputation, and no fingerprint trick overrides a blacklisted datacenter range. This post explains what the score measures, which parts a solver controls, and how to stop blaming the wrong layer.

what the v3 score actually is

reCAPTCHA v3 never shows a checkbox. It runs in the background, watches the session, and returns a float between 0.0 and 1.0 with every token. 1.0 means "almost certainly human", 0.0 means "almost certainly a bot".

The score is not pass/fail. The site owner picks the threshold, and you never see it. A login form might reject anything under 0.7; a low-stakes newsletter signup might accept 0.3. So "my score is 0.5" is meaningless until you know what the target site does with it.

That also means the same token can succeed on one site and bounce on another. You're not solving a captcha - you're trying to clear a threshold you can't read.

why the score is mostly your IP

Google scores the request, not just the widget. The single biggest input is the network the request comes from.

Rough tiers, from what's observable in practice:

Datacenter IPs (AWS, GCP, OVH, Hetzner ranges): capped low. Often 0.1-0.3 even with a flawless device fingerprint. Google maintains these ranges and discounts them by default.
Flagged or overused residential proxies: mid. If a proxy IP has been hammered by every scraper on the same provider, its reputation is already burned.
Clean residential / mobile IPs: this is where 0.7+ becomes reachable. Mobile carrier IPs (rotating behind CGNAT) tend to score highest because real humans share them.

This is why a solver that hits 0.7 on a clean residential proxy drops to 0.2 the moment you point it at a datacenter exit. Nothing about the client changed. The IP did.

If you take one thing from this post: fix the proxy before you blame the solver.

what a solver can and can't move

A solver controls the client side of the score. CapBypass sends each token with a coherent, real-device signal profile - a consistent device fingerprint plus human-shaped timing and a matching pageAction. That moves the client-derived portion of the score from "obvious automation" up to "ordinary user".

What no solver can do:

Launder a datacenter IP into a residential reputation.
Raise the score above whatever ceiling Google has already assigned your network range.
Change the site's threshold.

So the model is: the solver sets your ceiling, your proxy sets your floor. Bring a clean IP and the signal profile can actually reach it. Bring a burned datacenter IP and you've capped yourself before the first request.

solving v3 the right way

Because the IP is the dominant factor, you almost always want the proxied task type for v3, not proxyless. ReCaptchaV3Task requires you to pass your own proxy - and the gateway enforces it. Submit a proxied task with no proxy field and you get:

{ "errorId": 1, "errorCode": "PROXY_REQUIRED", "errorDescription": "..." }

That's deliberate. For v3 the proxy is part of the solve, not an afterthought. Here's the full create request with curl:

curl -s https://api.capbypass.pro/createTask \
  -H 'Content-Type: application/json' \
  -d '{
    "clientKey": "YOUR_API_KEY",
    "task": {
      "type": "ReCaptchaV3Task",
      "websiteURL": "https://example.com/login",
      "websiteKey": "6Lc...your-site-key",
      "pageAction": "login",
      "proxy": "host:port:user:pass"
    }
  }'

The proxy string is host:port:user:pass. pageAction must match the action the page actually fires - more on that below. The response hands back a task id:

{ "errorId": 0, "taskId": "5f9b...-uuid" }

Then poll getTaskResult until status flips to ready:

curl -s https://api.capbypass.pro/getTaskResult \
  -H 'Content-Type: application/json' \
  -d '{ "clientKey": "YOUR_API_KEY", "taskId": "5f9b...-uuid" }'

{
  "status": "ready",
  "solution": { "gRecaptchaResponse": "03AGdBq25..." }
}

gRecaptchaResponse is the token you submit to the target site exactly as a real client would - in the form field or request body the site expects.

The same flow in Python with the Python SDK:

import os
from capbypass import CapBypass

client = CapBypass(api_key=os.environ["CAPBYPASS_API_KEY"])

result = client.solve({
    "type": "ReCaptchaV3Task",
    "websiteURL": "https://example.com/login",
    "websiteKey": "6Lc...your-site-key",
    "pageAction": "login",
    "proxy": "host:port:user:pass",
})

token = result["solution"]["gRecaptchaResponse"]
# submit `token` as g-recaptcha-response in your login request

Prefer TypeScript or Go? The same task works through the TypeScript and Go SDKs.

If you genuinely can't supply a proxy, ReCaptchaV3TaskProxyLess routes through the CapBypass IP pool - but you inherit whatever reputation that pool's exit has for your target. For score-sensitive endpoints, a clean proxy you control beats a shared pool.

For the full task schema, including the Enterprise variants, see the reCAPTCHA v3 docs. Both createTask and getTaskResult are covered in the API reference.

Bonus: +5% credits on every top-up

New to CapBypass? Apply code WELCOME_2026 at checkout for an extra 5% in credits on every top-up, with no minimum and no expiry. Redeem it on the top-up page and run the ReCaptchaV3Task flow above on us.

debugging a score that won't go up

Work down this list before you open a support ticket:

Check the proxy class first. Run your proxy through an IP-reputation lookup. If it resolves to a datacenter ASN, that's your ceiling. Swap to clean residential or mobile and re-test before changing anything else.
Match pageAction to the real action. v3 scores are action-scoped. If the page fires grecaptcha.execute(siteKey, {action: 'login'}) and you submit pageAction: "homepage", the action mismatch alone drags the score down. Read the page's JS and copy the exact string.
Use the token once, fast. v3 tokens expire after ~2 minutes and are single-use. A token sitting in a queue for 90 seconds may verify as stale. Solve, then submit immediately.
Confirm v3 vs Enterprise. Enterprise sitekeys (loaded from enterprise.js / recaptcha/enterprise.js) need ReCaptchaV3EnterpriseTask, not ReCaptchaV3Task. Wrong family = wrong solve path.
Stop reusing one IP across thousands of solves. Even a clean residential IP degrades if you route every request through it. Rotate.

If you've cleared all five and still floor out, the target's threshold may simply be higher than any score that network can earn - at which point the fix is a better proxy, not a better solver.

faq

Is a higher score always better?
For you, yes - a higher score clears more thresholds. But you can't force an arbitrary score; you can only remove the things dragging it down. The realistic goal is "high enough for this site's threshold", not 1.0.

Why does the same setup score differently on two sites?
Each site sets its own threshold and fires its own action. Same token quality, different bar. Tune pageAction per site.

Will a proxyless task get me a good v3 score?
Sometimes, but you're gambling on the pool exit's reputation for your specific target. For score-sensitive flows, supply your own clean proxy with ReCaptchaV3Task.

Does the device fingerprint matter at all?
Yes - that's the half a solver controls. A broken or inconsistent fingerprint caps you low even on a great IP. CapBypass sends a coherent real-device fingerprint so the client side isn't the thing holding you back. The IP is the other half, and that half is on you.

Originally published on capbypass.pro. CapBypass is an AI-powered captcha-solving API for reCAPTCHA, hCaptcha, Cloudflare Turnstile and AWS WAF.

Bypass AWS WAF challenges in Python

Bob — Sun, 14 Jun 2026 21:02:03 +0000

AWS WAF drops a JavaScript challenge in front of a page before any real response comes back, and a plain Python request just gets the challenge HTML instead of your data. This post detects that challenge, solves it through CapBypass, and replays the returned cookie on the request - about 30 lines of Python.

what aws waf blocks

AWS WAF's challenge sits between your client and the origin. On a protected page you get an interstitial that runs challenge.js (a proof-of-work token generator) and only sets the real aws-waf-token cookie once the JavaScript finishes. A scraper that does not run that JS never gets the cookie, so every request loops back to the challenge.

The tell is in the response: a small HTML page that loads a script from *.token.awswaf.com, often with an HTTP 405 and a window.gokuProps object holding key, iv, and context.

detecting the challenge

Check the body for the AWS WAF markers before you trust a response:

def is_waf_challenge(resp):
    body = resp.text
    return (
        "token.awswaf.com" in body
        or "gokuProps" in body
        or 'awswaf' in resp.headers.get("server", "").lower()
    )

If is_waf_challenge is true, the page is gated and you need a solved cookie before retrying.

solving with capbypass

Send the target URL to CapBypass with the AntiAwsWafTaskProxyLess task. With auto-detection you only pass websiteURL - the solver visits the page, finds gokuProps and the challenge.js URL, and runs the challenge for you.

curl -s https://api.capbypass.pro/createTask \
  -H 'Content-Type: application/json' \
  -d '{
    "clientKey": "YOUR_API_KEY",
    "task": {
      "type": "AntiAwsWafTaskProxyLess",
      "websiteURL": "https://example.com/protected-page"
    }
  }'

{ "errorId": 0, "taskId": "5f9b...-uuid" }

Poll getTaskResult until status is ready:

{
  "status": "ready",
  "solution": {
    "cookie": "aws-waf-token=xxxxxxxx...",
    "token": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:...",
    "userAgent": "Mozilla/5.0 ...",
    "secChUa": "\"Chromium\";v=\"...\""
  }
}

The same flow in Python with the Python SDK:

import os
from capbypass import CapBypass

client = CapBypass(api_key=os.environ["CAPBYPASS_API_KEY"])

result = client.solve({
    "type": "AntiAwsWafTaskProxyLess",
    "websiteURL": "https://example.com/protected-page",
})

solution = result["solution"]
cookie = solution["cookie"]          # "aws-waf-token=..."
user_agent = solution["userAgent"]   # match this on your request

For the full parameter list (manual awsKey / awsIv / awsContext situations) see the AWS WAF docs and the API reference.

replaying the cookies

AWS WAF binds the token to the browser identity that solved it, so set the returned cookie and reuse the same userAgent on your retry. Match them or the origin re-challenges you.

import requests

session = requests.Session()
name, value = cookie.split("=", 1)
session.cookies.set(name, value, domain="example.com")
session.headers["User-Agent"] = user_agent

resp = session.get("https://example.com/protected-page")
# resp is now the real page, not the challenge

Keep the session alive and the cookie rides along on every subsequent request until it expires.

Bonus: +5% credits on every top-up

New to CapBypass? Apply code WELCOME_2026 at checkout for an extra 5% in credits on every top-up, with no minimum and no expiry. Redeem it on the top-up page and put it toward your first AntiAwsWafTaskProxyLess solves.

things that go wrong

User-Agent mismatch. The token is tied to the UA used during the solve. If your request UA differs from solution.userAgent, AWS WAF rejects the cookie. Always copy it over (and secChUa if you send client hints).
Token expired. aws-waf-token is short-lived. Solve, replay, and finish the request promptly instead of queueing the cookie for later.
Wrong domain on the cookie. Set the cookie on the exact host you are hitting, not a parent domain.
Visual CAPTCHA instead of JS challenge. Some WAF configs escalate to an image CAPTCHA; the solution then carries a captchaVoucher JWT rather than a plain cookie. Handle both keys.

faq

Do I need a proxy for AWS WAF?
Not with AntiAwsWafTaskProxyLess - it runs through the CapBypass pool. For IP-sensitive targets use AntiAwsWafTask and pass your own proxy as host:port:user:pass.

Why does the page still challenge me after I set the cookie?
Almost always a User-Agent mismatch. The token is bound to the UA from the solve, so send solution.userAgent on the same request that carries the cookie.

How long is the aws-waf-token valid?
It is short-lived (minutes). Treat it as single-session: solve, replay immediately, and re-solve when it expires rather than caching it.

What if there is no gokuProps on the page?
Pass just the websiteURL and let auto-detection find the challenge.js script. If the site hides it, supply awsChallengeJS (and awsKey / awsIv / awsContext) manually per the docs.

Originally published on capbypass.pro. CapBypass is an AI-powered captcha-solving API for reCAPTCHA, hCaptcha, Cloudflare Turnstile and AWS WAF.

reCAPTCHA error codes explained

Bob — Sun, 14 Jun 2026 21:01:59 +0000

When reCAPTCHA breaks, it rarely says why in plain language - you get a short code like invalid-input-response or a red "ERROR for site owner" banner. This post lists the codes you actually hit when automating, what each one means, and whether it is your token or your setup at fault.

where these codes surface

reCAPTCHA errors show up in two places:

Server-side, in the error-codes array of the siteverify response (https://www.google.com/recaptcha/api/siteverify). This is what the site you are hitting sees when it validates your token.
Client-side, as a rendered banner ("ERROR for site owner: Invalid site key") or a JS console message ("Cannot contact reCAPTCHA").

You usually only control the token you submit, so the useful split is: which codes mean a bad/expired token versus a misconfigured request.

the common codes

missing-input-response - no token was sent at all. Your g-recaptcha-response field is empty. Check you actually attached the solved token to the form/body.
invalid-input-response - the token is malformed, expired, or already used. v2/v3 tokens are single-use and expire in ~2 minutes. Solve, submit immediately, never reuse.
timeout-or-duplicate - the token was valid but submitted too late or twice. Same fix: one token, one request, fast.
missing-input-secret / invalid-input-secret - the site's own secret key is missing or wrong. This is server-side config on the target, not something your client can fix.
bad-request - the request to siteverify was malformed (wrong content type, garbled body).
browser-error / "Cannot contact reCAPTCHA" - the client could not reach Google. Usually a network/proxy issue on the solving side.

And the client-side banners:

"Invalid site key" - the websiteKey (siteKey) is wrong for that domain.
"Invalid domain for site key" - the siteKey is restricted to a domain you are not on.
"Invalid key type" - you are treating a v3 key as v2 or vice versa.

your fault vs the token's fault

Split the list before you debug:

Token quality / lifecycle (you can fix): missing-input-response, invalid-input-response, timeout-or-duplicate. Get a fresh token and submit it once, fast.
Key / domain config (the target's setup, or your task params): "Invalid site key", "Invalid domain", "Invalid key type", *-input-secret. Re-check the websiteKey and that you picked the right reCAPTCHA version.

A token that verifies but still gets you blocked is a different problem: a low v3 score. That is not an error code at all - the token is valid, the site just set a threshold you did not clear. See why your reCAPTCHA v3 score is low for that.

action mismatch (v3)

On reCAPTCHA v3, an action that does not match what the page expects quietly lowers your score (no error code, just a worse number). If the page calls grecaptcha.execute(siteKey, {action: 'login'}), your task must send the same:

curl -s https://api.capbypass.pro/createTask \
  -H 'Content-Type: application/json' \
  -d '{
    "clientKey": "YOUR_API_KEY",
    "task": {
      "type": "ReCaptchaV3Task",
      "websiteURL": "https://example.com/login",
      "websiteKey": "6Lc...your-site-key",
      "pageAction": "login",
      "proxy": "host:port:user:pass"
    }
  }'

solving cleanly with capbypass

A fresh, correctly-scoped token avoids the whole token-fault half of the list. Solve, read gRecaptchaResponse, submit it once:

import os
from capbypass import CapBypass

client = CapBypass(api_key=os.environ["CAPBYPASS_API_KEY"])

result = client.solve({
    "type": "ReCaptchaV2Task",
    "websiteURL": "https://example.com/login",
    "websiteKey": "6Lc...your-site-key",
    "proxy": "host:port:user:pass",
})

token = result["solution"]["gRecaptchaResponse"]
# attach as g-recaptcha-response, submit right away

CapBypass surfaces its own task-level problems through the errorCode field on the API response (for example ERROR_CAPTCHA_UNSOLVABLE, ERROR_ZERO_BALANCE, ERROR_INVALID_TASK_DATA) - distinct from Google's verify codes above. Full list in the API reference; task schemas in the reCAPTCHA v2 and reCAPTCHA v3 docs.

Bonus: +5% credits on every top-up

New to CapBypass? Apply code WELCOME_2026 at checkout for an extra 5% in credits on every top-up, with no minimum and no expiry. Redeem it on the top-up page and put it toward your first ReCaptchaV2Task solves.

faq

What does invalid-input-response mean?
The token you submitted is malformed, expired, or already used. reCAPTCHA tokens are single-use and expire in about 2 minutes - solve and submit in one quick step.

Is timeout-or-duplicate my fault?
Yes, usually. The token was fine but arrived too late or twice. Use each token exactly once and submit it immediately after solving.

Why do I see "Invalid site key"?
The websiteKey you passed does not match the one the page uses, or it is restricted to another domain. Re-read the siteKey from the live page and confirm the domain.

A token verifies but I still get blocked. Why?
That is not an error code - it is a low v3 score. The token is valid; the site's threshold is higher than the score your request earned. Improve the request (clean IP, matching action) rather than the token.

Originally published on capbypass.pro. CapBypass is an AI-powered captcha-solving API for reCAPTCHA, hCaptcha, Cloudflare Turnstile and AWS WAF.

Add captcha solving to a Python scraper

Bob — Sun, 14 Jun 2026 21:01:55 +0000

Your requests scraper runs fine until a target drops a reCAPTCHA on the login or search page, and then every response is the challenge instead of the data. This post wires CapBypass into a plain Python scraper: detect the captcha, solve it through the API, inject the token, and retry - no browser.

when your scraper hits a captcha

A captcha shows up as a page that is not your data: a reCAPTCHA widget (g-recaptcha div, a grecaptcha.execute call), an hCaptcha frame, or an AWS WAF challenge. With requests/httpx you cannot run the widget's JavaScript, so the form never gets a valid g-recaptcha-response and the server rejects the submit.

The fix is to get that token from a solving API and submit it yourself.

detecting it

Recognise the challenge deterministically before you waste a submit:

def needs_recaptcha(html: str) -> bool:
    return "g-recaptcha" in html or "grecaptcha.execute" in html

def site_key(html: str) -> str | None:
    import re
    m = re.search(r'data-sitekey="([^"]+)"', html)
    return m.group(1) if m else None

If needs_recaptcha is true, pull the data-sitekey and solve before submitting.

solving via capbypass

Send the site URL and key to CapBypass and read the token back. Use the Python SDK:

import os
from capbypass import CapBypass

solver = CapBypass(api_key=os.environ["CAPBYPASS_API_KEY"])

def solve_recaptcha_v2(url: str, key: str) -> str:
    result = solver.solve({
        "type": "ReCaptchaV2Task",
        "websiteURL": url,
        "websiteKey": key,
        "proxy": "host:port:user:pass",
    })
    return result["solution"]["gRecaptchaResponse"]

solve() handles the create-task / poll loop for you and returns once the token is ready.

injecting the token and retrying

reCAPTCHA tokens go into the form field named g-recaptcha-response. Add it to the POST body you were already sending:

import requests

session = requests.Session()

def login(url: str, username: str, password: str):
    page = session.get(url).text
    payload = {"username": username, "password": password}

    if needs_recaptcha(page):
        token = solve_recaptcha_v2(url, site_key(page))
        payload["g-recaptcha-response"] = token   # submit immediately

    return session.post(url, data=payload)

For challenge-based protection like AWS WAF the solution is a cookie, not a form field - set it on session.cookies and reuse the returned userAgent instead of adding a body field.

a retry pattern

Tokens are single-use and expire fast, so solve as late as possible (right before the submit) and retry once on a captcha response rather than reusing a stale token:

def submit_with_retry(url, payload, attempts=2):
    for _ in range(attempts):
        resp = session.post(url, data=payload)
        if not needs_recaptcha(resp.text):
            return resp
        payload["g-recaptcha-response"] = solve_recaptcha_v2(url, site_key(resp.text))
    return resp

Bonus: +5% credits on every top-up

New to CapBypass? Apply code WELCOME_2026 at checkout for an extra 5% in credits on every top-up, with no minimum and no expiry. Redeem it on the top-up page and put it toward your first ReCaptchaV2Task solves.

things that go wrong

Stale token. Solving at the top of the run and submitting minutes later fails with timeout-or-duplicate. Solve right before the submit.
Wrong site key. Read data-sitekey from the live page each run; do not hardcode it.
v3 instead of v2. If the page uses grecaptcha.execute(...) with an action, it is reCAPTCHA v3 - switch to ReCaptchaV3Task and pass the matching pageAction. See the reCAPTCHA v3 docs.

faq

Does this work with httpx instead of requests?
Yes. The flow is identical - swap requests.Session() for httpx.Client(); you still inject g-recaptcha-response into the POST data or set the cookie on the client.

Do I need a real browser?
No. The solving API runs the challenge for you; your scraper stays a plain HTTP client and just submits the returned token or cookie.

Where do I put the token?
In the form field g-recaptcha-response for reCAPTCHA/hCaptcha. For AWS WAF set the returned cookie on your session instead.

How do I avoid solving on every request?
Only solve when needs_recaptcha is true. Most requests in a session will not be challenged once you are past the gated page.

Originally published on capbypass.pro. CapBypass is an AI-powered captcha-solving API for reCAPTCHA, hCaptcha, Cloudflare Turnstile and AWS WAF.

Captcha solving with Playwright

Bob — Sun, 14 Jun 2026 21:01:51 +0000

Playwright drives a real browser, so people expect captchas to just work - but reCAPTCHA and hCaptcha still score the session and block automated traffic. This post keeps your Playwright flow and offloads only the captcha: detect it on the page, get a token from CapBypass over HTTP, inject it, and submit.

why playwright still hits captchas

A real browser clears the "can it run JavaScript" bar, but reCAPTCHA v3 and hCaptcha go further - they score behaviour and device signals. Automated Playwright sessions often score low enough to be challenged or silently rejected. You still end up needing a token the site will accept.

Rather than fight the in-page widget, solve it out of band and drop the token into the page Playwright already controls.

detecting the challenge on the page

Read the site key straight out of the DOM:

async def get_recaptcha_key(page):
    el = await page.query_selector("[data-sitekey]")
    return await el.get_attribute("data-sitekey") if el else None

If a key comes back, the page is gated and you solve before submitting.

solving via the api

Your Playwright script calls the CapBypass HTTP API for the token - no second browser, just a request. Use the Python SDK:

import os
from capbypass import CapBypass

solver = CapBypass(api_key=os.environ["CAPBYPASS_API_KEY"])

def solve(url: str, key: str) -> str:
    result = solver.solve({
        "type": "ReCaptchaV2Task",
        "websiteURL": url,
        "websiteKey": key,
        "proxy": "host:port:user:pass",
    })
    return result["solution"]["gRecaptchaResponse"]

injecting the token into the page

reCAPTCHA reads its answer from a hidden <textarea name="g-recaptcha-response">. Set it via page.evaluate, then submit the form as a user would:

async def pass_recaptcha(page, url):
    key = await get_recaptcha_key(page)
    token = solve(url, key)

    await page.evaluate(
        """(t) => {
            let ta = document.querySelector('textarea[name="g-recaptcha-response"]');
            if (!ta) {
                ta = document.createElement('textarea');
                ta.name = 'g-recaptcha-response';
                ta.style.display = 'none';
                document.body.appendChild(ta);
            }
            ta.value = t;
        }""",
        token,
    )

    await page.click("button[type=submit]")

For an invisible or callback-driven widget you may need to call the site's JS callback with the token instead of clicking submit - inspect what grecaptcha.render registered.

hcaptcha and v3

Same shape, different field/task:

hCaptcha: the token goes into textarea[name="h-captcha-response"]; solve with HCaptchaTaskProxyless.
reCAPTCHA v3: there is no checkbox - the page calls grecaptcha.execute(key, {action}). Solve with ReCaptchaV3Task and the matching pageAction, then feed the token wherever the page sends it. See the reCAPTCHA v3 docs.

Bonus: +5% credits on every top-up

New to CapBypass? Apply code WELCOME_2026 at checkout for an extra 5% in credits on every top-up, with no minimum and no expiry. Redeem it on the top-up page and put it toward your first ReCaptchaV2Task solves.

things that go wrong

Token set but nothing happens. The widget uses a JS callback, not a plain submit. Call the registered callback with the token instead of clicking.
Low v3 score even from a browser. Playwright sessions can score low; a clean proxy on the solve helps. See why your reCAPTCHA v3 score is low.
Stale token. Solve right before injecting; tokens expire in about 2 minutes.

faq

Does CapBypass run the browser for me?
No - you run Playwright. CapBypass solves the captcha over HTTP and returns a token; you inject it into your own page. It is a plain API call from inside your script.

Where do I put the token in Playwright?
For reCAPTCHA, into textarea[name="g-recaptcha-response"] via page.evaluate, then submit. For hCaptcha use h-captcha-response.

Why is my session still blocked with a valid token?
On reCAPTCHA v3 a valid token can still score below the site's threshold. Solve through a clean proxy and match the pageAction.

Playwright or just the HTTP API?
If the site works without a browser, the plain HTTP approach is cheaper and faster. Use Playwright only when the page genuinely needs a rendered session for the rest of the flow.

Originally published on capbypass.pro. CapBypass is an AI-powered captcha-solving API for reCAPTCHA, hCaptcha, Cloudflare Turnstile and AWS WAF.

Captcha solving with Scrapy

Bob — Sun, 14 Jun 2026 20:01:08 +0000

A Scrapy spider hums along until a target gates a page behind reCAPTCHA, and then your parse callbacks get challenge HTML instead of items. This post adds a downloader middleware that detects the captcha, solves it through CapBypass, and retries the request with the token - so your spiders stay clean.

where scrapy hits captchas

Scrapy issues plain HTTP requests, so any page protected by reCAPTCHA, hCaptcha, or AWS WAF comes back as the challenge, not your data. You do not want captcha logic scattered across every callback - the right place is a downloader middleware that inspects responses and re-issues the blocked ones once solved.

detecting in a middleware

A middleware's process_response sees every response. Flag the challenge and pull the site key:

import re

def is_recaptcha(response) -> bool:
    body = response.text
    return "g-recaptcha" in body or "grecaptcha.execute" in body

def site_key(response):
    m = re.search(r'data-sitekey="([^"]+)"', response.text)
    return m.group(1) if m else None

solving via capbypass

Wrap the solve in a small helper so the middleware stays readable. Use the Python SDK:

import os
from capbypass import CapBypass

_solver = CapBypass(api_key=os.environ["CAPBYPASS_API_KEY"])

def solve_token(url: str, key: str) -> str:
    result = _solver.solve({
        "type": "ReCaptchaV2Task",
        "websiteURL": url,
        "websiteKey": key,
        "proxy": "host:port:user:pass",
    })
    return result["solution"]["gRecaptchaResponse"]

the downloader middleware

Detect, solve, and re-issue the request with the token in the form body. Return the new request from process_response so Scrapy reschedules it:

from scrapy.http import FormRequest

class CaptchaMiddleware:
    def process_response(self, request, response, spider):
        if not is_recaptcha(response):
            return response

        key = site_key(response)
        if not key or request.meta.get("captcha_retried"):
            return response  # give up rather than loop forever

        token = solve_token(response.url, key)
        return FormRequest(
            url=response.url,
            formdata={**request.meta.get("formdata", {}),
                      "g-recaptcha-response": token},
            meta={**request.meta, "captcha_retried": True},
            dont_filter=True,
            callback=request.callback,
        )

Enable it in settings.py:

DOWNLOADER_MIDDLEWARES = {
    "myproject.middlewares.CaptchaMiddleware": 585,
}

The captcha_retried flag stops an infinite solve loop if the token is rejected; dont_filter=True lets the same URL through the dedupe filter on retry.

things that go wrong

Infinite retry loop. Without a captcha_retried flag a rejected token re-triggers the solve forever. Cap it at one retry.
Blocking call in the reactor. solve() is synchronous; for high concurrency run it in a thread pool or use the async API so you do not stall Scrapy's event loop.
AWS WAF, not reCAPTCHA. Challenge-based protection returns a cookie, not a form token - set it on the request cookies and reuse the userAgent. See the AWS WAF docs.

faq

Why a middleware instead of solving in the callback?
A downloader middleware sees every response in one place, so you detect and retry captchas centrally instead of repeating the logic in each spider callback.

Does solve() block the Scrapy reactor?
The synchronous solve() does. For concurrent spiders, offload it to a thread pool or use the async create-task / poll calls so the reactor keeps running.

How do I stop infinite retries?
Set a flag in request.meta (e.g. captcha_retried) and bail if it is already set, so a rejected token does not loop.

Does this handle hCaptcha too?
Yes - detect the hCaptcha frame, solve with HCaptchaTaskProxyless, and inject the token into h-captcha-response instead of g-recaptcha-response.

Originally published on capbypass.pro. CapBypass is an AI-powered captcha-solving API for reCAPTCHA, hCaptcha, Cloudflare Turnstile and AWS WAF.

How a captcha-solving API works

Bob — Sun, 14 Jun 2026 19:56:43 +0000

A captcha-solving API turns "a human has to click this" into a normal HTTP call: you describe the challenge, the service solves it, and you get back a token or cookie you submit like a real browser would. This post walks the model end to end so you know what you are actually paying for.

why captchas block automation

reCAPTCHA, hCaptcha, AWS WAF, and similar systems exist to tell a script apart from a person. They run JavaScript, score behaviour, and hand the page a token only once they are satisfied. A plain HTTP client never runs that JavaScript, so it never earns the token - and the protected action (login, checkout, search) stays closed.

A solving API closes that gap without you driving a full browser yourself.

the create-task / poll / token model

Almost every solving API (CapBypass included) uses the same two-step async shape:

createTask - you POST a description of the challenge (type, site URL, site key). You get a taskId back immediately.
getTaskResult - you poll that taskId until status flips to ready, then read the solution.

# 1. create
curl -s https://api.capbypass.pro/createTask \
  -H 'Content-Type: application/json' \
  -d '{ "clientKey": "YOUR_API_KEY",
        "task": { "type": "ReCaptchaV2Task",
                  "websiteURL": "https://example.com/login",
                  "websiteKey": "6Lc...",
                  "proxy": "host:port:user:pass" } }'
# -> { "errorId": 0, "taskId": "5f9b...-uuid" }

# 2. poll
curl -s https://api.capbypass.pro/getTaskResult \
  -H 'Content-Type: application/json' \
  -d '{ "clientKey": "YOUR_API_KEY", "taskId": "5f9b...-uuid" }'
# -> { "status": "ready", "solution": { "gRecaptchaResponse": "03AGdBq25..." } }

It is async because solving takes a few seconds, and polling keeps your code simple: fire, wait, read.

what happens server-side

While you poll, the service does the work you would otherwise have to: it runs the challenge's JavaScript, presents a coherent device fingerprint, routes through an appropriate IP, and produces a token the target will accept. Good APIs also hand back the userAgent (and client-hint headers) they used, so you can match them on your own request and keep the identity consistent.

You never see any of that machinery - you see a taskId and, a few seconds later, a solution.

token captchas vs cookie captchas

The solution shape depends on the challenge family:

Token-based (reCAPTCHA, hCaptcha): you get a token (gRecaptchaResponse) that you inject into the form field or request body the site checks.
Cookie / challenge-based (AWS WAF, Cloudflare-style): you get a cookie string (e.g. aws-waf-token=...) that you set on your HTTP session before retrying the original request.

Same flow, different thing to replay. The API reference lists every solution field per task type.

what you pay for

Solving APIs bill per solved task, not per month - a few hundredths of a cent to a few cents depending on the captcha type. You pay only when a task returns ready with a usable solution, which is why the model scales cleanly from a handful of solves to millions. New accounts start with the getting-started flow and a small balance.

faq

Is it just OCR?
No. Modern captchas (reCAPTCHA v3, AWS WAF) are not images to read - they are JavaScript challenges and behavioural scores. A solving API runs the challenge and returns a token, which is very different from reading text off a picture.

How long does a solve take?
Typically a few seconds. That is why the API is async: you create a task, poll getTaskResult, and read the solution when status is ready.

Do I still need a proxy?
For IP-sensitive captchas (reCAPTCHA v3, AWS WAF on strict targets), yes - you pass your own proxy so the solve happens from a clean IP. Proxyless task variants use a shared pool and suit lower-stakes targets.

What do I do with the result?
Token captchas: inject the token into the form/body. Cookie captchas: set the returned cookie on your HTTP session, then replay the original request.

Originally published on capbypass.pro. CapBypass is an AI-powered captcha-solving API for reCAPTCHA, hCaptcha, Cloudflare Turnstile and AWS WAF.