DEV Community: jujucodings

Cloudscraper Not Working in 2026? Here's Why, and the Fix

jujucodings — Wed, 09 Sep 2026 06:09:22 +0000

You pip-installed cloudscraper, pointed it at a Cloudflare site, and got a 403, a 503, or an endless "Just a moment…" page instead of your data. It's not your code. cloudscraper stopped being able to solve most Cloudflare sites a while ago, and here's the precise reason, plus a fix that keeps your existing requests-style code.

The 40-word version

cloudscraper only ever solved Cloudflare's old JavaScript-math challenge. Modern sites use Turnstile or the managed 5-second challenge, which run real browser JavaScript. cloudscraper has no browser and no JS engine, so it can't solve them. It just hands you the challenge page.

Why cloudscraper breaks now

It solved one specific thing: the legacy IUAM challenge. cloudscraper reads Cloudflare's old "I'm Under Attack" page, extracts a JavaScript math problem, and evaluates it. That's the whole mechanism.
It has no browser. Turnstile and the managed challenge execute obfuscated JavaScript and collect behavioral and fingerprint signals. There's nothing for a regex-and-eval tool to compute, so the request stays on the challenge page.
Its "Turnstile support" was always a hand-off. cloudscraper never solved Turnstile itself. It added hooks to pass the job to a paid CAPTCHA API and inject the returned token. Without one wired up, that does nothing.
It's effectively unmaintained. The last real release was in 2023. Cloudflare has shipped a lot of detection changes since.

How to confirm that's what's happening

Look at the body you're getting back, not just the status code:

resp = scraper.get("https://protected.example/")
print(resp.status_code)          # 403 / 503
print("cf-turnstile" in resp.text or "Just a moment" in resp.text)  # True -> it's the challenge

If that's True, you received the challenge page, not a block. You don't need better headers or a new user-agent. You need something that can actually pass Turnstile.

The fix: change one import

We built cloudscraper-turnstile for exactly this. It subclasses requests.Session the same way cloudscraper does and mirrors its API, so migration is one line. When it hits a Turnstile or 5s challenge, it solves it through Peak and retries the request transparently.

pip install cloudscraper-turnstile
export PEAK_API_KEY=pk_your_key

import cloudscraper_turnstile as cloudscraper   # the only line that changes

scraper = cloudscraper.create_scraper()
resp = scraper.get("https://protected.example/")   # Turnstile solved, real page returned
print(resp.status_code)   # 200

Your existing create_scraper keyword arguments (browser, delay, sess, and the rest) are accepted and don't raise, so you don't have to rewrite the rest of your call. Cookies from the solve, including cf_clearance, persist on the session for every later request.

Use your own proxy

Cloudflare ties clearance to the requesting IP, so pass a proxy and it's used for the solve too:

scraper = cloudscraper.create_scraper(
    api_key="pk_your_key",
    proxy="http://user:pass@ip:port",
)

What it costs

You pay only for solves that land (failed solves are free), starting at $0.90 per 1,000 and dropping to $0.35 at volume. New accounts get 1,000 free solves with no card, so you can run it against your own target before deciding.

If you're not on cloudscraper

Same idea, different library: scrapy-turnstile, selenium-turnstile, playwright-turnstile, and turnstile-curl for curl_cffi. All of them wrap the same one-call Peak API.

Get a free key · Docs · Source on GitHub

Use this for automation, QA, and scraping public data you're allowed to access. Respect each site's Terms of Service and robots.txt, and rate-limit yourself.

Cloudflare Turnstile 403: Why Your Scraper Is Blocked (and How to Fix It)

jujucodings — Tue, 08 Sep 2026 14:25:29 +0000

You point your scraper at a page and instead of HTML you get a 403. Or worse: the Turnstile widget loads, spins, and never resolves — an infinite loop that eats your timeout budget and returns nothing. You swap proxies. You bolt on stealth plugins. Same wall.

Here's what's actually happening, and the one thing that reliably gets you past it.

Why a real browser still fails

Turnstile isn't a checkbox you click. It's a background challenge that scores the request across two axes: the browser environment (canvas, WebGL, timing, a hundred small signals) and the IP reputation. Get either wrong and the token never issues — the widget just retries, which is the loop you're staring at.

Automation frameworks fix the first axis reasonably well now. Playwright, Selenium with undetected-chromedriver, camoufox, botasaurus — they drive a real Chromium with plausible fingerprints. But they can't fix the second one. Your traffic is coming from a datacenter IP (AWS, GCP, a cheap VPS, or a datacenter proxy), and Cloudflare has known those ranges for years. A perfect browser on a burned IP still scores as a bot. That's the 403.

So the two dead ends people burn days on:

More stealth. Helps the fingerprint, does nothing for the IP. You'll shave the block rate, not remove it.
More proxies. Rotating through more datacenter IPs is rotating through more IPs Cloudflare already distrusts. Residential helps, but it's expensive and you're still solving the challenge in-browser, which is slow and brittle.

The fix: get the token out-of-band, then inject it

You don't need your scraper's browser to pass Turnstile. You need a valid cf-turnstile-response token for that sitekey, from somewhere that scores clean. Get the token from a solving API, drop it into the response field, and continue.

The token is what the site actually validates server-side (via siteverify), and it isn't bound to your scraper's IP on most deployments — which is exactly why this works. One request in, a token out, keep moving:

import requests

# 1. Read the sitekey off the page (the data-sitekey attribute on .cf-turnstile)
sitekey = "0x4AAAAAAA..."
target  = "https://the-site-you-are-scraping.com/"

# 2. Solve it via Peak — one call, token back in about a second
resp = requests.post(
    "https://api.peak.fo/solve",
    headers={"X-API-Key": "pk_your_api_key"},
    json={"task_type": "TurnstileTaskProxyLess", "url": target, "sitekey": sitekey},
    timeout=60,
)
token = resp.json()["data"]["token"]

# 3. Submit the token with your request (form field, or inject into the widget)
#    e.g. include cf-turnstile-response=<token> in the POST body / headers the site expects

That's the whole pattern. TurnstileTaskProxyLess lets the solver supply a clean exit IP so you don't have to bring one. If the target binds the token to the requesting IP (rare, but it happens on hardened setups), switch to task_type: "turnstiletask" and pass your own proxy so the solve and your request share an address.

If you're already in a framework

You usually don't have to hand-roll the request. Drop-in wrappers keep your existing code and just make the 403 go away:

scrapy-turnstile — a Scrapy middleware
playwright-turnstile and selenium-turnstile
cloudscraper-turnstile and turnstile-curl (curl_cffi, no browser)
crawl4ai-turnstile for AI-scraping pipelines

FAQ

Why does the Turnstile widget loop forever instead of erroring?

Because the challenge is retrying, not failing outright. Cloudflare scored the request too low to issue a token but not low enough to hard-block, so the widget keeps re-attempting. From your side it looks like a hang; underneath, it's a soft fail on IP or fingerprint.

Is the token tied to my IP?

On most Turnstile deployments, no — the site validates the token against the sitekey and hostname, not the solving IP, which is why a token solved elsewhere still passes. A minority of hardened setups do bind it; for those, solve through the same proxy you'll use for the request.

Do I pay for failed solves?

Not with Peak — you're billed only when a valid token comes back. Turnstile is $0.90 per 1,000 (dropping to $0.35 at volume), token in about a second. See the Turnstile docs.

Grab a free key — 1,000 solves, no card — and run it against the site that's 403-ing you.

For legitimate automation, QA, and scraping of data you're authorized to access. Respect each site's Terms of Service and robots.txt.

The Cloudflare 5-Second Challenge, Explained (and how to pass it)

jujucodings — Tue, 08 Sep 2026 12:34:59 +0000

You request a page and Cloudflare hands you a spinner instead. "Checking your browser before you access the site." A few seconds later it either lets you through or loops you back to the spinner forever. That interstitial is the Cloudflare 5-second challenge, and whether you clear it comes down to what your client looks like, not how long you wait.

The 5-second challenge (Cloudflare calls it a managed challenge, or the older JS challenge) is a full-page interstitial that runs JavaScript in your browser to score whether you're a real visitor. Pass it and Cloudflare sets a cf_clearance cookie so it stops re-challenging you. Fail it and you never reach the actual site.

How it's different from Turnstile

People mix these up constantly, so it's worth being precise. Turnstile is a widget embedded in a form; it returns a token you submit. The 5-second challenge is a full-page gate in front of the whole site; it returns a cookie you reuse on later requests. Different mechanism, different output, different handling.

	Turnstile widget	5-second challenge
Where it appears	Inside a form (login, signup)	Full page, before you see the site
What it returns	A cf-turnstile-response token	A cf_clearance cookie
How you use it	Submit the token with the form	Send the cookie on subsequent requests
Lifetime	Single-use, ~300s	Session-length, tuned per site

If it's the widget you're facing, go to what is Cloudflare Turnstile instead. This page is about the interstitial.

What the challenge actually checks

During those seconds, Cloudflare runs a JavaScript workload that probes your environment: the browser's fingerprint, whether APIs behave like a real browser, TLS characteristics of the connection, and the reputation of your IP. A plain HTTP client with no JavaScript engine can't run the workload at all, so it fails instantly. A headless browser can run it but often trips fingerprint checks. This is why raw requests or curl gets stuck in the loop no matter how many times you retry.

How to pass it

Two options, same as with Turnstile. Drive a real browser hardened against detection, or call an API that returns the clearance. With Peak the cloudflare5stask hands back the cookie, the matching user-agent, and the supporting headers in one response:

import requests

sol = requests.post(
    "https://api.peak.fo/solve",
    headers={"X-API-Key": "pk_your_api_key"},
    json={
        "task_type": "cloudflare5stask",
        "url": "https://target.com/",
        "proxy": "http://user:pass@ip:port",
    },
    timeout=30,
).json()["data"]

session = requests.Session()
session.headers["User-Agent"] = sol["headers"]["user-agent"]
for name, value in sol["cookies"].items():
    session.cookies.set(name, value)
# keep using the same proxy IP that earned the clearance
page = session.get("https://target.com/data",
                   proxies={"https": "http://user:pass@ip:port"})

The one rule people forget: the clearance is bound to the IP and user-agent that earned it. Reuse it from a different IP or with a different user-agent and Cloudflare throws it out. That's the whole reason to send it on a sticky proxy with the returned user-agent. The mechanics of reusing the cookie correctly are in the cf_clearance cookie, explained.

Why retrying never helps

The instinct when you hit the loop is to retry harder. It doesn't work, because nothing about the retry changes what Cloudflare is scoring. Same client, same fingerprint, same verdict. You either change what your client looks like or you get a valid clearance from something that can pass the check. Hammering the endpoint just gets your IP rate-limited faster.

FAQ

What is the Cloudflare 5-second challenge?

It's a full-page interstitial that runs a JavaScript workload to decide whether you're a real browser. Pass it and Cloudflare sets a cf_clearance cookie so it stops challenging you. Fail and you can't reach the site.

Why do I keep getting the 5-second loop?

Because your client can't pass the check, so Cloudflare never issues clearance. A plain HTTP client has no JavaScript engine; a headless browser often fails fingerprinting. Retrying with the same client changes nothing. Use a hardened browser or an API that returns the clearance.

How long does clearance last?

Cloudflare tunes it per site, so treat it as a session token: use it until a request gets challenged again, then refresh. Because Peak bills only on success, a refresh costs about a tenth of a cent and a miss costs nothing. See pricing.

Stuck in the 5-second loop? Grab a key free at peak.fo.

How to solve Cloudflare Turnstile in 2026 (a developer's guide)

jujucodings — Tue, 08 Sep 2026 12:16:05 +0000

If you scrape or automate anything, you've hit this wall. Requests that ran fine for months start coming back as 403s, or you get an HTML page that just says "Just a moment...". Open the URL in a real browser and there's a little Cloudflare box sitting on the form — Turnstile.

This post covers three things: what Turnstile actually checks, why your headless script gets blocked no matter how many stealth patches you stack on it, and the handful of approaches that actually get you a token when you need one. There's runnable requests and playwright code at the end.

Upfront disclosure: I work on Peak, and one of the options below is our own solving API — I'll list the pricing and free tier plainly. But most of this is method and mechanics that cost nothing, and you can read only that part if you want. This isn't a pitch dressed up as a tutorial.

Turnstile isn't a puzzle, it's a trust score

Old-school CAPTCHAs — reCAPTCHA v2 with the fire hydrants, hCaptcha's "pick the buses" — test whether you can solve a task. Turnstile mostly doesn't give you a task. It tests whether your browser looks like a real person's browser.

Here's what it does in the background:

Runs a chunk of obfuscated JS that collects a browser fingerprint — navigator properties, screen, timezone, WebGL and Canvas rendering, fonts, and whether anything on window smells like automation (navigator.webdriver, CDP-injected objects, and so on).
Runs consistency checks on behavior and environment: mouse movement, the isTrusted flag on events, execution timing, and whether the User-Agent you claim matches your real TLS and HTTP/2 fingerprint.
Weighs the reputation of your exit IP. Datacenter IPs — AWS, GCP, most VPS providers — start with a low score by default.

Score high enough and the widget fills in the cf-turnstile-response token itself, fires its callback, and your form submits like normal. Score too low and you either get an interactive challenge or you simply never get a token.

So the load-bearing idea: Turnstile isn't "solve one puzzle," it's "assemble one trustworthy browser session." That single fact explains why every method below works or doesn't.

Why your headless script gets a 403

Three reasons, biggest first:

Datacenter IP. This is the number-one killer. You're firing requests from a cloud box, the IP reputation is near the floor, and Turnstile cranks the difficulty. The exact same code from a home broadband (residential) IP often just passes.
Headless is loud. A headless=True Chromium has a pile of detectable tells. Even after undetected-chromedriver or playwright-stealth, the leftover CDP traces (Runtime.enable and friends) still light up for Cloudflare.
A plain HTTP client has no JS engine at all. requests, httpx, curl_cffi can't run the challenge script Turnstile needs, so they never produce a token in the first place.

Before you reach for a solver, the cheapest fix that people skip: a lot of the time you don't need to solve anything. If the only reason you're being challenged is a datacenter IP, swap in a residential proxy and a browser with a real fingerprint (Camoufox, or a properly stealth-patched Playwright), and the widget frequently passes on its own — cf-turnstile-response fills itself in. Try that first. It's the lowest-cost path by a mile.

To tell which wall you've hit, look at the response:

The page embeds a <div class="cf-turnstile" data-sitekey="0x..."> form widget → that's the kind this guide handles.
The whole domain is behind a full-screen "Just a moment..." interstitial (the managed challenge / "5-second" page) → that's a different beast. It runs on the cf_clearance cookie, and I'll flag it separately at the end.

Step 1: read the sitekey

The sitekey is public. It's written into the HTML, starts with 0x, and every approach needs it first.

import re
import requests

resp = requests.get("https://target-site.com/login", headers={
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                  "AppleWebKit/537.36 (KHTML, like Gecko) "
                  "Chrome/128.0.0.0 Safari/537.36"
})

# Most common: the data-sitekey attribute
m = re.search(r'data-sitekey=["\'](0x[A-Za-z0-9_-]+)["\']', resp.text)
sitekey = m.group(1) if m else None

# Some sites render it in JS: turnstile.render(el, {sitekey: "0x..."})
if not sitekey:
    m = re.search(r'sitekey["\']?\s*[:=]\s*["\'](0x[A-Za-z0-9_-]+)', resp.text)
    sitekey = m.group(1) if m else None

print("sitekey:", sitekey)

If it's not in the HTML, the widget is being injected by JS. Open DevTools, filter the Network tab for challenges.cloudflare.com, and you'll see the sitekey in the request params — or just run document.querySelector('.cf-turnstile').dataset.sitekey in the console.

Step 2: get a valid token

Minting the token yourself on a server — as covered above — mostly fails on datacenter IPs with automation tells. Two realistic paths:

Path A — residential proxy + real browser, let the widget pass itself. Good when volume is low and you can afford to spin up a browser. Use Camoufox (a Firefox fork with solid fingerprint spoofing) behind a residential proxy, and a lot of checkbox-style Turnstiles clear silently. The downside is it's slow and memory-hungry, and it doesn't scale once volume climbs.

Path B — call a solving API, get back just the token. You send the url and sitekey to a service, it runs the challenge on its own browser farm with residential IPs, and hands you back a token. Your scraper stays pure HTTP the whole time — no browser. At scale this is basically the only sane option.

Services that do this: 2Captcha, CapSolver, YesCaptcha, and Peak, where I work. The interfaces are all similar; here's Peak as the example (swap the URL and field names for another provider).

import requests

API_KEY = "your_API_KEY"

def solve_turnstile(url, sitekey, proxy=None):
    payload = {
        "task_type": "turnstiletask",
        "url": url,          # keep the trailing slash
        "sitekey": sitekey,
    }
    if proxy:
        payload["proxy"] = proxy  # http://user:pass@ip:port
    r = requests.post(
        "https://api.peak.fo/solve",
        headers={"X-API-Key": API_KEY, "Content-Type": "application/json"},
        json=payload,
        timeout=120,
    )
    data = r.json()
    if not data.get("success"):
        raise RuntimeError(f"solve failed: {data}")
    return data["data"]["token"]

token = solve_turnstile("https://target-site.com/login/", sitekey)
print("token:", token[:40], "...")

A real response looks like {"success": true, "data": {"token": "0.abc123..."}, "cost": 0.0009} — on a live hard target one came back in a few seconds. You only get charged on success.

One trap people hit constantly: if the token is going to be used behind a proxy afterward, the proxy you pass at solve time should match the exit IP your later request uses. Cloudflare sometimes binds the token to the IP that solved it. Mismatch the two and the server rejects the token as invalid even though it looks fine.

Step 3: use the token

Once you have a token, there are two cases.

Case one: pure HTTP, POST the token straight in

The Turnstile token ends up as a single form field named cf-turnstile-response. You POST it alongside your other form fields and you're done. A lot of login and submit endpoints are exactly this simple:

session = requests.Session()
session.headers.update({"User-Agent": "Mozilla/5.0 ... Chrome/128.0.0.0 Safari/537.36"})

r = session.post("https://target-site.com/login/", data={
    "username": "me",
    "password": "secret",
    "cf-turnstile-response": token,   # the key field
})
print(r.status_code, r.url)

How do you find the field name? Open DevTools, submit the form once by hand, look at the form data on that POST in the Network tab, and copy it into your Python data=. It isn't always literally cf-turnstile-response — some sites rename it — so go by what you actually capture.

Case two: inject the token in the browser and keep going

If the target is a heavy single-page app and the submit logic lives in JS, put the token back into the page and manually fire Turnstile's callback so the frontend believes it passed:

from playwright.sync_api import sync_playwright

def inject_token(page, token):
    page.evaluate(
        """(token) => {
            // 1) fill the hidden response inputs
            document.querySelectorAll(
                'input[name="cf-turnstile-response"], input[name="g-recaptcha-response"]'
            ).forEach(el => {
                el.value = token;
                el.dispatchEvent(new Event('input',  { bubbles: true }));
                el.dispatchEvent(new Event('change', { bubbles: true }));
            });
            // 2) fire the widget's success callback (many sites rely on this, not the hidden input)
            const el = document.querySelector('.cf-turnstile');
            const cb = el && el.getAttribute('data-callback');
            if (cb && typeof window[cb] === 'function') {
                window[cb](token);
            }
        }""",
        token,
    )

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()
    page.goto("https://target-site.com/login/")

    sitekey = page.get_attribute(".cf-turnstile", "data-sitekey")
    token = solve_turnstile(page.url, sitekey)   # reuse the function above

    inject_token(page, token)
    page.click("button[type=submit]")
    page.wait_for_load_state("networkidle")
    print(page.url)

Two details decide whether this works:

data-callback. Plenty of sites have no visible submit button — the moment verification passes, they move on, driven entirely by this callback. Fill the hidden input without firing the callback and nothing happens. The code above does both.
Dispatching input / change. React and Vue don't notice a raw .value assignment. You have to dispatch the events by hand so the framework sees the change.

The traps worth memorizing

Tokens are single-use and short-lived. Usually valid for a minute or two, once. Don't cache, don't reuse — solve and spend immediately.
Keep the trailing slash on the URL. https://x.com and https://x.com/ can validate differently in some implementations. Pass the exact URL your request actually hits.
action / cData mismatch. If the site passes an action or cData into turnstile.render, you have to pass the matching value at solve time or the token won't validate. Both are findable in the HTML/JS.
The full-screen "Just a moment" page is not the form widget. That's the managed challenge (the "5-second shield"). It clears via the cf_clearance cookie, not a cf-turnstile-response token. Solving APIs usually have a separate endpoint for it (on Peak it's cloudflare5stask, and it returns cookies, not a token). Different call, different params — don't mix them up.

Choosing an approach

Running your own browser farm plus a residential proxy pool is fine at small volume, but the maintenance cost is higher than it looks once you scale — fingerprints have to keep pace with Cloudflare's changes, proxies rotate, failures need retries. So most people move to a ready-made API past a certain point. When you compare them, look at three things: billed per success or per request, residential proxy included or bring-your-own, and the real-world pass rate on Turnstile.

On price, roughly: 2Captcha's Turnstile runs about $1 per 1,000; CapSolver hovers around $0.80. For comparison, Peak's pricing is $0.90 per 1,000 successful Turnstile solves, dropping to $0.35 per 1,000 at volume, billed on success only — failures cost nothing; a new account gets roughly 1,000 free solves to run the flow end-to-end first. The endpoint is the POST https://api.peak.fo/solve from the code above.

If you're on Playwright, Selenium, Scrapy, curl_cffi or Puppeteer and don't want to hand-roll the injection, there are ready MIT-licensed wrappers (on GitHub under CircuitSavage — playwright-turnstile, selenium-turnstile, scrapy-turnstile, cloudscraper-turnstile, turnstile-curl) that bundle the whole read-sitekey → call-API → inject-token loop. There are drop-ins for JavaScript and Go too. Swap in an API key and go.

One thing worth saying plainly: this is for legitimate automation — scraping public data, testing your own sites, QA. Respect the target's terms of service and robots rules, and don't point any of it at logins that aren't yours. Solving a challenge doesn't grant permission you didn't already have.

Wrap-up

The right mental model for Turnstile isn't "solve a CAPTCHA," it's "assemble a trustworthy browser session." So the order is always:

First swap in a residential proxy + real-fingerprint browser and let the widget pass on its own. This step is free and clears a large share of the datacenter-IP blocks.
If it won't pass, or you need to scale, add a solving API for the token only and keep the scraper pure HTTP.
With the token in hand, either POST it straight into the cf-turnstile-response field, or inject it in the browser and fire data-callback.
Watch the traps: single-use tokens, IP binding, the trailing slash, action matching.

Understand what it's checking and you'll get further than by grinding on any single tool.