DEV Community

Cover image for Why Your Playwright Scraper Gets Blocked: TLS Fingerprinting (JA3/JA4) Explained
PromptCloud
PromptCloud

Posted on

Why Your Playwright Scraper Gets Blocked: TLS Fingerprinting (JA3/JA4) Explained

You launched a real Chromium, set a real User-Agent, and still got blocked on the first request, before your script even touched the page. The likely culprit sits below HTTP, in the TLS handshake. Here is what JA3 and JA4 actually see, and why the fix is usually not the one people reach for.

You did everything the tutorials said. Real browser via Playwright, a genuine Chrome User-Agent, sensible delays. And the site still served you a challenge page or a 403 on the very first request, before a single line of your scraping logic ran. That timing is the tell. A block that lands that early is not about your behaviour on the page, because you have not done anything yet. It is about how your client looked the instant it opened the connection, and a large part of that first impression is your TLS fingerprint.

What TLS fingerprinting actually is

Before any HTTP request is sent over HTTPS, the client and server complete a TLS handshake. The very first message the client sends is the ClientHello, and it is surprisingly revealing. It advertises the TLS version, the ordered list of cipher suites the client supports, the extensions it offers, the elliptic curves, the signature algorithms, and the ALPN protocols. None of this is your data. It is the shape of your networking stack, decided by whichever library or browser opened the socket.

TLS fingerprinting takes those fields and reduces them to a compact identifier. JA3, the original method, concatenates the key ClientHello fields into a string and hashes it with MD5 to produce a 32-character fingerprint. A real Chrome build produces one recognisable JA3. Python's requests produces a completely different one. Go's net/http produces another. The server can compute your fingerprint before it answers, compare it to a list of known-good browser fingerprints, and decide whether you look like a browser or like a script wearing a browser's User-Agent.

That mismatch is the classic failure. A request whose header says "I am Chrome" but whose TLS handshake says "I am a Python HTTP library" is trivially flagged, because the two halves of the same request disagree about what the client is.

# A JA3 string is built from ordered ClientHello fields:
#   TLSVersion,Ciphers,Extensions,EllipticCurves,ECPointFormats
# e.g.
771,4865-4866-4867-49195-49199-...,0-23-65281-10-11-...,29-23-24,0
# MD5(that) -> e7d705a3286e19ea42f587b344ee6865   <- your fingerprint

# The problem is not that the hash exists.
# It is that a browser User-Agent + a non-browser JA3 = a contradiction.
Enter fullscreen mode Exit fullscreen mode

So why does a real Playwright browser get caught

Here is the part most articles skip. If you are driving a genuine, unmodified Chromium through Playwright, your TLS handshake is Chromium's handshake. The JA3 or JA4 you present is a real browser fingerprint, because a real browser is making the connection. Pure TLS fingerprinting, on its own, does not usually catch vanilla Playwright plus bundled Chromium. So when a real browser still gets blocked early, one of a few specific things is usually going on.

The most common is that somewhere in the stack, the request stopped being made by the browser. Teams reach for Playwright's HTTP request API, or drop down to an HTTP client for the "simple" endpoints to save resources, and those requests carry the library's fingerprint, not the browser's. You now have a session that is part real browser and part script, and the scripted parts are the ones that get flagged.

The second is a TLS-terminating proxy in the path. If your traffic is routed through a proxy that terminates and renegotiates TLS, the ClientHello the destination sees is the proxy's, not Chromium's. Your carefully-real browser fingerprint is replaced by the proxy's fingerprint before it ever reaches the site. This one is easy to miss because everything on your side still looks like Chrome.

The third is that TLS is only the first layer. Modern detection does not stop at JA3. There is an HTTP/2 fingerprint derived from how the client sets up frames, header ordering, and settings; there is the ordering and casing of your HTTP headers; there is IP reputation; and there is the actual browser environment tested with JavaScript once the page loads. A real Chromium can pass the TLS check and still fail one of these, and the block can arrive early enough that TLS gets the blame.

JA3 is fading, and JA4 is why

There is a live reason not to over-index on JA3 specifically. Chrome began randomising the order of its TLS extensions in the ClientHello (the change shipped in Chrome 110, in early 2023). Because JA3 hashes the extensions in the order they appear, that order changing on every connection means a real Chrome now produces a different JA3 each time. JA3 as a stable identifier for Chrome effectively broke.

The industry response is JA4, part of the JA4+ suite developed by John Althouse at FoxIO, the same person behind the original JA3. JA4 is built to survive the randomisation: it sorts the extension list before hashing, so a permuted ClientHello still maps to the same fingerprint, and it is more structured and human-readable than a single MD5 blob. JA4+ extends the idea beyond TLS into a family of fingerprints, including one for HTTP and one for the server side. The practical upshot for a scraper: detection is moving to fingerprints that are harder to accidentally spoof and harder to destabilise, so the days of matching a single JA3 string and calling it solved are ending.

What actually fixes it

The instinct is to hunt for a magic fingerprint to paste in. That is the wrong level to solve this at, because you are fighting a whole family of signals that all have to agree, and they have to keep agreeing as browsers and detection both change.

The durable principle is consistency. Every layer of a request should tell the same story: if the User-Agent says Chrome on Windows, then the TLS handshake, the HTTP/2 fingerprint, the header order, and the JavaScript environment should all be that same Chrome on Windows. The moment two layers disagree, you are detectable, no matter how real any single layer is.

In practice that means making all your requests through the real browser stack rather than mixing in HTTP clients for convenience; being careful that proxies pass TLS through rather than terminating it; and keeping the browser build current so its fingerprint matches the real population of browsers, not a version nobody runs any more. Understanding how a headless browser for web scraping actually presents itself at every layer is what separates a scraper that quietly keeps working from one that gets blocked and leaves you guessing which of a dozen signals gave it away.

This is also, frankly, why a lot of teams stop maintaining this themselves at scale. Keeping every layer consistent across many sources, as both browsers and anti-bot systems keep moving, is ongoing work, not a one-time configuration. It is solvable, but it is a job, not a snippet.

The takeaway

If your Playwright scraper gets blocked before it does anything, look below HTTP. TLS fingerprinting reads the shape of your ClientHello and flags clients whose handshake contradicts their User-Agent. A genuine Chromium usually passes that specific check, so an early block on a real browser normally means either a non-browser request slipped into your session, a proxy rewrote your handshake, or a different layer such as the HTTP/2 fingerprint caught you. JA3 is fading as Chrome randomises its extensions, and JA4 is replacing it. The fix is not a magic string. It is making every layer of the request consistent, and keeping it that way.

FAQ

What is a TLS or JA3/JA4 fingerprint?

It is an identifier computed from the fields in the TLS ClientHello a client sends at the start of an HTTPS handshake: the TLS version, cipher suites, extensions, elliptic curves, and more. JA3 hashes those fields with MD5; JA4 is a newer, more robust successor. A server can read your fingerprint before it answers and tell whether you look like a real browser or like a script, independently of your User-Agent.

Why does my scraper get blocked even with a real browser and a real User-Agent?

Because detection compares layers. A genuine Chromium usually presents a legitimate TLS fingerprint, so an early block normally means something else: a request made by an HTTP client rather than the browser, a proxy that terminated and rewrote your TLS handshake, or a non-TLS signal such as the HTTP/2 fingerprint, header ordering, or IP reputation. The block lands early, so TLS gets blamed, but the real cause is often a mismatch elsewhere.

Is JA3 still used, or has JA4 replaced it?

Both are in use, but JA3 has weakened. Since Chrome began randomising its TLS extension order in early 2023, a real Chrome produces a different JA3 on each connection, which makes JA3 unreliable as a stable browser identifier. JA4, part of the JA4+ suite from the creator of JA3, sorts the extensions before hashing so it survives that randomisation, and detection is steadily moving towards it and its relatives.

Top comments (0)