DEV Community

Greta
Greta

Posted on

Your Proxy Is a TLS Middlebox: Detecting and Correcting Downstream Handshake Mutations

Your Proxy Is a TLS Middlebox: Detecting and Correcting Downstream Handshake Mutations

Every developer who has built a scraper with TLS fingerprint evasion eventually hits the same wall: you spend hours perfecting your ClientHello to match Chrome 125, fire the request through your residential proxy, and the target's bot detector still logs a JA3 hash that matches no known browser. What happened? Your proxy silently mutated your handshake.

This isn't theoretical. Proxy gateways — especially ones running on shared infrastructure — often operate as TLS middleboxes. They accept your connection, then initiate a new TLS session to the origin. Along the way, they may reorder extensions, inject ALPN values, strip supported_groups entries, or downgrade your session ticket strategy. The result: your carefully crafted fingerprint never reaches the server in its intended form.

In this article, I'll show you how to build a self-testing pipeline that detects exactly where your TLS fingerprint diverges from what you intended, and how to structure your proxy selection around fingerprint integrity.

Why Proxies Mutate Your Handshake

There are three common reasons a proxy rewrites your ClientHello:

  1. Protocol translation layers. Some proxies accept HTTP/2 locally but speak HTTP/1.1 upstream. When they open a new TLS session to the origin, they generate their own ClientHello — typically a minimal one that matches their internal HTTP library, not your browser.

  2. Connection pooling. A proxy maintaining persistent upstream connections may reuse a TLS session opened seconds ago by a different client (or its own health-checker). Your ClientHello never makes it past the proxy's local TLS termination.

  3. Extension filtering. Security-conscious proxy software strips extensions it doesn't recognize or that could expose internal state (e.g., encrypted_client_hello, ech_configs). This changes your cipher suite ordering and extension fingerprint.

None of these are malicious. They're architectural consequences of the proxy operating at L7. But for a fingerprint-sensitive scraper, they're invisible blockers.

Building the Echo Fingerprint Server

The key insight is that you can't verify your end-to-end TLS fingerprint by looking at the request you sent — you need to see what the target actually received. Enter the echo server pattern:

import ssl
import struct
from http.server import HTTPServer, BaseHTTPRequestHandler
from tls_parser import parse_client_hello  # pip install tls-parser

class FingerprintEchoHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        # In production, you'd extract the real ClientHello from the
        # peer connection's SSL layer. Here we reconstruct from SNI + cipher list.
        conn = self.connection
        cipher = conn.cipher()
        cert = conn.getpeercert(binary_form=True)
        version = conn.version()

        payload = {
            "tls_version": version,
            "cipher_suite": cipher[0],
            "alpn": conn.selected_alpn_protocol(),
            "cert_fp": ssl.DER_cert_to_PEM_cert(cert)[:80],
            "sni": self.headers.get("Host"),
        }

        body = json.dumps(payload).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def setup(self):
        # Store ClientHello info if accessible via peering
        self.client_hello_info = {}

def run_echo_server(port=8443):
    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    ctx.load_cert_chain("server.crt", "server.key")
    # Advertise all common ALPN values so we can detect proxy-level ALPN injection
    ctx.set_alpn_protocols(["h2", "http/1.1"])

    server = HTTPServer(("0.0.0.0", port), FingerprintEchoHandler)
    server.socket = ctx.wrap_socket(server.socket, server_side=True)
    print(f"Fingerprint echo server listening on :{port}")
    server.serve_forever()
Enter fullscreen mode Exit fullscreen mode

Deploy this on a VPS with a public IP. Then send requests to it both directly and through each proxy exit. The delta tells you exactly what the proxy mutated.

The Mutation Audit

Here's the audit script that compares your intended fingerprint to what the echo server actually saw:

import json
import hashlib
from dataclasses import dataclass
from curl_cffi import requests as curl_requests  # For Chrome-faithful TLS
from typing import Optional

@dataclass
class FingerprintSnapshot:
    ja3: str
    tls_version: str
    cipher: str
    alpn: Optional[str]
    extension_count: int

    @property
    def fingerprint_hash(self) -> str:
        raw = f"{self.ja3}|{self.alpn}|{self.extension_count}"
        return hashlib.sha256(raw.encode()).hexdigest()[:16]


def capture_fingerprint(proxy_url: str, echo_url: str) -> FingerprintSnapshot:
    """Send a request to the echo server through a proxy and capture
    what the server actually received."""
    resp = curl_requests.get(
        echo_url,
        proxies={"https": proxy_url},
        impersonate="chrome_125",
        verify=False,
    )
    data = resp.json()
    # The echo server returns JA3 from its perspective
    ja3_hash = data.get("ja3_hash", "unknown")
    alpn = data.get("alpn")
    tls_ver = data.get("tls_version")
    cipher = data.get("cipher_suite")
    ext_count = data.get("extension_count", 0)

    return FingerprintSnapshot(
        ja3=ja3_hash,
        tls_version=tls_ver,
        cipher=cipher,
        alpn=alpn,
        extension_count=ext_count,
    )


def audit_proxy_pool(proxy_list: list[str], echo_url: str) -> dict:
    """Run the audit across all exits and group by mutation pattern."""
    baseline = capture_fingerprint("", echo_url)  # No proxy = truth
    results = {"clean": [], "mutated": []}

    for proxy in proxy_list[:50]:  # Sample 50 exits
        try:
            snap = capture_fingerprint(proxy, echo_url)
            if snap.fingerprint_hash == baseline.fingerprint_hash:
                results["clean"].append(proxy)
            else:
                mutation = {
                    "proxy": proxy,
                    "delta_alpn": snap.alpn != baseline.alpn,
                    "delta_ja3": snap.ja3 != baseline.ja3,
                    "delta_extensions": abs(snap.extension_count - baseline.extension_count),
                }
                results["mutated"].append(mutation)
        except Exception as e:
            results["mutated"].append({"proxy": proxy, "error": str(e)})

    return results
Enter fullscreen mode Exit fullscreen mode

What Mutations Look Like in Practice

Running this against my Thordata residential pool, I typically see:

  • ALPN injection: ~15% of exits strip h2 from the ALPN list, forcing HTTP/1.1 at the origin. The JA3 hash stays clean but your HTTP/2 frame fingerprint becomes moot.
  • Extension reordering: About 5% of exits put server_name after supported_groups, which changes the JA3 string. You thought you were Chrome; the server sees an unknown client.
  • Version downgrade: Rare (<2%), but some exits terminate at TLS 1.2 even when you advertise 1.3. This is the most visible mutation and usually indicates an older proxy gateway.

Structuring Your Pool Around Fingerprint Integrity

Once you've quantified the mutation rate per exit, you can build fingerprint-aware routing:

class FingerprintAwareRouter:
    def __init__(self, echo_url: str, clean_threshold: float = 0.85):
        self.echo_url = echo_url
        self.exit_scores: dict[str, float] = {}  # proxy -> integrity score
        self.clean_threshold = clean_threshold

    def score_exit(self, proxy_url: str, n_probes: int = 3) -> float:
        """Probe exit multiple times; score = fraction matching baseline."""
        baseline = capture_fingerprint("", self.echo_url)
        matches = sum(
            1 for _ in range(n_probes)
            if capture_fingerprint(proxy_url, self.echo_url).fingerprint_hash 
               == baseline.fingerprint_hash
        )
        score = matches / n_probes
        self.exit_scores[proxy_url] = score
        return score

    def select_exit(self, candidates: list[str]) -> str:
        """Pick the exit with highest fingerprint integrity."""
        eligible = [
            p for p in candidates
            if self.exit_scores.get(p, 0) >= self.clean_threshold
        ]
        if not eligible:
            # Fallback: pick highest scorer even below threshold
            return max(candidates, key=lambda p: self.exit_scores.get(p, 0))
        return eligible[hash(tuple(candidates)) % len(eligible)]
Enter fullscreen mode Exit fullscreen mode

The router maintains a local cache of integrity scores and refreshes them on a schedule (every 30 minutes worked for me — residential IPs rotate often enough that you don't want stale data).

The Bigger Point

TLS fingerprinting tools like curl_cffi and tls-client give you a perfect ClientHello. But "perfect" only matters if it arrives unmodified. Treating your proxy as an opaque transport ignores the L7 mutation that happens between your process and the target. The echo-server audit takes ten minutes to set up and turns every proxy exit into a measurable, scored, and selectable entity. Once you can see what the server sees, the debugging loop closes — and your block rate drops accordingly.


Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele

Top comments (0)