DEV Community

Greta
Greta

Posted on

Prior-Knowledge or ALPN Upgrade? The HTTP/2 Connection Preface Tells You What Your Proxy Got Wrong

Prior-Knowledge or ALPN Upgrade? The HTTP/2 Connection Preface Tells You What Your Proxy Got Wrong

Everyone who has spent a weekend debugging anti-bot detection knows about JA3, header ordering, and the User-Agent/TLS mismatch. Far fewer people talk about the layer right above TLS: the HTTP/2 connection preface and its first control frames. That's a shame, because the preface is where the disagreement between what you asked for and what your proxy actually negotiated becomes visible to the origin — and it's the single most common way a well-fingerprinted scraper quietly loses its h2 identity on the last hop.

The core idea of this piece: your HTTP/2 handshake is a fingerprint too, and the connection preface plus the opening SETTINGS/WINDOW_UPDATE frames reveal whether your proxy terminated and re-originated the connection, and how it made that decision. Once you can see that signal, you can audit exits for HTTP/2 integrity exactly the way you'd audit them for TLS coherence — before the origin does.

The two ways a client can speak HTTP/2

There are two legitimate paths to an h2 connection, and they differ in the first bytes on the wire:

  1. ALPN negotiation. During the TLS handshake the client offers h2 in its Application-Layer Protocol Negotiation extension, the server picks h2, and the client immediately sends the connection preface — the magic string PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n — over the already-established TLS session. This is what every modern browser does.
  2. Prior knowledge. The client skips TLS (or is already on a connection it knows supports h2) and starts with the preface anyway, typically against http://host:port. Rare, and mostly used by gRPC servers and HTTP/2-aware internal load balancers.

There is also the deprecated third path — the Upgrade: h2c header on an http/1.1 connection followed by a 101 Switching Protocols. Modern browsers do not do this. Any traffic that looks like it came from Chrome 124 but uses the Upgrade dance is definitionally not Chrome 124. This is one of the most under-diagnosed silent tells in the anti-bot world.

Why proxies get this wrong

Residential and ISP gateways often terminate TLS. Your client negotiates h2 with the gateway, but the gateway has to originate a fresh connection to the origin. What happens next is where most "we have a real browser fingerprint, why are we still blocked" mysteries live:

  • The gateway doesn't support h2 on its upstream leg and silently falls back to http/1.1. The origin sees a Chrome-flavored TLS handshake with ALPN: h2 but then a series of HTTP/1.1 requests. Incoherent. Blocked.
  • The gateway supports h2 but re-frames your SETTINGS with defaults (typically MAX_CONCURRENT_STREAMS=100, INITIAL_WINDOW_SIZE=65535, and no ENABLE_CONNECT_PROTOCOL), so the SETTINGS fingerprint looks like nghttp2 or curl, not Chrome.
  • The gateway sends the connection preface after a partial http/1.1 request (a bug in some MITM proxies that treat the preface as a request line), which is a definitive non-browser signature.
  • The gateway reorders or omits your HPACK dynamic table updates (already a topic of its own fingerprint — see RFC 7541 §2.3.2). Different from your client's initial table, but the way it evolves in the first few responses is what origins key on.

Every one of these is invisible to your code. Your httpx or curl_cffi call succeeded; the status was 200. Only the origin, which sees the whole wire, notices. The way to make it visible is to stand up an HTTP/2 echo server and route one probe per exit through it.

Building the probe

The minimum useful echo response captures, per exit:

  • The negotiated ALPN from the TLS handshake.
  • The first frame the client sent (was it the preface string? a raw SETTINGS? a broken PRI line?).
  • The initial SETTINGS payload (all six parameters, ordered as received).
  • The initial WINDOW_UPDATE on stream 0, if any, and its increment.
  • The header-block representation on the first request (indexed vs. literal, pseudo-header ordering, use of dynamic table).

Here's a Python sketch using h2 (the python-hyper project) — the same library many production scrapers already depend on:

import h2.config, h2.connection, h2.settings, h2.events
from dataclasses import dataclass, field

@dataclass
class PrefaceFingerprint:
    negotiated_alpn: str = ""
    preface_bytes: int = 0            # bytes observed before first frame (1+24 expected)
    initial_settings: dict = field(default_factory=dict)
    initial_window_update: int = None
    pseudo_header_order: tuple = ()
    first_request_method: str = ""

class Http2Echo:
    """Very small h2 server-side handler: reads a request, records the fingerprint,
    and returns 204 so the caller can safely tear down."""
    def __init__(self, conn_bytes: bytes, alpn: str):
        self.fp = PrefaceFingerprint(negotiated_alpn=alpn)
        self.config = h2.config.H2Configuration(client_side=False)
        self.conn = h2.connection.H2Connection(config=self.config)
        self.conn.initiate_connection()
        self.data = conn_bytes
        self._parse()

    def _parse(self):
        # Frame the raw bytes; h2 will feed events as we ingest.
        events = self.conn.receive_data(self.data)
        for ev in events:
            if isinstance(ev, h2.events.SettingsAcknowledged):
                self.fp.initial_settings = dict(self.conn.local_settings)
            elif isinstance(ev, h2.events.WindowUpdated):
                if ev.stream_id == 0 and self.fp.initial_window_update is None:
                    self.fp.initial_window_update = ev.delta
            elif isinstance(ev, h2.events.RequestReceived):
                self.fp.first_request_method = dict(ev.headers).get(b":method", b"")[:].decode()
                self.fp.pseudo_header_order = tuple(
                    k.decode() for k, _ in ev.headers if k.startswith(b":")
                )
Enter fullscreen mode Exit fullscreen mode

That's enough to reconstruct the shape of the client's opening move. What you're not looking for is byte-perfect equivalence with a reference Chrome capture — proxies will almost never match perfectly. What you're looking for are categorical mismatches: negotiated_alpn == "h2" but the first request arrived as HTTP/1.1 (which the handler above never even fires for), or initial_settings matching nghttp2 defaults when you intended Chrome.

The four fingerprints that actually differ

Across a pool of a few hundred exits, HTTP/2 prefaces cluster into four buckets, and each bucket tells you something different about the underlying gateway:

Bucket A — Real browser h2. MAX_CONCURRENT_STREAMS=1000, INITIAL_WINDOW_SIZE=65536, HEADER_TABLE_SIZE=65536, followed within the same packet by a WINDOW_UPDATE of ~15,663,649 bytes (the Chrome-specific DEFAULT_WINDOW_UPDATE_RATIO trick). Pseudo-header order is :method, :scheme, :authority, :path and first HEADERS uses dynamic-table indexing for common names. If your exit is Bucket A, HTTP/2 coherence is intact.

Bucket B — Library h2 (curl, nghttp2, hyper). Default MAX_CONCURRENT_STREAMS=100, no WINDOW_UPDATE on connection 0 in the opening burst, and pseudo-headers sometimes emitted as :method, :path, :scheme, :authority (a common library ordering that Chrome never uses). If your exit is Bucket B while your TLS voice screams Chrome, the gateway re-originated TLS with a library client, and the origin will notice.

Bucket C — Gateway re-mux. The gateway terminated your connection and multiplexed many clients onto a single upstream h2 connection. Symptoms: your SETTINGS frame is absent (the gateway sent its own at upstream-open time, before you arrived), your stream IDs continue past what a single session would plausibly emit, and the very first frame you observe looks like a mid-session HEADERS, not a preface.

Bucket D — Silent downgrade. ALPN said h2, but the connection never sent the preface; instead it opened with GET / HTTP/1.1 on the same TLS session. This is a proxy bug that is nearly invisible from the client side, because your HTTP library just sees a normal response. This bucket is where identity-bearing traffic should never be routed.

A one-line classifier over the fields above will tell you which bucket each of your exits is in, per probe. Chart the distribution across the pool and you have an HTTP/2 health score for your provider.

Routing on the HTTP/2 signature

Once every exit carries a bucket label, the routing decisions write themselves:

  1. Authenticated and identity-bearing traffic → Bucket A only. Chrome-shaped preface required. If your residential gateway is emitting Bucket B for 40% of its pool, that's a real constraint on how much of the pool you can actually use for login flows.
  2. Anonymous fetches → Bucket A, B, or C. Anything but D. A library-shaped h2 fingerprint on an anonymous catalog fetch is not a big deal.
  3. Bucket D → quarantine and re-probe. A downgrade-in-flight is a bug or a misconfiguration on the provider side, and it may self-correct on the next session, so cool-down-and-check is friendlier than drop.

The broader lesson is that HTTP/2 is a fingerprinting surface that most teams have not instrumented. It sits above the layer everyone obsesses over (TLS) and below the layer everyone logs (application HTTP). That gap is exactly where proxy gateways tend to be sloppy, and where a scraper's real browser identity quietly evaporates on the last hop. Standing up one echo server and running one probe per exit per hour converts that invisible failure mode into a number you can alert on and route around. Start there, and the next time you see a Chrome-shaped TLS fingerprint blocked with zero HTTP-layer errors to explain it, you'll know where to look first.


Disclosure: I use Thordata's residential proxies for this project. New users get 500MB free — code thor020 (10% off): https://www.thordata.com/?ls=uXcSHJzx&lk=02-tele

Top comments (0)