The /sdp endpoint that trusted its callers: fixing a memory-amplification DoS in py-libp2p

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.

Project Overview

py-libp2p is the Python implementation of libp2p — the peer-to-peer networking stack that underpins IPFS, Filecoin, and Ethereum-class nodes. I've been working on its WebRTC-Direct transport, which lets two peers connect without a certificate authority: the peer's multiaddr carries a hash of its TLS cert, and the DTLS handshake is verified against it.

Before that encrypted transport exists, the two sides have to swap SDP offer/answer blobs. Until the STUN-based listener lands (#1352), py-libp2p ships a minimal dev harness for this: a tiny hand-rolled HTTP server (no aiohttp dependency) that accepts an SDP offer over POST /sdp and hands the body to an offer handler. This post is about a memory-amplification DoS I found and fixed in that harness.

Bug Fix or Performance Improvement

The POST /sdp handler read the caller's Content-Length and buffered exactly that many bytes — with no upper bound — before the handshake ever happened. Here's the pre-fix read path in _aiortc_helpers.py:

# Read HTTP request line (consumed but not used) + headers
await asyncio.wait_for(reader.readline(), timeout=_SDP_HTTP_TIMEOUT)
headers: dict[str, str] = {}
while True:
    line = await asyncio.wait_for(reader.readline(), timeout=_SDP_HTTP_TIMEOUT)
    if line in (b"\r\n", b"\n", b""):
        break
    key, _, value = line.decode().partition(":")
    headers[key.strip().lower()] = value.strip()

content_length = int(headers.get("content-length", "0"))
body = b""
if content_length > 0:
    body = await asyncio.wait_for(
        reader.readexactly(content_length),
        timeout=_SDP_HTTP_TIMEOUT,
    )

Two things are attacker-controlled and unbounded:

1. The body. content_length comes straight from the request. reader.readexactly(content_length) accumulates that many bytes into memory — and readexactly bypasses the StreamReader's default 64 KiB flow-control limit, so nothing throttles it. body.decode() makes a second copy, the handler string a third. Net amplification ≈ 2× the bytes on the wire, with no ceiling.
2. The headers. while True with only a per-line timeout: an attacker can keep sending header lines forever — the loop never counts them.

This is pre-handshake, unauthenticated input. Anyone who can reach the port can trigger it.

Where I found it: a reviewer flagged the shape of this inside my own WebRTC PR (#1309) — a one-line "reads the whole body into memory with no cap, availability risk" note. Easy to read as a style nit; it wasn't. Pre-handshake input is attacker-controlled by definition, and "reads the whole body with no cap" is the entire exploit.

The numbers. I built a reproduction harness that imports the real run_signaling_server at the pre-fix parent (9506041) and the merged fix (759c75b), fires the malicious request, and samples RSS on a 20 ms timer:

Metric	Before (pre-fix)	After (#1396)
Peak RSS, 512 MiB malicious body	1,113 MB (+1,052)	61 MB (+0)
Peak RSS, 1 GiB malicious body	2,137 MB (+2,076)	rejected, idle
Peak RSS, 4 concurrent × 300 MiB	1,575 MB (+1,514)	88 MB (+27)
Time to refuse oversized request	never (buffers to OOM)	0.3 ms (413 on headers alone)
Body size accepted	unbounded	32 KiB (413 above)

(Measured in a Python 3.11 / aiortc 1.15 sandbox on a single asyncio loop — re-run the harness on your own hardware for local figures; the ~2× ratio holds.)

Code

PR: libp2p/py-libp2p#1396 — fix(webrtc): harden /sdp HTTP server against memory-amplification DoS · merged Aug 14, 2026 · closes #1354 · commit 759c75b.

The fix adds three bounded constants and rejects before touching any body buffer:

# Bounds for the HTTP /sdp dev harness — defend against memory-amplification
# DoS while the harness exists (until the STUN-based listener lands, #1352).
_MAX_SDP_BODY_SIZE = 32 * 1024  # 32 KiB; SDP offers are typically 1–4 KiB
_MAX_HEADER_LINES = 64
_MAX_HEADER_BYTES = 8 * 1024    # 8 KiB total across all header lines

Headers are bounded by line count and cumulative bytes (the for/else fires a 400 if the terminator never arrives):

headers: dict[str, str] = {}
header_bytes = 0
for _ in range(_MAX_HEADER_LINES + 1):
    line = await asyncio.wait_for(reader.readline(), timeout=_SDP_HTTP_TIMEOUT)
    if line in (b"\r\n", b"\n", b""):
        break
    header_bytes += len(line)
    if header_bytes > _MAX_HEADER_BYTES:
        writer.write(b"HTTP/1.1 400 Bad Request\r\nContent-Length: 0\r\n\r\n")
        await writer.drain(); return
    key, _, value = line.decode().partition(":")
    headers[key.strip().lower()] = value.strip()
else:
    writer.write(b"HTTP/1.1 400 Bad Request\r\nContent-Length: 0\r\n\r\n")
    await writer.drain(); return

And Content-Length is validated before any body buffer exists — malformed/negative → 400, oversized → 413:

raw_cl = headers.get("content-length", "0")
try:
    content_length = int(raw_cl)
except ValueError:
    writer.write(b"HTTP/1.1 400 Bad Request\r\nContent-Length: 0\r\n\r\n")
    await writer.drain(); return
if content_length < 0:
    writer.write(b"HTTP/1.1 400 Bad Request\r\nContent-Length: 0\r\n\r\n")
    await writer.drain(); return
if content_length > _MAX_SDP_BODY_SIZE:
    writer.write(b"HTTP/1.1 413 Payload Too Large\r\nContent-Length: 0\r\n\r\n")
    await writer.drain(); return

The PR also ships a 228-line regression test (test_aiortc_helpers.py) whose key assertion checks that RSS didn't grow after a 413 — not just the status code:

async def _too_large(self) -> None:
    rss_before = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    server, port = await _start()
    line = await _status(port, b"POST /sdp HTTP/1.1\r\nContent-Length: 999999999\r\n\r\n")
    assert b"413" in line
    rss_after = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    assert (rss_after - rss_before) < one_mib   # body buffer was never allocated

My Improvements

I deliberately chose a hard cap over a streaming-with-ceiling read. SDP offers are 1–4 KiB in practice; no legitimate caller needs to POST megabytes here, so a fixed 32 KiB ceiling with early rejection is simpler and safer than draining a bounded stream. Rejecting on the Content-Length header alone means the oversized request never allocates a buffer at all — which is why the "after" column of the table plateaus at the idle baseline and the refusal lands in 0.3 ms.

Impact in context: py-libp2p underpins Python IPFS/Filecoin-class nodes. A signaling endpoint that OOM-kills the whole process on unauthenticated, pre-handshake input is a cheap remote crash — no credentials, no handshake, one connection. Bounding it to 32 KiB turns "attacker picks my memory ceiling" into "attacker gets a 413."

What I learned: the reviewer's one-line "availability risk" aside was a working exploit, and I almost filed it under style. Pre-handshake input is attacker-controlled by definition — "trust the Content-Length" is never a safe default. And the test matters as much as the fix: asserting 413 proves the status code; asserting RSS-didn't-grow proves the property — that no buffer was allocated.

Reproduce it:

git clone https://github.com/libp2p/py-libp2p && cd py-libp2p
pip install aiortc
git checkout 9506041   # PRE-FIX: RSS climbs to ~1.1 GB, 200 OK
python3 exploit.py PREFIX body 512
git checkout 759c75b   # POST-FIX (#1396): 413 in 0.3 ms, RSS flat
python3 exploit.py POSTFIX bodyreject 512

Best Use of Sentry

There's no first-class trio integration for Sentry (its Python integrations are asyncio / Django / Flask / FastAPI / AIOHTTP / etc.), so I attached manually — and it fit cleanly because the pieces genuinely apply:

• The /sdp server is plain asyncio, so wrapping the handler in sentry_sdk.start_transaction(op="sdp.post", name="/sdp handler") gave me a real trace waterfall — the span sitting on reader.readexactly() is visibly the culprit.
• AsyncioIntegration actually applies — the WebRTC bridge from #1309 runs a real asyncio event loop in a daemon thread, so the integration isn't decorative.
• SocketIntegration adds DNS/connect spans for free, on-topic for a P2P lib.
• I emitted RSS as a custom measurement so the before/after shows up inside Sentry:

import resource, sentry_sdk
from sentry_sdk.integrations.asyncio import AsyncioIntegration
from sentry_sdk.integrations.socket import SocketIntegration

sentry_sdk.init(dsn="<YOUR_DSN>", traces_sample_rate=1.0, profiles_sample_rate=1.0,
                integrations=[AsyncioIntegration(), SocketIntegration()])

def rss_mb():
    return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024  # Linux: KB→MB

with sentry_sdk.start_transaction(op="sdp.post", name="/sdp handler") as tx:
    tx.set_measurement("rss_before_mb", rss_mb(), "megabyte")
    # ... handle request ...
    tx.set_measurement("rss_after_mb", rss_mb(), "megabyte")

Here's the captured trace — the sdp.read_body span running reader.readexactly(157286400) is the whole story, and the rss_before_mb / rss_after_mb measurements ride along on the transaction:

Seer's root-cause analysis — this was the standout. I ran Seer on the captured event and, with no knowledge of the fix I'd already shipped, it pinned the exact cause and proposed essentially the same patch:

Quoting it: Seer flagged a "+196 MB in one request" spike, found that handle() "reads Content-Length directly from the attacker-controlled HTTP header and passes it verbatim to reader.readexactly(content_length) … with no upper-bound check," and separately caught the header loop as an "unbounded while-loop." Its recommended fix — a MAX_SDP_BODY_SIZE cap that returns 413 Payload Too Large before the read, plus per-line/total header caps — is the same shape as #1396. The only real difference: Seer suggested a 64 KiB cap; I shipped 32 KiB. Honest verdict: right, and specific enough to act on — the uncommon case where the AI root-cause analysis lands the actual fix rather than a vague "add validation."

On Session Replay: not applicable — there's no browser in this path, so I didn't fake a screenshot. The honest "N/A" is more useful to a maintainer than a contrived one.

Best Use of Google AI

I pasted the pre-fix read path into Google AI Studio (Gemini) and asked for three concrete hypotheses for the unbounded memory growth, ranked by likelihood, with a one-line fix for each.

Its #1, ranked "Most Likely," was exactly right: "the server trusts the client-provided content-length header and attempts to buffer the full amount into RAM … an attacker can send a large value (e.g. 10^9) and slowly stream data, forcing the server to grow its internal buffer to match." It even independently named the ~2× amplification I'd measured — "Python's object-copying behaviour is acting as a memory multiplier." The guard it proposed (if content_length > 1_048_576: raise …) is the same shape as the fix I shipped in #1396 — the only difference is the cap size (Gemini picked 1 MiB; I used 32 KiB, since real SDP offers are 1–4 KiB). Honest read: for a tightly-scoped prompt, Gemini's top-ranked hypothesis landed the real bug on the first try.