48-Hour Field Notes: The Client Looked Fast. Each Call Opened a Fresh Socket.

Have you ever congratulated yourself on a snappy HTTP client that never actually left your laptop? I did that recently, then watched the same request loop stall once a real network sat between the processes. The localhost numbers looked kind because a loopback handshake costs almost nothing compared with a cross-host TLS dance. I spent the next forty-eight hours proving that my so-called benchmark had been timing the wrong machine.

Was the API actually slow, or had I been measuring a loopback that never paid for TLS? That question sat on a sticky note while I chased pools, timeouts, and a very confident local dashboard. This write-up is a lab notebook you can rerun, not a production war story dressed up with invented percentiles. Every command below runs on a laptop first, then on any remote host you already control.

Hour 0–8: I trusted a local loop and a pretty mean

I started with the smallest possible client because I wanted a number before I wanted a theory. Fifty requests.get calls against a local http.server felt honest, fast, and easy to paste into a gist. The mean landed in the low milliseconds, so I told myself the client was fine and later staging lag would be "the server."

That story did not survive contact with a second machine. Loopback TCP is not a dress rehearsal for a handshake that crosses a real network interface. If you only time the happy path on 127.0.0.1, you are mostly timing Python call overhead and a kernel shortcut.

Here is the first client I actually ran. It looks harmless. It is also the bug.

# naive_client.py
import os
import time
import requests

URL = os.environ.get("TARGET_URL", "http://127.0.0.1:8080/ping")
N = int(os.environ.get("N", "50"))

start = time.perf_counter()
for _ in range(N):
    response = requests.get(URL, timeout=5)
    response.raise_for_status()
elapsed = time.perf_counter() - start
print(f"n={N} total={elapsed:.3f}s mean={elapsed / N * 1000:.2f}ms")

Why does this keep paying setup costs on every iteration instead of warming a pool? The requests convenience helpers still build a fresh Session, send one request, and close that session in a context manager. urllib3 never gets a chance to keep the socket warm across the loop. On localhost you barely notice the tax. Off localhost you notice it immediately.

Hour 8–24: the pool looked busy while the peer port kept changing

I opened the server logs expecting one client port and a boring keep-alive stream. Instead, the remote address stayed the same while the source port jumped on almost every request. That is the fingerprint of a new TCP connection, not a healthy pool.

Do you log the peer port today, or only the status code and the request path? Status codes cannot tell you whether the handshake happened again. A changing source port can, and it costs one print statement.

The server I used is standard library only, so the experiment does not depend on Flask, FastAPI, or a container image.

# conn_watch.py
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

BODY = b"ok"
seen_ports = set()

class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def do_GET(self):
        host, port = self.client_address[0], self.client_address[1]
        seen_ports.add(port)
        print(
            f"peer={host}:{port} path={self.path} "
            f"unique_ports={len(seen_ports)}"
        )
        self.send_response(200)
        self.send_header("Content-Type", "text/plain")
        self.send_header("Content-Length", str(len(BODY)))
        self.send_header("Connection", "keep-alive")
        self.end_headers()
        self.wfile.write(BODY)

    def log_message(self, fmt, *args):
        return  # keep stdout for the peer lines only

if __name__ == "__main__":
    ThreadingHTTPServer(("0.0.0.0", 8080), Handler).serve_forever()

Run it in one terminal, then fire the naive client in another:

python conn_watch.py
TARGET_URL=http://127.0.0.1:8080/ping python naive_client.py

If you see a new peer= port on every line, the client is not pooling. If the port sticks for many lines, keep-alive is doing its job. I stared at jumping ports longer than I will admit, because I had assumed requests would pool by default at the module level.

It does not. The module-level helpers are convenience wrappers, not a process-wide connection cache. That distinction is still true for the requests 2.x API I ran this against, and it is the whole lesson.

The artifact: two clients, one server, a decision table

The fix is not a new framework. Hold one requests.Session for the lifetime of the worker, and let urllib3 reuse the underlying HTTP connection.

# pooled_client.py
import os
import time
import requests

URL = os.environ.get("TARGET_URL", "http://127.0.0.1:8080/ping")
N = int(os.environ.get("N", "50"))

with requests.Session() as session:
    start = time.perf_counter()
    for _ in range(N):
        response = session.get(URL, timeout=5)
        response.raise_for_status()
    elapsed = time.perf_counter() - start

print(f"n={N} total={elapsed:.3f}s mean={elapsed / N * 1000:.2f}ms")

Run both clients against the same conn_watch.py process. Compare three things, not one pretty mean:

1. Mean time per request on localhost, which will almost always flatter the naive client.
2. Mean time per request against a host that is not your loopback interface.
3. Whether the server's peer= source port stays still or chatters through the run.

Localhost will understate the gap between the two clients. That understatement is the entire point of the next twelve hours, and it is why a second machine belongs in the checklist.

A tiny test plan you can tick off

I treat this as a checklist now, because I will forget the lesson the next time a dashboard looks green.

1. Start conn_watch.py on the target host and leave stdout visible in a dedicated terminal.
2. Point naive_client.py at that host and record mean time plus the final unique_ports count.
3. Point pooled_client.py at the same host without restarting the server process in between.
4. Repeat once with plain HTTP on a LAN, then once with HTTPS if you already have a certificate.
5. Fail the check if unique source ports on the naive client roughly equal N.

You do not need a load-testing product for this audit. You need a second machine, a print statement, and the discipline to read the port column.

Decision table

What you see	What it usually means	What I do next
Source port changes every request	New TCP connection on each call	Search for requests.get or httpx.get inside loops
Source port sticks, time still high	Pool works; cost is CPU or the handler	Profile the server path, not the handshake
Localhost is fast, remote is slow, ports jump	Handshake cost was hidden by loopback	Move the timing run off the laptop
Remote is slow, ports stick	You are timing application work or waits	Add handler-side timestamps around real work
HTTPS gap much larger than HTTP gap	TLS setup dominates the first request	Reuse the session before you tune ciphers

I keep that table next to the client because it stops me from rewriting the server when the client is the one lighting new sockets.

Hour 24–48: localhost lied, so I left the laptop

The remaining failure was methodological rather than a missing import. I kept rerunning the pooled client on the same machine that hosted the server, then wondering why staging still felt heavier. Loopback never made me pay for a real round trip, packet delay, or certificate check. A remote hop did.

I needed a throwaway host that was not my loopback device, plus a second pair of eyes on the client. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access to review the client for per-call Session construction, and I used the free server option as the other end of conn_watch.py so the handshake cost could actually show up.

The model pass is not magic, and I do not treat it as a benchmark or a quota story. I paste the client, ask for every site where a session is created or closed inside a loop, and then I verify those sites with the peer-port log. The free server is useful only because it is not 127.0.0.1. Any VM, container host, or spare laptop on another network would have taught the same lesson.

If you already have a remote box, use that and keep the same TARGET_URL export. The important move is leaving loopback, not collecting another dashboard.

Review prompts I actually typed

These are labeled prompts, not proof that a model is always right, and I reran the server log after each one.

• Find every place this file builds a requests.Session, httpx.Client, or urllib3 PoolManager. Does any of them live inside a per-request helper?
• If I call this helper fifty times against one host, how many TCP connections should I expect to observe?
• What would break if I hoist the Session to module scope inside a threaded worker that also stores cookies?

The third question matters more than the first two combined. requests.Session is not a toy you share across threads without a lock. Hoisting it blindly can trade a handshake bug for a race around cookies, headers, and the adapter pool.

What broke when I "just reused Session"

The first pooled run still opened extra sockets because I passed a different host string. 127.0.0.1 and localhost are different pool keys. So are http:// and https://, and so is an extra trailing slash if some layer normalizes poorly. urllib3 pools by scheme, host, and port, not by "the API I mean in my head."

I also mixed a timeout on the Session with a timeout on the individual call, then could not explain which value actually won. Pick one layer and write it down beside the client. Ambiguous timeouts turn a pooling bug into a ghost story.

Keep-alive died once when a reverse proxy injected Connection: close. The Python client was pooling correctly. The hop in the middle refused to play along. The peer-port log on my stdlib server stayed stable while the proxy in front still forced new connections from the public side. If you only instrument the app process, you will miss that class of break.

Limitations, and who should skip this

This notebook is a connection-reuse check, not an API performance program. It will not replace a proper load test with independent arrivals, realistic payloads, or saturation of worker pools. I would not publish the mean of fifty serial gets as a capacity number, a percentile, or anything a customer should believe.

Skip this approach when any of the following is already true for your setup:

• You are unit-testing JSON decoding and never leaving the current process boundary.
• Your client already uses a long-lived Session, httpx.Client, or a multiplexer you have verified with peer logs.
• You need thread-safe sharing of cookies and connection pools across many workers.
• You do not control the remote host and cannot read its connection logs at all.
• You are tempted to treat one remote box as a soak-test cluster or an SLO source.

I also would not point a public URL at an unauthenticated conn_watch.py. The handler is a flashlight for source ports, not a service, and it will answer whoever knocks. Python's time.perf_counter is the right clock for this lab, and it is still not a tracing backend. If you need distributed traces, use traces.

What I would repeat

I would still start with a mean, because a mean is how I notice that something is off. I would not stop on localhost, and I would not trust a pool I cannot see in a log line. The peer-port print is the cheapest instrumentation I have found for this class of bug.

I would repeat the two-client harness before I rewrite a backend handler. I would repeat the decision table before I reach for a profiler. I would ask a model only after the server log already shows jumping ports, so the review has a failing example instead of a vague complaint about "latency."

Would I still run fifty serial requests and then call the result a benchmark that staging should match? No. I would run them as a connection audit, then hand real load to a tool that understands concurrency and independent arrivals. The forty-eight hours were expensive in attention. The artifact is small enough that the next time should take forty-eight minutes.

The next time a client looks fast, I will ask a ruder question first. Am I timing the work, or am I timing a kernel that never left my own machine?