I Trusted stream=True for 48 Hours. The Client Still Called response.text.

I needed a worker that could follow a chunked event stream without waiting for the remote side to hang up. That sounds like a one-hour job until you notice every test is green and the process is still blocked. Have you ever watched an HTTP client sit quiet while the server is clearly sending ticks? I spent forty-eight hours in that gap, and the notes below are what I would actually keep.

This write-up is a reconstructed field notebook, not a vendor benchmark and not a claim about production traffic. The snippets are labeled examples you can run locally. If a sentence sounds like a war story, treat it as a debugging script you can repeat, not as a metric.

The problem I actually had

The worker had to print each data: line as it arrived, then keep the socket open for the next tick. A coding assistant drafted the client, and the unit suite went green on the first try. Why would I doubt a function that already asserted tick-0 and tick-1 were present?

I doubted it because the process never emitted a line until the server closed. The timestamps on disk were clustered at the end of the run, not one second apart. If your stream looks batched at the finish line, are you streaming at all?

What I tried in the first night

I started with the usual network ghosts, because they are easy to name and hard to disprove. Each of the following ate a few hours and left the same symptom standing.

1. I blamed the reverse proxy for coalescing chunked responses into one body.
2. I blamed DNS caching after the hostname flipped between two loopback aliases.
3. I blamed missing Cache-Control headers on the event endpoint.
4. I blamed the assistant for forgetting stream=True, then found it already set.

That last item should have been the clue. The generated client set stream=True and then called response.text anyway. Does a flag help if the next line waits for EOF? I had been debugging the network while the buffer sat in user space.

The false-green test that wasted a day

The suite never opened a socket. It replaced requests.get with an object that already had a .text attribute. Of course the split on newlines worked. Of course the assertion saw both ticks. The test was checking a string constant, not a stream.

# example: a test that cannot fail the buffering bug
import requests

def read_events(url):
    response = requests.get(url, stream=True)
    response.raise_for_status()
    return response.text.splitlines()


def test_read_events_looks_fine(monkeypatch):
    class FakeResponse:
        text = "data: tick-0\n\ndata: tick-1\n\n"

        def raise_for_status(self):
            return None

    monkeypatch.setattr(requests, "get", lambda *args, **kwargs: FakeResponse())
    lines = read_events("http://example.invalid/events")
    assert any("tick-0" in line for line in lines)

Would you trust a streaming client whose only test never called iter_lines? I did, for too long. The mock made stream=True look intentional, and it hid the blocking read behind a fixture.

A tiny server that tells on you

I needed an endpoint that slept between writes so a buffered client would hang in public. The standard library is enough. Run this in one terminal and leave it alone.

# example: chunked ticks, one per second, then close
from http.server import BaseHTTPRequestHandler, HTTPServer
import time

class TickHandler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def do_GET(self):
        if self.path != "/events":
            self.send_error(404)
            return
        self.send_response(200)
        self.send_header("Content-Type", "text/event-stream")
        self.send_header("Cache-Control", "no-cache")
        self.send_header("Connection", "close")
        self.end_headers()
        for index in range(5):
            payload = f"data: tick-{index}\n\n".encode("utf-8")
            self.wfile.write(payload)
            self.wfile.flush()
            time.sleep(1)

if __name__ == "__main__":
    HTTPServer(("0.0.0.0", 8080), TickHandler).serve_forever()

Point the broken client at that port and watch the wall clock. If five ticks appear together after five seconds, the body was buffered. If they appear one second apart, the iterator is honest. Can your laptop lie about that? Yes, if the test never leaves the mock.

Where a second machine earned its keep

Laptop mocks and a local server still share one kernel, one clock, and one set of open files. I wanted a listener that was not my pytest process. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access to draft the first client, then ran the tick server on the free server option so the client had to cross a real network hop.

That hop did not prove product superiority, and I am not attaching quotas, model names, or hardware claims. It only removed the mock. The remote process printed ticks on a one-second cadence while my worker stayed silent until the handler returned. After that, response.text was no longer a theory.

The client I would keep

The fix is small, and the important part is the test shape around it. Read bytes as they arrive, fail fast on connect, and keep a bound on stalled reads.

# example: iterate lines without waiting for the server to hang up
import time
import requests

def read_events(url):
    with requests.get(url, stream=True, timeout=(3, 30)) as response:
        response.raise_for_status()
        for raw_line in response.iter_lines(decode_unicode=True):
            if raw_line:
                yield raw_line, time.monotonic()

Then assert cadence, not just membership. A stream that delivers tick-0 and tick-1 in the same millisecond is still a buffer.

# example: integration check against a live listener
import time

def test_ticks_arrive_spread_out():
    stamps = []
    for line, seen_at in read_events("http://127.0.0.1:8080/events"):
        if line.startswith("data:"):
            stamps.append(seen_at)
        if len(stamps) == 3:
            break
    gaps = [stamps[i] - stamps[i - 1] for i in range(1, len(stamps))]
    assert min(gaps) >= 0.5
    assert max(gaps) < 5.0

Is this a perfect contract? No. Proxies can still buffer. It is enough to catch the generated .text footgun before you ship the worker.

Decision table I taped above the keyboard

Symptom	Looks like	Check this first	Reject this fake fix
All events print together at the end	Slow upstream	response.text or response.content	Raising the read timeout
Client hangs, server logs show writes	Network stall	Iterator not used; mock still in place	Retrying the same GET
Unit tests green, staging silent	Flaky host	Test patched requests.get	Adding more fixtures
First tick delayed, later ticks fine	Cold start	Connect timeout vs read timeout	Removing timeouts
Remote hop batches, laptop does not	Proxy buffer	flush() on server and iter_lines on client	Switching libraries blindly

I kept that table because the assistant kept offering the left column as a root cause. The right column is what I actually needed to refuse.

Commands I reran until they bored me

When the story got muddy, I stopped reading stack traces and started measuring bytes and time. These are the boring commands that ended arguments.

# watch the server write without a Python client in the way
curl -N -v --max-time 20 http://127.0.0.1:8080/events

# confirm the worker is blocked in a read, not in DNS
ps -o pid,etime,wchan,command -p "$WORKER_PID"

# compare first-byte time against full-body time
curl -N -w 'ttfb=%{time_starttransfer} total=%{time_total}\n' -o /tmp/events.out http://127.0.0.1:8080/events

If curl -N prints ticks on a cadence and Python prints them in one blast, stop blaming the network. The client is collecting the body. If both tools batch, look at the server flush() path or an intermediary.

What broke after the obvious fix

Switching to iter_lines did not end the notebook. Two follow-on failures showed up as soon as the mock was gone, and both are worth expecting.

• iter_lines can hold a partial line across chunks, which is correct, until you log the empty separator lines as errors.
• A connect timeout of three seconds is fine; a read timeout that fires between ticks is not, if your idle gap is longer than the timeout.
• decode_unicode=True hides a bad encoding until a non-ASCII payload arrives. Keep the raw bytes in the debug log.
• Running the tick server on 127.0.0.1 and the worker in a container still looks like a remote bug. Bind and connect to the address you actually published.

Did the assistant cause all of that? No. It caused the first lie. The rest appeared only after I forced a live socket.

What I would repeat

I would still let a model draft the first client, because the happy-path shape is tedious to type. I would not let it own the test double for the HTTP library. I would keep one integration check that records monotonic timestamps, and I would run that check against a listener that is not in-process.

I would also keep the curl control experiment. If the control streams and the client does not, the next patch belongs in the reader, not in DNS, retries, or a larger timeout. Would I skip the second machine when the control already fails on localhost? Yes. The extra hop is for when localhost and pytest are in on the joke.

Limitations, said plainly

This workflow catches buffered reads and dishonest mocks. It does not certify an event bus, and it does not measure throughput. A free remote listener is not a staging environment, not an SLA, and not a place to send secrets or personal data.

Chunked encoding can still be flattened by a proxy, a TLS terminator, or a helper that calls .json(). The cadence test can flake if the host is overloaded and sleeps longer than your upper bound. Tune the gaps to the server you actually run, and do not treat the table as a benchmark.

Who should not use this approach

Skip this if you already have a dedicated staging host and a contract test that talks to it. Skip it if your stream is a file on disk, because HTTP is not the bug. Skip it if you need guaranteed uptime, compliance boundaries, or multi-region failover from the extra machine.

Also skip it if the failure is CPU-bound serialization. A second socket will not explain a stuck encoder. And if you cannot run even the standard-library server, fix that packaging problem before you invite another network hop.

The forty-eight hours were not about streaming theory. They were about a green bar that never opened a socket, and a flag named stream=True that did not stream. Next time the ticks arrive in one polite paragraph, I will ask the same rude question first: who actually called iter_lines?