The Retry That Fired After the Stream Ended

A stream can succeed and still look failed to your client. That gap is where retry logic doubles side effects. The current AI conversation is full of fatigue and agent demos. The useful question is smaller: what did the generated retry wrapper actually do?

I'll use a synthetic repro, not a customer incident. The failure is real enough to run on your laptop. It takes one endpoint, one client, and one assertion.

The Symptom: Two Jobs for One Submit

You hit POST once. The database shows two rows. No exception reached your logs.

The client logged a retry, because the stream ended without a terminal event. Is that a failure? Not necessarily. The server may have accepted the job and finished the work.

That is the trap. The client treats "no done event" as "request failed." That is a guess about server state, and it is often wrong.

Step 1: Build a Server That Closes Cleanly

Here is a small FastAPI app. It streams one progress event and then closes. It never sends a terminal event.

# app.py
import asyncio
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse

app = FastAPI()
CALLS: list[str] = []

@app.post("/jobs")
async def create_job(req: Request):
    body = await req.json()
    CALLS.append(body["id"])

    async def events():
        yield 'event: progress\ndata: {"pct": 50}\n\n'
        await asyncio.sleep(0.05)
        # BUG: no terminal event before close

    return StreamingResponse(events(), media_type="text/event-stream")

@app.get("/calls")
async def calls():
    return {"calls": CALLS}

Start it and watch the raw stream:

uvicorn app:app --port 8000
curl -N -X POST localhost:8000/jobs \
  -H 'content-type: application/json' \
  -d '{"id":"job-1"}'

The output stops after the progress event. No error. No done.

Step 2: Add the Retry Wrapper

Now add the client an assistant might generate. It looks reasonable. It retries when the sentinel is missing.

# client.py
import httpx

URL = "http://127.0.0.1:8000/jobs"

def run(job_id: str, attempts: int = 2) -> str:
    for attempt in range(1, attempts + 1):
        done = False
        with httpx.stream("POST", URL, json={"id": job_id}, timeout=5.0) as r:
            r.raise_for_status()
            for line in r.iter_lines():
                if line == "event: done":
                    done = True
        if done:
            return "ok"
        print(f"attempt {attempt}: no terminal event")
    raise RuntimeError("stream ended without done")

Write the test that matters. Count the POSTs, not the exceptions.

# test_repro.py
import httpx
from client import run

def test_one_job_per_submit():
    run("job-1")
    calls = httpx.get("http://127.0.0.1:8000/calls").json()["calls"]
    assert calls == ["job-1"]

Run it:

pytest -q test_repro.py

The assertion fails with ['job-1', 'job-1']. That is the whole bug in one line. One user action became two server-side jobs.

Step 3: Classify the Failure Before Retrying

Retry safety is a property of the phase, not the exception class. Walk through these questions in order.

1. Did the server receive any bytes? Check the access log or an idempotency store.
2. Did the failure happen before the first response byte? A retry is usually safe.
3. Did the stream break after the first byte? Query the job state instead of reposting.
4. Did the stream close cleanly after a 200? Treat it as success, then fix the contract.
5. Did the client time out while the server kept working? Retrying can duplicate the work.

Here is the decision table I keep next to the code.

Failure point	Safe to retry the POST?	Better action
Connection refused	Yes	Backoff, then retry
503 before body	Yes	Backoff, then retry
Stream closed mid-body	No	GET /jobs/{id}
Clean close, no sentinel	No	Mark success, fix server
Client timeout, server alive	No	Poll state by key

Would your client pass that table? Most generated wrappers key off except Exception, which ignores all five rows.

Step 4: Fix Both Sides of the Contract

The server must always end the stream. The client must never retry a non-idempotent POST blindly. Add an idempotency key and a terminal event.

# app.py (fixed)
SEEN: dict[str, dict] = {}

@app.post("/jobs")
async def create_job(req: Request):
    body = await req.json()
    key = req.headers.get("idempotency-key", body["id"])

    if key in SEEN:
        async def replay():
            yield 'event: done\ndata: {"ok": true, "replay": true}\n\n'
        return StreamingResponse(replay(), media_type="text/event-stream")

    SEEN[key] = {"id": body["id"], "status": "accepted"}

    async def events():
        yield 'event: progress\ndata: {"pct": 50}\n\n'
        yield 'event: done\ndata: {"ok": true}\n\n'

    return StreamingResponse(events(), media_type="text/event-stream")

In production, write the key before the work starts. Wrap the work in a transaction or a job record. Do not hold the whole stream in memory for a replay.

On the client, make the absence of a sentinel a state query, not a resubmit.

# client.py (fixed)
def submit(job_id: str, key: str) -> str:
    with httpx.stream(
        "POST", URL,
        json={"id": job_id},
        headers={"Idempotency-Key": key},
        timeout=5.0,
    ) as r:
        r.raise_for_status()
        saw_done = any(line == "event: done" for line in r.iter_lines())
    if not saw_done:
        raise RuntimeError("unknown state: check job by key, do not resubmit")
    return "ok"

That error message is the fix. It forces the next person to ask what the server already did.

Step 5: Keep a Four-Case Test Matrix

One happy-path test will not catch this. Test the phases.

Case	Expected POSTs	Expected jobs
Reject before headers	2	1
Close after first event	1	1
Clean close without sentinel	1	1
Duplicate key	2	1

Assert all three columns. A 200 response proves almost nothing here. The job count is the real contract.

Where the Free Model and Free Server Fit

MonkeyCode is described by its operator as an open-source project with free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator also describes 10 million free tokens as part of the offer. I have not verified model names, quotas, hardware, or how long the offer lasts. Treat those details as claims to check before you depend on them.

Here is the workflow I would use. It fits a free tier because the loop is short and the output is testable.

1. Ask the model for three client variants: strict sentinel, no sentinel, and resumable.
2. Run each variant against the four-case matrix on a disposable server.
3. Keep the variant that passes, and delete the rest.
4. Record the failure mode of each rejected variant in the test file.

That last step matters. The rejected variants become regression cases, not just deleted code.

Limitations are real. A shared free server is a poor place for latency numbers. Never paste production keys or customer payloads into a free model prompt. Free capacity is not an SLA, and the offer can change.

Who Should Skip This

Do not use this workflow for live incident response with customer data. Do not use it to publish benchmark claims. Skip it if you cannot read the generated client code before running it.

Use it for drills, for new endpoints, and for reviewing agent-written retries. The test is cheap. The duplicate job is not.

A retry is a statement about what the server already did. Make the statement true, then test it. If you want a sandbox for the matrix, the free tier is a reasonable place to start.