48-Hour Field Notes: My Stream Ended at [DONE] and the File Was Still Half a Function

The transport told me the response was finished, and I believed it for about six hours. My collector had exited cleanly, the ledger showed [DONE], and the artifact on disk ended mid-expression inside a perfectly ordinary function body. Nothing raised, nothing logged, and the only reason I caught it was a linter run I almost skipped.

That is the failure mode I keep hitting when the model is free and the box is free: nothing is broken enough to page you. Over two days I rebuilt my collection loop around a simple rule — a stream is not a message, and only the message can be verified. This is what I tried, what broke, and what I would repeat on the next long job.

Why free access changes the shape of the bug

When inference costs real money per token, you naturally keep outputs short and your retry budget tight. Free model access plus a free server option inverts that incentive: long generations feel affordable, so you ask for whole modules, and you run the client somewhere you are not watching. Availability is the part I was given; latency, quotas, and durability are the part I refuse to assume.

So the interesting question is not "is the model good enough?" — it is "what does a truncated answer look like from the outside?" A dropped read timeout, a proxy that closes an idle connection, or a sleeping laptop all produce the same artifact: a prefix. And a prefix of valid code is often still valid code, which is exactly why it survives your smoke test.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Three files and one rule

My rule is boring: never append to a file you cannot describe. The collector keeps three artifacts in one directory, and each one answers a different question.

1. answer.part — the raw bytes received so far, fsynced after every chunk.
2. ledger.jsonl — append-only events: chunk offsets, prefix hashes, stream errors, verification verdicts.
3. prompt.txt — the exact request text, written once before the first byte arrives.

Why the prefix hash and not just a byte count? Because a byte count tells you how much you have, while a hash tells you whether the thing you are about to continue is the thing you actually collected. If a resume writes into a different file than the one you hashed, you will not notice until the diff is unreadable.

The collector, labelled honestly

The script below is the version I would keep. Treat it as a lab artifact: it is unverified as written against your endpoint, so read it before you run it, and fill in the environment variables yourself.

#!/usr/bin/env python3
"""stream_ledger.py - collect a long completion with a resumable, verifiable ledger."""
import hashlib, json, os, pathlib, time
import requests  # pip install requests

BASE  = os.environ["MC_BASE_URL"].rstrip("/")   # OpenAI-compatible base URL
KEY   = os.environ["MC_API_KEY"]
MODEL = os.environ.get("MC_MODEL", "your-model-id")
SENTINEL = "# ---END---"

WORK = pathlib.Path("run"); WORK.mkdir(exist_ok=True)
PART, LEDGER = WORK / "answer.part", WORK / "ledger.jsonl"


def note(event, **fields):
    rec = {"t": round(time.time(), 3), "event": event, **fields}
    with LEDGER.open("a", encoding="utf-8") as f:
        f.write(json.dumps(rec, sort_keys=True) + "\n"); f.flush(); os.fsync(f.fileno())


def append_chunk(text):
    with PART.open("a", encoding="utf-8") as f:
        f.write(text); f.flush(); os.fsync(f.fileno())
    return PART.stat().st_size


def prefix_sha():
    h = hashlib.sha256()
    with PART.open("rb") as f:
        for block in iter(lambda: f.read(65536), b""):
            h.update(block)
    return h.hexdigest()


def stream(messages, read_timeout=90):
    with requests.post(f"{BASE}/chat/completions",
                       json={"model": MODEL, "messages": messages, "stream": True},
                       headers={"Authorization": f"Bearer {KEY}"},
                       stream=True, timeout=(10, read_timeout)) as r:
        r.raise_for_status()
        for line in r.iter_lines(decode_unicode=True):
            if not line or not line.startswith("data:"):
                continue
            data = line[5:].strip()
            if data == "[DONE]":
                note("transport_done", bytes=PART.stat().st_size, sha=prefix_sha()[:16])
                break
            try:
                piece = json.loads(data)["choices"][0]["delta"].get("content") or ""
            except (json.JSONDecodeError, KeyError, IndexError):
                note("bad_frame"); continue   # one malformed frame is not a fatal error
            if piece:
                size = append_chunk(piece)
                note("chunk", bytes=size, sha=prefix_sha()[:16])
                yield piece

Two details matter more than the rest. The [DONE] frame is logged as transport_done, never as complete, because those are different claims. And a malformed frame skips instead of raising, since a single bad line should not cost you the whole generation.

The verifier that refuses a plausible prefix

The prompt ends with an instruction: finish the code, then emit the exact line # ---END--- on its own line. That sentinel is the whole trick, because it distinguishes "the model finished" from "the socket closed."

def verify(text):
    if not text.rstrip().endswith(SENTINEL):
        return False, "no sentinel: model output was cut, not completed"
    body = text.rsplit(SENTINEL, 1)[0]
    try:
        compile(body, "<answer>", "exec")
    except SyntaxError as e:
        return False, f"syntax error at line {e.lineno}: {e.msg}"
    return True, "parsed"

Notice the order. The sentinel check runs first, so a truncated but syntactically valid prefix fails fast instead of sailing through compile. If you only ever run the parser, you will ship half a function with a green checkmark.

Resuming without repeating the model to death

On failure I send back the last ~1200 characters as an assistant turn plus one short instruction: continue exactly there, no preamble, no summary. Two resumes maximum, then the run fails loudly. jq reads the ledger afterwards:

jq -r 'select(.event=="verify") | "\(.attempt)\t\(.ok)\t\(.why)"' run/ledger.jsonl

The 48-hour log

What broke, roughly in the order it broke.

• Hour 2: I trusted [DONE]. Fix: a sentinel in the prompt, checked before anything else.
• Hour 9: a resume duplicated the tail because I re-sent the full prompt without the partial answer. Fix: continuation messages always carry the tail as an assistant turn.
• Hour 20: my verifier passed a file with balanced braces that ended inside a string literal. Fix: sentinel first, parser second, never the reverse.
• Hour 31: I had two answer.part files and hashed the wrong one. Fix: the ledger records the absolute path next to every hash.
• Hour 44: a long idle gap killed the connection and the retry loop ran four times silently. Fix: a resume cap of two, and a non-zero exit when it is hit.

What I would repeat without hesitation: one directory per run, fsync after every chunk, and a verdict line in the ledger for every attempt. The ledger is the only reason hour 31 was a ten-minute fix and not a rewrite.

Where this belongs, and where it does not

Workload	Free model + free server?	Guardrail
Long codegen you review before use	Yes	Sentinel, parser, two-resume cap
Batch docstring or test scaffolding	Yes	Same ledger, smaller outputs
Latency-sensitive request paths	No	No SLA to lean on
Regulated or secret data	No	You own the data-flow review, not the network
Anything you cannot re-run	No	Re-runnability is the whole safety net

Who should not use this? Anyone whose output feeds a deploy without a human read, anyone who cannot afford a second run, and anyone whose team will not look at ledger.jsonl when a job fails. If the ledger is noise to you, the workflow is just extra typing around a truncated file.

What I did not measure, and am not quoting

I am deliberately not publishing timings, throughput, or success rates here, because those numbers belong to your endpoint and your prompt shape, not to mine. The script prints its own offsets and verdicts, so your first run replaces every placeholder I could have invented. What I can say honestly is that the sentinel check caught truncations that compile() alone accepted, in more than one run.

If you already have free model access and a free server slot, the cheapest next step is small: point this collector at one long generation you already trust, and watch which hour your verify line flips to false after a transport_done.

FAQ

Is [DONE] ever a completion signal? It is a transport signal. The model's own end-of-output marker is the only thing that speaks for the content.

Do I need fsync on every chunk? It costs throughput and buys you a truthful file after a crash. For long codegen on a free box, I take the trade.

Can I skip the ledger if I only run once? You can, and then you will have no way to tell a truncated answer from a short one when it looks wrong.