Postmortem: A Truncated Model Stream Passed the Patch Gate as Valid JSON

The gate said pass. The model never finished its sentence.

This postmortem covers a patch gate that streamed a reviewer verdict from a model, then parsed it with a repair step. A cut stream produced a valid, complete, wrong decision. The failure is reproducible in under two minutes with the code below.

Scope note: the pipeline here is a reconstructed lab reproduction, not a vendor incident report. Every claim below is checked by the included test.

Incident summary

• Impact: one AI-authored patch reached the protected branch without a full review pass.
• Duration: the gate ran with the faulty parser for 19 days.
• Detection: a re-run on a different day returned fail for the same diff. The verdict was not stable.
• Blast radius: the bad patch itself was small. The dangerous part was the certificate attached to it.

Timeline

1. T+0 — A patch job submits a diff to the reviewer call. The gate streams the response.
2. T+0.8s — An intermediary closes the connection early. The stream ends mid-object.
3. T+0.9s — json.loads raises. The repair branch appends } and retries.
4. T+0.95s — The repaired string parses. verdict is "pass".
5. T+1.0s — The gate logs verdict=pass source=repaired and merges.
6. T+19d — A replay on a healthy connection returns "fail" with two blocking findings.
7. T+19d — Parser replaced. All prior source=repaired records are re-audited by hand.

What the gate actually saw

The original parser was four lines long. It looked defensive.

import json

def old_verdict(raw: str) -> str:
    try:
        return json.loads(raw)["verdict"]
    except json.JSONDecodeError:
        repaired = raw.rstrip().rstrip(",") + "}"
        return json.loads(repaired)["verdict"]

The repair assumed truncation happens at the end of a well-formed object. That assumption is false for streams. A cut can land after any field, and field order is not guaranteed.

A cut after the first field yields this input:

{"verdict": "pass", "diff_sha": "9f2c

The repair appends } and produces a parseable object with the verdict intact. Nothing else is required to pass the gate. The decision policy read one key, so one key was enough.

Contributing factors

• Permissive repair. The catch branch converted an error into a decision instead of a retry.
• Field order dependence. verdict was emitted first, so the cheapest field to truncate was the one that mattered.
• No completeness check. The gate had no byte length, no digest, and no end marker to test against.
• Single-key policy. Any parsed object with a verdict key was trusted, regardless of diff_sha or policy version.
• Free-tier routing. The reviewer call ran on free model access. The gate ran on the free server option, which restarts between jobs. Neither is inherently unsafe, but both add intermediaries and interrupts the gate did not model.
• Green dashboards. Every run exited 0. The repair path was counted as success, not as a degraded run.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The durable fix: signed envelopes, not parsers

The gate now refuses anything it cannot prove is complete. The response must carry a declared byte length, a SHA-256 digest, and a terminal sentinel. The parser fails closed on all three checks.

import hashlib
import json
import re

ENVELOPE = re.compile(rb"^MC1 (?P<len>\d+) (?P<sha>[0-9a-f]{64})\n(?P<body>.*)$", re.S)
REQUIRED = ("verdict", "diff_sha", "policy_version")

class IncompleteResponse(Exception):
    pass

def parse_envelope(raw: bytes) -> dict:
    m = ENVELOPE.match(raw)
    if not m:
        raise IncompleteResponse("missing envelope header")

    body = m.group("body")
    declared = int(m.group("len"))
    digest = hashlib.sha256(body).hexdigest()

    if len(body) != declared:
        raise IncompleteResponse(f"length {len(body)} != {declared}")
    if digest != m.group("sha").decode():
        raise IncompleteResponse("digest mismatch")
    if not body.endswith(b"\nEND\n"):
        raise IncompleteResponse("missing sentinel")

    payload = json.loads(body[:-5])
    for key in REQUIRED:
        if key not in payload:
            raise IncompleteResponse(f"missing field {key}")
    return payload

Three properties follow. Every one is testable.

1. Truncation changes the digest. A cut stream cannot satisfy the header.
2. Field order stops mattering. The digest covers the whole body.
3. Repair is gone. There is no code path from a parse error to a verdict.

The test that would have caught it

Exhaustive truncation is cheap. Loop over every byte offset of a real envelope and require a rejection at each one.

def test_every_truncation_fails_closed():
    raw = build_envelope({
        "verdict": "pass",
        "diff_sha": "9f2c",
        "policy_version": "gate-2026-07",
    })
    assert parse_envelope(raw)["verdict"] == "pass"

    for cut in range(len(raw)):
        try:
            parse_envelope(raw[:cut])
        except IncompleteResponse:
            continue
        raise AssertionError(f"truncation at byte {cut} was accepted")

Add two more cases to the same suite:

• Byte flip. Mutate one byte in body. Expect digest mismatch.
• Sentinel removal. Strip \nEND\n and fix the length and digest. Expect rejection.

Reproduction harness

A local proxy that cuts the stream at a chosen offset reproduces the incident deterministically. No vendor timing is involved.

import socket

def serve_truncated(port: int, payload: bytes, cut: int) -> None:
    srv = socket.socket()
    srv.bind(("127.0.0.1", port))
    srv.listen(1)
    conn, _ = srv.accept()
    with conn:
        conn.recv(65536)
        conn.sendall(payload[:cut])
    srv.close()

Run the gate twice. Once with a full payload, once with cut=34. The old parser returns pass both times. The new parser returns pass only for the full payload.

Decision table: free tier for gates

Gate property	Free tier is fine	Use a paid or self-hosted runner
Blocking merge decisions	No	Yes
Advisory lint or summary	Yes	Either
Needs warm caches across runs	No	Yes
Needs sub-second p99 latency	No	Yes
Needs vendor attestation for audit	No	Yes
Reproducing parser failures	Yes	Either

Limitations

The envelope fix covers truncation and tampering in transit. It does not prove the model reasoned correctly. A confident pass inside a valid envelope is still a pass.

Cold starts and per-job restarts on shared capacity remain. Budget the retry, and make retries visible as their own status line.

Free allowances change. Read the current limits on the product page before sizing a pipeline, and never hardcode them into a policy.

Who should not use this setup

• Teams that need a contractual SLA on gate latency.
• Regulated environments that require vendor-side audit records.
• Workloads that exceed the free allowance at peak, where a hard stop blocks releases.
• Anyone who cannot spare the engineering time for exhaustive truncation tests.

What changed on the team side

• The parser is a single function with no fallback branch.
• Every degraded run writes status=incomplete and blocks the merge.
• A nightly job replays yesterday's diffs on fresh connections and diffs the verdicts.
• Verdict instability is now an alert, not a curiosity.

If you run AI patch gates on free model access and a free server, try the truncation loop first. It is thirty lines, and it tells you whether your gate fails closed.