CI Is Green. The Behavior Is Gone. A Receipt Gate for AI-Generated C++ Patches.

Your CI pipeline just turned green, and the AI-generated patch looks clean. Why does that combination still make me nervous?

Because green CI proves the code compiles and the tests you remembered to write still pass, and it says nothing about the behavior you fixed three months ago and forgot to protect. I have watched a model-generated patch sail through review, pass every unit test, and quietly reintroduce a bug the team had already fixed twice. The fix is not more tests. The fix is a receipt gate: a small, repeatable check that compiles the candidate, runs a fixed set of behavior receipts, and reports per-case results.

Receipts are not test cases. They are promises the code must keep.

Step 1: Write behavior receipts, not test cases

A test case documents what you think the code should do. A receipt documents what the code once did and must keep doing. The difference is the intent, and the intent changes what you store.

Each receipt needs an id, an input, an expected output, and a tolerance. The id is the part that saves you later, because a score without per-case ids is just a number. When the total stays the same but one receipt flips from PASS to FAIL, the id tells you which promise broke.

[
  {
    "id": "normalize-whitespace",
    "input": "  hello   world  ",
    "expected": "hello world",
    "tolerance": 0
  },
  {
    "id": "normalize-utf8",
    "input": "caf\u00e9",
    "expected": "caf\u00e9",
    "tolerance": 0
  },
  {
    "id": "normalize-empty",
    "input": "",
    "expected": "",
    "tolerance": 0
  }
]

Three receipts beat thirty that all exercise the same path. The UTF-8 case is the one that catches the regression. The empty case is the one nobody writes until production finds it.

Step 2: Compile the candidate, run the receipts

The gate is deliberately boring. It compiles candidate.cpp, runs it against each receipt, and compares stdout with the expected value. The intelligence lives in the receipts and the grader, not in the runner.

import json
import subprocess
import sys

def build():
    return subprocess.run(
        ["g++", "-std=c++17", "-O2", "candidate.cpp", "-o", "candidate"],
        capture_output=True,
        text=True,
    )

def run_receipt(receipt):
    result = subprocess.run(
        ["./candidate"],
        input=receipt["input"],
        capture_output=True,
        text=True,
        timeout=5,
    )
    return result.stdout.strip()

def grade(receipt, actual):
    expected = receipt["expected"]
    tolerance = receipt.get("tolerance", 0)
    if tolerance == 0:
        return actual == expected
    try:
        return abs(float(actual) - float(expected)) <= tolerance
    except ValueError:
        return False

def main():
    build_result = build()
    if build_result.returncode != 0:
        print("BUILD FAILED")
        print(build_result.stderr)
        sys.exit(1)
    with open("receipts.json") as fh:
        receipts = json.load(fh)
    passed = 0
    for receipt in receipts:
        actual = run_receipt(receipt)
        ok = grade(receipt, actual)
        passed += ok
        print(f"[{'PASS' if ok else 'FAIL'}] {receipt['id']}")
        if not ok:
            print(f"  expected: {receipt['expected']!r}")
            print(f"  actual:   {actual!r}")
    print(f"{passed}/{len(receipts)} receipts kept")
    sys.exit(0 if passed == len(receipts) else 1)

if __name__ == "__main__":
    main()

The runner is short because it should be run, not maintained. A gate that needs infrastructure will be skipped. A gate that fits in one file will be run on every patch.

Step 3: Route the whole loop through a free tier

Now the workflow gets practical. The model generates a candidate patch, and the gate verifies it. Both halves cost something: model calls consume tokens, and a build machine costs time to provision.

MonkeyCode's free models can generate the candidate patch, and its free server can run the gate without you spinning up a VM. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That combination removes the two excuses that kill eval loops: "I don't want to pay for model calls" and "I don't have a box to run this on." The gate stays the same whether it runs on a laptop, a CI runner, or a free server. The artifact is the receipts, not the hosting.

Treat the free tier as a claim to verify, not a permanent fact. Free models and free servers change quotas and availability without warning, so pin the versions you rely on and re-check the policy before you build a workflow around it.

Grader selection: a decision table

The grader is a contract, and each contract has a failure mode. I use this table when I set up a gate for a new function.

Grader	Use when	Failure mode
exact	deterministic output like normalized strings	brittle to harmless formatting changes
numeric tolerance	measurements, timings, scores	hides slow drift toward the boundary
regex	logs and error messages	narrow; misses semantic changes
LLM rubric	subjective summaries, release notes	noisy; needs a versioned prompt

Exact match is the default because it is cheap and impossible to argue with. The other graders exist for cases where exact match is wrong, not for cases where it is inconvenient. An LLM rubric can grade things with no single correct form, but a noisy grader turns a regression detector into a coin flip.

Where this gate breaks

The gate is only as good as its receipts. Stale receipts rot: a promise that no longer matches the intended behavior fails forever, and teams start ignoring red rows. Review receipts the same way you review code, because they are code.

The gate does not prove correctness. It proves that a fixed set of behaviors survived. Fuzzing, sanitizers, and formal verification cover different ground, and I have written about sanitizer harnesses and race detection separately. A receipt gate complements those tools; it does not replace them.

Who should not use this: teams with no existing test culture, because the gate becomes another ignored report. Teams with a spec that changes weekly, because the receipts chase a moving target. And teams that need proof rather than signals, because receipts are evidence, not proof.

The next patch you review

Your next AI-generated PR deserves a receipt, not just a green checkmark. Pick a function that has burned your team before, write five receipts, and run the gate before you approve. The first regression it catches will pay for the whole setup.