A Free Model Suggested a C++ Fix. The Subprocess Contract Decided Whether It Could Merge.

#cpp #ai #testing #devops

Text review is the wrong security boundary for generated C++ patches. A patch can be syntactically valid, compile cleanly, and still exhaust memory, change ABI, or pass the wrong contract. The decision that matters is not whether a suggestion looks reasonable; it is whether the patched tree survives build, sanitizer, and resource limits in isolation. This article turns that decision into a subprocess contract: extract the diff, apply it in a worktree, compile under ASan/UBSan, run a fixed probe, and record the evidence.

A free endpoint makes the tempting path cheap enough to become dangerous. If the first step is "read the answer and paste it," a model can produce plausible C++ that breaks only under conditions a human reviewer will not simulate. The endpoint becomes useful when its output is treated as untrusted input.

MonkeyCode's free model access is one way to source candidate suggestions, and its free server option can host the runner if the account provides a runnable container. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The workflow treats both options as black boxes; no model-specific API, quota, or endpoint behavior is assumed.

Why text review loses the C++ cases

Generated C++ fails in ways that are invisible in a diff view. A change can introduce undefined behavior that only ASan catches. It can allocate in a tight loop and hit the test runner's memory limit. It can change overload resolution or ABI while the words around it stay plausible. Text review defaults to accepting likely-looking changes. A subprocess gate defaults to rejecting until evidence arrives.

The harness below is small enough to run in CI or a container, and it outputs a single JSON verdict. It does not attempt to understand the model's reasoning. It only asks whether a patch can build and pass a deterministic probe under resource limits.

Contract stages

The input is a Markdown file with exactly one unified diff block. The output is summary.json with one of three verdicts: reject, accept_for_human_review, or no_diff_found. The stages are:

Extract the first fenced diff or patch block.
Apply it in a detached git worktree.
Build the patched tree with address and undefined-behavior sanitizers.
Run a fixed contract_runner under a wall-clock timeout and memory cap.
Record the stage, exit code, sanitizer log, and raw patch path.

Any stage failure stops the chain. A pass at stage 4 is still not a merge approval; it is only the minimum evidence for a human maintainer.

The intake script

Save this as intake_suggestion.sh. It expects a suggestion file, a git repository, and an output directory.

#!/usr/bin/env bash
set -euo pipefail

suggestion_md="${1:?usage: intake_suggestion.sh suggestion.md repo output}"
repo="${2:?}"
out="${3:?}"

mkdir -p "$out"
summary="$out/summary.json"

awk '
  /^```

(diff|patch)?[[:space:]]*$/ {in_block=1; next}
  /^

```[[:space:]]*$/ {if (in_block) exit}
  in_block {print}
' "$suggestion_md" > "$out/candidate.patch"

if ! grep -qE '^--- ' "$out/candidate.patch" || ! grep -qE '^[+][+][+] ' "$out/candidate.patch"; then
  printf '{"verdict":"reject","stage":"extract","reason":"no_unified_diff"}\n' > "$summary"
  exit 0
fi

worktree="${out}/worktree"
git -C "$repo" worktree add --detach "$worktree" HEAD >/dev/null
trap 'git -C "$repo" worktree remove --force "$worktree" 2>/dev/null || true' EXIT

if ! git -C "$worktree" apply --check "$out/candidate.patch" 2>"$out/apply.err"; then
  printf '{"verdict":"reject","stage":"apply","reason":"patch_does_not_apply"}\n' > "$summary"
  exit 0
fi
git -C "$worktree" apply "$out/candidate.patch"

cmake -S "$worktree" -B "$worktree/build" -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -O1" >"$out/cmake.log" 2>&1

cmake --build "$worktree/build" -j2 >"$out/build.log" 2>&1

set +e
( ulimit -v 1048576; timeout 10s "$worktree/build/contract_runner" ) >"$out/run.log" 2>"$out/sanitizer.log"
code=$?
set -e

if [ "$code" -eq 0 ] && [ ! -s "$out/sanitizer.log" ]; then
  verdict="accept_for_human_review"
  stage="run"
  reason="contract_passed"
else
  verdict="reject"
  stage="run"
  reason="contract_failed_or_sanitizer_signal"
fi

printf '{"verdict":"%s","stage":"%s","reason":"%s","exit_code":%d}\n' "$verdict" "$stage" "$reason" "$code" > "$summary"

The ulimit -v cap is a coarse guard, not a container boundary. Run the whole script inside a container when the patch comes from an external service, not on a developer workstation.

A deterministic contract runner

The harness runs a fixed binary, so the repository must provide one. Keep it small, offline, and deterministic. The point is to catch crashes, sanitizer findings, and runaway resource use, not to validate every feature.

#include <cstddef>
#include <iostream>
#include <vector>

int main() {
  constexpr std::size_t n = 1 << 16;
  std::vector<int> values;
  values.reserve(n);
  for (std::size_t i = 0; i < n; ++i) {
    values.push_back(static_cast<int>(i & 0x7fff));
  }
  if (values.size() != n) return 1;
  std::cout << "contract: ok\n";
  return 0;
}

In a real repository, replace this probe with the smallest test target that exercises the code path touched by the candidate patch. The key property is that a passing run gives a concrete artifact, while a failing run gives an actionable sanitizer log.

Decision table

Stage	Check	Failure verdict
extract	exactly one unified diff block	`reject`
apply	`git apply --check`	`reject`
build	compiler + ASan/UBSan build	`reject`
run	exit 0 within 10s and memory cap	`reject`
evidence	`summary.json` written	no merge without it

The table exposes a common mistake: teams treat build as the final gate. The run stage is where many model-generated C++ failures appear.

What the harness does not prove

A passing contract_runner is not a correctness proof. It proves the patched tree compiled and passed one deterministic probe under one set of limits. It does not prove thread safety, absence of data races, portability, or algorithmic complexity. It also depends on the probe being relevant to the patch.

Free model output can be non-deterministic. Record the raw suggestion and the commit hash before applying the patch, so a later reproducer starts from the same bytes. A second run may produce a different suggestion, which is why the patch file, not the chat transcript, is the artifact under test.

Teams that should avoid this approach include those without a reproducible build, those without a maintainer able to read sanitizer output, and those working on safety-critical code where a subprocess gate would create false confidence. In those cases, generated patches should remain read-only suggestions until they pass the normal review process.

If you already have MonkeyCode free model access, the smallest upgrade is not another prompt. It is feeding every generated diff into a subprocess gate and only letting a human review the ones that survive.

DEV Community