Matching Prints Hide Freed Storage: A Twin-Run Sanitizer Gate for C++ Patches

A coding-model case accepted a C++ patch because the program printed case-7 and exited zero. The same binary, started again with the same arguments, printed that line a second time. Nothing in the functional golden file moved. A sibling build, compiled with AddressSanitizer and UndefinedBehaviorSanitizer, aborted while materializing the return value. The characters in the golden file had been read from storage that the patch had already deleted.

A green print is a contract check. It is not a memory check, and it is not a stability check. The workflow below records three bits for each case: functional match, twin-run stability, and a sanitizer-clean lane. A lucky first sample cannot close the case by itself.

What the three bits mean

The harness grades one fixture at a time. It does not rank vendors, and a full accept is not a production sign-off. Every listing in this article is a proposed lab script. None of it was executed against a named model for this draft, and none of it is a benchmark table.

The model may replace src/impl.cpp only. The driver, the expected stdout, and the grader stay read-only. A patch that touches those paths is rejected before any bit is computed. Permission bits on the oracle are a tripwire for accidental edits, not a security boundary. A separate account or a read-only mount is what actually keeps an untrusted patch away from the expected file.

1. Freeze a fixture the patch cannot edit

The operator creates the case tree once, then removes write permission from the driver and the golden stdout before any generation step.

mkdir -p cases/label7/src cases/label7/expected cases/label7/work
cat > cases/label7/src/main.cpp << 'EOF'
#include <iostream>
#include <string>

std::string label_for(int id);

int main() {
  std::cout << label_for(7) << '\n';
  return 0;
}
EOF
printf 'case-7\n' > cases/label7/expected/stdout.txt
chmod a-w cases/label7/src/main.cpp cases/label7/expected/stdout.txt

The implementation under test still tends to emit the expected line on a quiet allocator. That tendency is not a guarantee. Use after free is undefined on the functional build as well, which is why a single green print was never evidence. The copy into out is a decoy. The returned object is built from a freed buffer.

#include <string>

std::string label_for(int) {
  std::string* heap = new std::string("case-7");
  std::string out = *heap;
  delete heap;
  return *heap;
}

A second fixture covers drift that is not a heap error. The accumulator is uninitialized, so agreement between two runs is a property to measure, not a courtesy to assume.

int fold_ids(int n) {
  int acc;
  for (int i = 0; i < n; ++i) acc += i;
  return acc;
}

The prompt for this second case asks for the sum of 0 .. n-1. A first print of 45 when n is 10 can collide with a golden file. That collision is one sample. The twin bit exists so the sample cannot stand alone.

2. Build a functional binary and a sanitized sibling

Both binaries compile from the same pair of translation units. The functional build turns warnings into errors, so a patch that only survives under -w never reaches the behavior bits. The sanitized sibling keeps frame pointers and debug info so an abort names a line.

CASE=cases/label7
mkdir -p "$CASE/work"
g++ -std=c++20 -O1 -Wall -Wextra -Werror \
  -o "$CASE/work/fn" "$CASE/src/main.cpp" "$CASE/src/impl.cpp"
g++ -std=c++20 -O1 -g -fno-omit-frame-pointer \
  -fsanitize=address,undefined \
  -o "$CASE/work/san" "$CASE/src/main.cpp" "$CASE/src/impl.cpp"

Failure of either command records build_fail and skips the behavior bits. A binary that never linked does not receive a partial behavior score. The fragments in this article are sequential parts of one lab session. They are not a packaged installer, and they assume the previous variables are still in the shell.

3. Require the same stdout on two runs

The functional binary runs twice from the case work directory. Locale is pinned. Arguments stay identical. The first stdout must match the golden file, and the second stdout must match the first byte for byte.

export LC_ALL=C
( cd "$CASE/work" && ./fn > run1.out )
status1=$?
( cd "$CASE/work" && ./fn > run2.out )
status2=$?
diff -q "$CASE/expected/stdout.txt" "$CASE/work/run1.out"
func_match=$?
diff -q "$CASE/work/run1.out" "$CASE/work/run2.out"
twin_stable=$?

A non-zero process status fails the functional bit even when the captured text looks plausible. This script does not bound runtime. Operators who need a bound wrap each launch with timeout 2 and treat status 124 as a functional failure. That limit is local lab policy, not a property of any model host.

4. Score sanitizer stderr as its own bit

The sanitized binary is not asked to reproduce the golden file for the accept decision. Its exit status and diagnostic stderr are the signal. A zero exit and no sanitizer diagnostic set the bit. Any other outcome clears it.

( cd "$CASE/work" && ./san > san.out 2> san.err )
san_status=$?
if [[ $san_status -eq 0 ]] && ! grep -E -q 'AddressSanitizer|UndefinedBehaviorSanitizer|runtime error:' san.err; then
  sanitizer_clean=1
else
  sanitizer_clean=0
fi

Leak detection stays enabled. A case that intentionally keeps a process-lifetime buffer should say so in the prompt and may export ASAN_OPTIONS=detect_leaks=0 for that case alone. Turning leaks off for the whole suite would erase a defect class this lane is meant to keep visible. Partial stdout from an aborted sanitizer run is ignored. The accept bit does not reward a crash that happened to flush the golden line first.

5. Fold the bits into one verdict file

The recorder stores a single JSON object. Later summaries may count accepts. They may not promote one true bit into a pass.

functional=false
twin=false
sanitizer=false
if [[ $func_match -eq 0 && $status1 -eq 0 && $status2 -eq 0 ]]; then
  functional=true
fi
if [[ $twin_stable -eq 0 && $status1 -eq 0 && $status2 -eq 0 ]]; then
  twin=true
fi
if [[ $sanitizer_clean -eq 1 ]]; then
  sanitizer=true
fi
accept=false
if [[ $functional == true && $twin == true && $sanitizer == true ]]; then
  accept=true
fi
cat > "$CASE/work/verdict.json" << EOF
{
  "case": "$CASE",
  "functional": $functional,
  "twin": $twin,
  "sanitizer": $sanitizer,
  "accept": $accept
}
EOF

functional	twin	sanitizer	accept	reading
true	true	true	true	stdout held twice and instrumentation stayed quiet
true	true	false	false	expected text hid a memory or undefined-behavior fault
true	false	true	false	the first print matched and the second print drifted
false	any	any	false	the functional contract failed; other bits stay diagnostic
build failed	—	—	false	no behavior score is recorded

An accept means the case may move to human review. It does not mean the patch is free of data races, of logic errors outside the fixture, or of faults inside uninstrumented libraries.

6. Archive the failure before touching the prompt

When accept is false, sanitizer stderr is read first if the sanitizer bit is false. The two stdout files are compared next if the twin bit is false. The prompt stays frozen in that same step. Editing the prompt and the patch together makes the next verdict incomparable to the one just archived.

mkdir -p logs
cp "$CASE/work/verdict.json" logs/label7-verdict.json
if [[ -f "$CASE/work/san.err" ]]; then
  cp "$CASE/work/san.err" logs/label7-san.err
fi

The log directory sits outside the case tree so a later patch cannot overwrite the evidence. Only after that copy does the operator delete work/ and request another implementation. A rerun that changes compilers without noting the compiler identity will not be comparable either, so the archive step should also store the output of g++ --version beside the verdict.

Where free model access and a free server fit

The gate stands without a particular vendor. Something still has to draft impl.cpp, and the two binaries still have to run outside the editor that stored the prompt. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source coding assistant. The operator supplied two availability claims for this draft: free model access, and a free server option. No token quota, hardware size, rental duration, or permanence claim is stated here. Those figures were not checked against a current primary document, and unpublished token counts are not treated as measurements.

One workable split sends the prompt through free model access and writes the reply into src/impl.cpp. The grade then runs in a disposable tree, with the expected file mounted read-only. The free server option matters when that tree should not live on a laptop. Sanitizer links often demand more memory than an ordinary -O1 binary. If the free server cannot host the sanitized sibling, generation can remain on the free model path and the sanitizer bit should move to a machine that can link it. Dropping the third bit to squeeze onto a small host recreates the original miss: a matching print, and an untested heap.

Keys do not belong in the case directory. The script reads sources and writes verdict.json only.

Limitations

AddressSanitizer and UndefinedBehaviorSanitizer miss logic bugs on paths the fixture never calls. They also miss data races unless a separate ThreadSanitizer build is added, and this script does not build one. A twin run will not expose a bug that is stably wrong. Two identical wrong lines still fail the functional bit, which is why the golden file sits outside the patch tree.

False sanitizer reports show up when a case links prebuilt objects that were not instrumented the same way. The commands assume a GNU-style g++, diff, and bash. A host without those tools will not produce comparable bits. Untrusted patches belong in a disposable environment with no credentials and no network. This article does not specify that sandbox. It only refuses to treat a local green print as a finished review.

Who should skip this gate

Language suites without a comparable instrumentation story will not gain a third bit by renaming the script. Leaderboards that need latency, throughput, or cross-model ranks need a different artifact. This gate covers a narrower procedural point: whether one patch kept its stdout contract across two runs without tripping the sanitizers already installed. Operators who cannot isolate untrusted builds should not aim a free server at raw patches and call the JSON a review.

Teams that already store golden stdout can add the twin diff and the sanitized sibling without rewriting prompts. If the current MonkeyCode documentation still lists free model access and a free server option, either can feed this case tree. The verdict file, not a product banner, decides whether the print was enough.