The service had compiled clean for eight months. Then a routine dependency bump surfaced a new warning in a template-heavy translation unit. Finley Zhou, the engineer on call, sent the compiler diagnostic to a free model endpoint and asked for a minimal patch.
The patch looked reasonable. It changed one overload's constraint, removed a nested type alias, and touched three lines. In the default build it compiled. Finley almost merged it. Then he tested the patch against the other build modes the service actually shipped. It failed in debug, no-exceptions, and libc++.
That failure was not really a model problem. It was a validation gap. The model had optimized for the first build it saw. The service had four build modes, a canary test, and a deployment policy that required all of them to pass. Finley had reviewed the diff before those facts were checked.
After the incident, the patch process moved to a sidecar. The model suggestions were fetched through MonkeyCode's free model access, and the disposable build box was the free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The new rule was simple: a suggested diff is not a patch until it survives the patch court.
The patch court
The patch court is a small runner. It receives one proposed diff, applies it to a clean worktree, runs every required build preset, and then runs a canary. It never edits the real branch.
The reduced version looks like this:
#!/usr/bin/env python3
'''Reduced patch-court harness. Adjust paths and build presets before use.'''
import subprocess
import sys
import json
import pathlib
BASE = pathlib.Path('/work/service')
PATCH = pathlib.Path(sys.argv[1])
PRESETS = ['ci-debug', 'ci-release', 'ci-noexcept']
CANARY = 'build/ci-release/tests/canary'
def run(args, cwd, timeout=900):
return subprocess.run(
args,
cwd=cwd,
capture_output=True,
text=True,
timeout=timeout,
)
def main():
apply_check = run(['git', 'apply', '--check', str(PATCH)], BASE, timeout=60)
if apply_check.returncode != 0:
print(json.dumps({
'decision': 'REJECT',
'reason': 'patch-apply-failed',
'log': apply_check.stderr[-2000:],
}))
return 2
for preset in PRESETS:
build = run(['cmake', '--build', '--preset', preset], BASE, timeout=1200)
if build.returncode != 0:
print(json.dumps({
'decision': 'REJECT',
'reason': 'build-failed:' + preset,
'log': build.stderr[-3000:],
}))
return 3
canary = run([CANARY, '--small', '--json'], BASE, timeout=300)
if canary.returncode != 0:
print(json.dumps({
'decision': 'QUARANTINE',
'reason': 'canary-failed',
'log': canary.stdout[-2000:] + canary.stderr[-2000:],
}))
return 4
print(json.dumps({
'decision': 'ACCEPT',
'presets': PRESETS,
'canary': 'pass',
}))
return 0
if __name__ == '__main__':
sys.exit(main())
This is not a full CI system. It is a gate in front of code review. The output is deliberately small: ACCEPT, REJECT, or QUARANTINE. A human can read that in seconds.
The three build modes
The service used three presets. The debug preset enabled assertions and address sanitizer. The release preset enabled optimizations and stripped debug symbols. The no-exceptions preset disabled exceptions because the service ran in a compiled benchmark target where exception overhead was not allowed.
The original model patch had compiled under release. It broke under debug because it removed a type alias that an assert macro still referenced. It broke under no-exceptions because a fallback branch threw an object in a code path that could not be compiled without exception support. Those failures never showed up in the single default build.
The fix was not to throw away model suggestions. It was to make the build matrix non-negotiable.
The canary must be a lie detector
A passing build is not proof. A model can produce a patch that compiles and still changes behavior. The canary test checked exactly the behavior that the patch had touched.
#include <cassert>
#include <type_traits>
template <typename T, typename = void>
struct has_custom_sort : std::false_type {};
template <typename T>
struct has_custom_sort<T, std::void_t<decltype(T{}.sort())>> : std::true_type {};
static_assert(has_custom_sort<Widget>::value);
int main() {
Widget w;
w.sort();
return w.empty() ? 0 : 1;
}
The patch had changed the overload constraint so the trait no longer matched. The compiler caught that mismatch before the patch landed. That was the point.
Decision table
| Result | Decision | Next action |
|---|---|---|
| patch apply fails | REJECT | return the apply error to the model |
| any build preset fails | REJECT | return the first compiler error |
| canary fails | QUARANTINE | store logs and keep the patch out |
| all build presets and canary pass | ACCEPT | attach the evidence to the patch |
| patch touches more than eight files or more than two hundred lines | HUMAN_REVIEW | do not auto-apply |
The last row was important. A large patch from a free model needed a human even when it built. The sidecar enforced that policy instead of hoping reviewers remembered it.
Where the free server helps
The sidecar was cheap enough to run on every suggestion because the build box was disposable and remote. The service's operator pointed the runner at a free server instance rather than a paid CI runner. That changed the economics: the team could reject fifty bad patches a day without burning queue time or local disk space.
The free server did not need to be fast. It needed to be isolated. Each run started from a clean checkout. Each run removed the applied diff after the decision. A bad suggestion could not poison the next run.
Limitations
This approach is not a substitute for real CI. The patch court only ran a three-mode build and a small canary. It did not run integration tests, fuzzers, benchmarks, or platform-specific builds.
A free server may be ephemeral. The team did not assume a warm dependency cache. If the free server restarted, the next run fetched dependencies again. The logs were pushed to an object store before the box vanished.
Do not send confidential source code to a remote model without checking the data policy. This workflow is for code that can be shared with a model endpoint and a disposable build box. Regulated services, code with licensee data, or builds that require a fixed hardware profile should not copy it directly.
Passing the patch court also does not prove the model is correct. It proves the suggested diff survived a compact set of checks. A model can still introduce a subtle data race or a slow path that the canary never touches.
Closing
The useful lesson was not that a free model made a bad patch. The useful lesson was that Finley almost accepted it because the default build passed. The safety rail was not a stronger prompt. It was a three-way build and a canary that held the veto.
If a team already receives free model suggestions, the cheapest upgrade is not a larger model. It is a disposable build box that treats every suggested diff as an unverified patch until the compiler says otherwise.
Top comments (0)