A Two-Gate Harness for AI-Generated Patches You Can Run on a Free Server

A generated patch can be visually coherent, pass CI, and still break something no reviewer was planning to inspect. The usual failure is not the model being clever; it is the review process treating 'looks okay' as 'safe to merge.' This article describes a small reproducible harness that puts AI-generated patches through two cheap gates—static shape checks and a bounded runtime smoke test—before any human has to spend time on the diff.

The environment used for this pattern is MonkeyCode's free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What the two gates are for

Gate 1 is a shape check. It answers three questions:

• Does the patch apply cleanly to the current HEAD?
• Does it change only the files it claims to change?
• Does it avoid protected paths such as migrations and secrets?

Gate 2 is a runtime smoke test. It applies the patch in a throwaway Git worktree, runs one narrow smoke-test target, and rejects the patch if the target fails or times out.

Neither gate proves correctness. They act as a triage filter: they stop the patches that are not worth human review and leave the interesting risks for a person. That is especially useful when you generate several candidate patches at once from a free model endpoint. You can let the harness discard the noise and review only the candidates that survive.

A runnable harness

The script below implements both gates with standard Git and a shell. Keep it in the repository as tools/ai-patch-gate.sh, make it executable, and run it against a candidate .patch file.

#!/usr/bin/env bash
set -euo pipefail

repo="${REPO_DIR:?set REPO_DIR to the repository root}"
patch_file="${1:?usage: tools/ai-patch-gate.sh /path/to/candidate.patch}"
timeout_s="${TEST_TIMEOUT_S:-120}"

# Gate 1a: the patch must apply to HEAD without conflicts.
if ! git -C "$repo" apply --check "$patch_file"; then
  echo "REJECT: patch does not apply cleanly to HEAD" >&2
  exit 2
fi

# Gate 1b: reject patches that touch sensitive or high-risk paths.
scope="$(git -C "$repo" apply --numstat "$patch_file")"
if grep -Eq '(^|/)migrations/|(^|/)secrets/|(^|/)\.env' <<< "$scope"; then
  echo "REJECT: patch touches a protected path" >&2
  echo "$scope" >&2
  exit 3
fi

# Gate 2: run the smoke target in an isolated worktree.
scratch="$(mktemp -d)"
git -C "$repo" worktree add --detach "$scratch" HEAD >/dev/null
trap 'git -C "$repo" worktree remove --force "$scratch" >/dev/null 2>&1 || rm -rf "$scratch"' EXIT

git -C "$scratch" apply "$patch_file"
if ! timeout "$timeout_s" make -C "$scratch" smoke-test; then
  echo "REJECT: smoke-test failed or timed out" >&2
  exit 1
fi

echo "ACCEPT_FOR_REVIEW: apply, scope, and smoke-test passed"

The smoke-test target should be specific to the change, not the full suite. For a patch that modifies a cache layer, the target might rebuild the module and run one read/write sequence. For an API patch, it might start the server and issue one request against the affected route.

If the free server option provides a shell, the same script can run there. If it is a web runtime instead, replace the make line with a request to the deployed candidate and keep the timeout. The free model access is useful for producing candidate patch files to feed through the harness; the harness itself does not require any particular model.

How to read the results

Output	What it means	Next action
REJECT: patch does not apply cleanly	The candidate was generated against a different base or contains malformed text	Discard or regenerate against HEAD
REJECT: patch touches a protected path	The change is broader than intended or includes risky files	Narrow the patch or require a human sign-off
REJECT: smoke-test failed or timed out	The candidate breaks behavior covered by the smoke target	Save the log, then either fix the prompt or discard the patch
ACCEPT_FOR_REVIEW	The patch passed the cheap gates and is worth human review	Review the remaining risk areas: edge cases, performance, compliance

This table is more useful than a single CI pass/fail because it separates automatically detectable failure from a reviewable candidate. A passing harness result should not be read as an endorsement of the patch; it only means the patch met the minimum conditions you decided in advance.

Limitations and when to skip this

The harness catches only the failure classes that your smoke test already knows about. A patch can pass by deleting the test that would have failed, by changing behavior not covered by the smoke target, or by introducing a slow path that does not time out. It does not validate security properties, database migrations, user-facing accessibility, or effects on stateful services.

If a patch cannot be tested in a bounded way—for example, a schema migration that requires a copy of production data, or a change to a payment flow where a failed smoke test could still have side effects—do not use this harness as the primary control. The same applies to teams without a reproducible smoke target.

This approach is also not a substitute for reviewing the parts that are hard to automate. The point is to spend that reviewing time on the candidates that are most likely to have subtle problems, not to eliminate review entirely.

Start with one module

The smallest useful version is not a complete CI pipeline. Pick one module with a reproducible smoke target, add the script, and run it against the next five generated patches. Record which reject reasons occur most often; that tells you what the generator is getting wrong and which rules to tighten.

Small, automatic rejection rules usually improve a review queue faster than a stronger model. Once the harness is running, you can evaluate generated patches by the failures they do not produce, not by how confident they look.

If you want to try this pattern with a free model prompt and a free server, start by defining the smoke-test target for one module and then feed the next generated patch through the gate.