A Zero-Budget Verification Layer for Agent Patches: Compile, Property, and Bound Before Merge

#testing #ai #workflow #cpp

Whether the patch comes from a human or an AI agent, the risk is the same: it looks correct in review, but a rare input breaks an invariant you did not test. The difference is that an agent can produce many patches quickly, each one confidently stated. You need a cheap, repeatable gate that sits before human review and answers three questions: does it compile, does it preserve the properties you care about, and is it fast enough? This article shows how to build that gate with free model access for generating the patch and a free server for running the verification. You will get a script, a property-test skeleton, and a decision table you can adapt to your own codebase.

Why a cheap gate matters more now

When a human writes a patch, you can ask them about their intent. An agent patch is a hypothesis produced by an opaque system. You cannot interrogate its reasoning, but you can probe its output. The smallest useful probe is a compile. The next is a unit test. But unit tests encode examples, not invariants. A property test encodes a rule that should hold for every input, and that is where many agent-generated edits fail.

You might think you need expensive infrastructure to run such checks. In practice, a small C++ project with one source file fits easily inside a free server. The free model generates the change; the free server executes a script like the one below; the decision table tells you whether to invest human time. This pattern is not about replacing review. It is about spending review time only on changes that pass the cheap checks.

The verification layer, component by component

The layer has three components. First, the patch generator: MonkeyCode's free model access can produce or explain the change. Second, the runner: MonkeyCode's free server option can execute the verification script without you renting a host. Third, the property set: a small set of invariants you believe the code must always satisfy. The script ties them together. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Let's make it concrete with a sorting routine. Suppose the agent changed a function that merges two sorted vectors. The property you care about is: the result is sorted, and the multiset of elements is preserved. The following main.cpp checks that property on a thousand random inputs:

#include <algorithm>
#include <cassert>
#include <random>
#include <vector>

std::vector<int> merge_sorted(const std::vector<int>& a,
                              const std::vector<int>& b) {
  std::vector<int> out;
  out.reserve(a.size() + b.size());
  auto it_a = a.begin(), it_b = b.begin();
  while (it_a != a.end() && it_b != b.end()) {
    if (*it_a <= *it_b) out.push_back(*it_a++);
    else out.push_back(*it_b++);
  }
  out.insert(out.end(), it_a, a.end());
  out.insert(out.end(), it_b, b.end());
  return out;
}

int main() {
  std::mt19937 rng(12345);
  for (int trial = 0; trial < 1000; ++trial) {
    std::vector<int> a, b;
    for (int i = 0; i < 20 + rng() % 30; ++i) a.push_back(rng() % 1000);
    for (int i = 0; i < 20 + rng() % 30; ++i) b.push_back(rng() % 1000);
    std::sort(a.begin(), a.end());
    std::sort(b.begin(), b.end());
    auto result = merge_sorted(a, b);
    assert(std::is_sorted(result.begin(), result.end()));
    assert(result.size() == a.size() + b.size());
    std::multiset<int> left(a.begin(), a.end());
    std::multiset<int> right(b.begin(), b.end());
    for (int x : result) {
      auto pos = left.find(x);
      if (pos == left.end()) pos = right.find(x);
      else left.erase(pos);
      if (pos == left.end() && pos == right.end()) return 1; // mismatch
    }
  }
  return 0;
}

This is not a bulletproof proof. It is a probabilistic check that catches many but not all violations. For the purpose of a budget gate, that probability is worth it.

The runner script

The script below expects a patch file and a project directory. It applies the patch, compiles with warnings enabled, runs the executable, and measures how long the run took. The elapsed time is important because an agent optimization should not make the common case slower.

#!/usr/bin/env bash
set -euo pipefail

PATCH_FILE="${1:?usage: verify_patch.sh <patch>}"
PROJECT_DIR="${2:-/tmp/agent-project}"

# Validate and apply
git -C "$PROJECT_DIR" apply --check "$PATCH_FILE"
git -C "$PROJECT_DIR" apply "$PATCH_FILE"

# Compile with strict warnings
if ! g++ -std=c++20 -Wall -Wextra -Werror -O2 \
  "$PROJECT_DIR/main.cpp" -o "$PROJECT_DIR/main"; then
  echo "BUILD_FAIL"
  exit 1
fi

# Run property tests
if ! "$PROJECT_DIR/main"; then
  echo "PROPERTY_FAIL"
  exit 1
fi

# Bound the common-case time
START_NS=$(date +%s%N)
"$PROJECT_DIR/main" >/dev/null
END_NS=$(date +%s%N)
ELAPSED_MS=$(( (END_NS - START_NS) / 1000000 ))

echo "PASS MS=$ELAPSED_MS"

You can run this on a free server that has a shell and a C++ compiler. The same script works for any compiled language; adjust the build line and the executable path. The key idea is that the free server gives you a clean, disposable environment. If the patch introduces a virus-like side effect or deletes files, only the throwaway server suffers.

Decision table: what to do with each result

Result	Meaning	Action
`BUILD_FAIL`	The patch breaks the build	Reject. Ask the agent to fix the compile error and resubmit.
`PROPERTY_FAIL`	An invariant is violated	Reject. Attach the failing trial to the agent as feedback.
`PASS` with high `MS`	The change works but is slower than your threshold	Reject or mark for optimization. Do not let a regression slip through.
`PASS` with acceptable `MS`	The patch survives this layer	Human review only now. Look at design, security, and intent.

The table is a policy. You can adjust it: some teams accept a 10% slowdown for a simpler implementation. Others set a hard baseline. The important part is that you decide before you see the patch, so the gate stays objective.

Why include MonkeyCode here

MonkeyCode's free model access is what turns a suspicion into a testable patch. You can take a failing property case and ask the model, in natural language, to propose a fix. The free server option then runs the verification script in a clean environment, so the cost of a bad suggestion is a few seconds and zero dollars. That combination makes this workflow practical for a side project or a small team.

Free does not mean unlimited. The free model has a rate limit, and the free server is not designed for heavy parallel workloads. Keep your verification runs short and your test suite small enough to finish in minutes. If your build takes an hour, this gate will not fit the free tier. That is the first limitation.

Limitations and who should not use this

This verification layer catches compile errors, property violations, and gross performance regressions. It does not catch design flaws, security issues, or race conditions that your property set does not express. The random seed is fixed in the example, which means the same inputs run every time; change the seed periodically to explore different regions of the input space.

Do not use this approach for code that requires formal verification, regulatory compliance, or strict reproducibility guarantees. A free server is a shared or power-capped resource; its elapsed time can vary, so a high MS reading is a hint, not a definitive benchmark. Also, if you work with private source code, a free server may not meet your data handling requirements. Check the terms of your provider before sending a patch there.

The method also assumes you can express invariants. For a complex GUI or a distributed system, property tests are harder to write. In those cases, focus the gate on build and smoke tests, and treat the property layer as a long-term investment.

Try it on the next patch

The next time an agent suggests a change, do not merge it just because the diff looks tidy. Apply the patch, run the script, and let the free server tell you whether the invariant still holds. A human should still make the final call, but the human should not be the first line of defense against a plausible patch. Free resources are enough for that job, and your review time is better spent on decisions, not on re-checking what a machine can check for you.

If you have a small C++ project or any compiled project, copy the script, write a property test for one rule, and run it against your next agent patch. The experience will show you how often a convincing patch is actually... well, not.

DEV Community