Case Study: A Small C++ Build-Log Triage Tool and Four Merge Gates for Free Model Patches

A free model endpoint is useful for a C++ refactor only when the patch can prove it did not regress the tool. In this case study, nine suggested changes produced two merges. The filter was not model quality. It was a four-gate pipeline that made every patch build, test, sanitize, and add a regression test before it touched main.

Background

logtriage is a small C++ tool that reads Ninja or Make output, extracts the first compiler error per source file, and prints a sorted summary. It started as a 487-line CLI with no tests and no sanitizer target. The team used it during long builds because the first error is often the only error that matters. The problem was that every model-suggested improvement looked reasonable until it changed CLI behavior or broke a path with no coverage.

The starting parser looked like this:

#include <fstream>
#include <iostream>
#include <string>
#include <vector>

std::string firstError(const std::string& line) {
  auto pos = line.find("error:");
  if (pos == std::string::npos) return "";
  return line.substr(pos);
}

int main(int argc, char** argv) {
  std::ifstream in(argv[1]);
  std::string line;
  std::vector<std::string> errors;
  while (std::getline(in, line)) {
    auto e = firstError(line);
    if (!e.empty()) errors.push_back(e);
  }
  for (const auto& e : errors) std::cout << e << '\n';
}

The first obvious bug: if a line contains both warning: and error:, the function returns the whole tail starting at error:. That is fine for a first attempt, but it was not a tested contract.

Goal

The goal was not to replace human review. The goal was to make free-model suggestions cheap to reject and easy to audit. I wanted a merge rule that would be exactly the same on my laptop and on the free server option.

For this case study I used MonkeyCode's free model access to generate candidate patches and hosted the gate loop on its free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Implementation

Step 1: Capture a failing fixture

I added fixtures/mixed_line.log with this line:

src/parser.cpp:42: warning: unused variable x | error: expected ';' before '}'

The naive firstError output was error: expected ';' before '}', but the tool's intended contract was to return only the source location and message, not duplicate the warning prefix. I made that contract a test.

Step 2: Generate candidate patches

Each prompt asked for a one-file patch and a new test named regression_<n>.cpp. I did not ask the model to explain why the patch was correct. The patch and the test had to earn that explanation through the gates.

Step 3: Run the four gates

The gate script lived in the repository:

#!/usr/bin/env bash
set -euo pipefail

PATCH_FILE=$1
git apply --check $PATCH_FILE || { echo 'gate1: patch failed'; exit 10; }
git apply $PATCH_FILE

cmake -S . -B build -DCMAKE_CXX_FLAGS='-Wall -Wextra -Werror'
cmake --build build -j2 || { echo 'gate2: build failed'; exit 20; }

ctest --test-dir build --output-on-failure || { echo 'gate3: tests failed'; exit 30; }

cmake -S . -B build-asan -DCMAKE_CXX_FLAGS='-fsanitize=address,undefined -fno-omit-frame-pointer'
cmake --build build-asan -j2
ASAN_OPTIONS=detect_leaks=1 ctest --test-dir build-asan --output-on-failure || { echo 'gate4: sanitizer failed'; exit 40; }

echo 'all gates passed'

Gate 3 required the patch to add a test. I enforced that with a separate check: git diff --name-only HEAD~1 | grep -q 'regression_'. If the patch touched only production code, it failed before the build.

Step 4: Record the results

Patch	Claimed effect	Failure	Outcome
1	Split firstError into two helpers	Unused parameter under -Werror	Rejected at gate 2
2	Replace std::string::find with regex	Existing test broke on empty line	Rejected at gate 3
3	Return source location only	No new regression test	Rejected at gate 3 precheck
4	Strip warning: before error:	None	Merged
5	Add --summary flag	Changed CLI without permission	Rejected at gate 2
...	...	...	...

The run produced nine candidate patches. Four failed the build gate, two failed the existing test gate, one failed the missing-test precheck, and two passed all gates. One accepted patch fixed the mixed-line bug. The other added deterministic sorting for files with the same error count.

Limits

This is a small case study, not a model benchmark. Nine patches cannot tell you whether any model is good. It tells you what a gate can reject.

The gates are not neutral. -Werror rejects valid refactors that introduce harmless warnings. The sanitizer gate catches memory bugs but not timing bugs. The free server option may queue jobs, so this loop is not realistic for tight CI feedback.

Do not use this pattern when every diff needs a human compliance review. Do not use it when a full sanitizer run is too slow for your build. Do not use it when the repo has no baseline tests, because the gate then becomes only a build check.

Lessons

A gate script stored next to CMakeLists.txt was more useful than the model's explanation. The two patches that merged did not look smarter than the rejected seven. They were just easier to verify.

The missing-test precheck mattered more than the sanitizer. It converted the model from a patch generator into a hypothesis writer. A patch that cannot state its regression test is not ready for main.

Keep the gate script in the repository. The script, not the model output, is the actual merge review.