Case Study: A C++ Build-Failure Triage Bot — Cache First, Recompile as the Gate

A compiler log is evidence, not a diagnosis. Most CI failures are boring: a missing include, a template deduction that works on GCC but not on Clang. Boring still costs a human a log-open, a scroll, and a category lookup. This case study covers a small triage bot that automates that step — it extracts the first error, classifies it, proposes a fix, and refuses to trust the proposal until a real recompile confirms it. The whole pipeline runs on free infrastructure: a free model endpoint for classification and a free server for the webhook.

Background

The project that motivated this was a single-header C++ library compiled in CI against GCC, Clang, and MSVC. The failures were repetitive. The triage was not.

I wanted a bot that takes a raw compiler log and returns one JSON object: category, confidence, candidate fix, and a gate verdict. The gate verdict is the part that matters. A model can propose anything; the compiler decides.

Goal

The bot had five requirements:

1. Extract the first error from a GCC/Clang log.
2. Classify it into one of six categories: missing_include, template_deduction, type_mismatch, linker_undefined, abi_mismatch, other.
3. Produce a candidate fix as a patch.
4. Verify the fix by recompiling a copy of the file with the same flags.
5. Cache only verified results, keyed by a normalized error signature.

Requirement 5 is the one that makes free infrastructure viable. Model calls are the scarce resource. The cache is the difference between one call per failure and one call per unique failure.

Implementation

Step 1: Normalize the error

The first error in a GCC or Clang log has a stable shape:

src/parser.cpp:42:9: error: no matching function for call to 'max(int, double)'

A small C++17 extractor handles the rest:

struct ErrorSig {
  std::string file;
  int line = 0;
  int col = 0;
  std::string message;
};

ErrorSig first_error(const std::string& log) {
  static const std::regex re(
      R"(^([^:]+):(\d+):(\d+):\s+error:\s+(.*)$)",
      std::regex::multiline);
  std::smatch m;
  if (std::regex_search(log, m, re)) {
    return {m[1].str(), std::stoi(m[2]), std::stoi(m[3]), m[4].str()};
  }
  return {};
}

std::string cache_key(const ErrorSig& e) {
  // Paths and line numbers change between runs; the message usually does not.
  return e.message;
}

The cache key deliberately drops the file path and line number. A missing include in parser.cpp and a missing include in lexer.cpp are the same disease.

Step 2: Cache by signature

The bot checks the cache before it touches the network:

if (auto hit = verified_cache.find(key); hit != verified_cache.end()) {
  emit_report(hit->second, /*cache_hit=*/true);
  return 0;
}

Only a cache miss reaches the classifier. This is not an optimization; it is the design decision that makes a free-tier call budget survivable when a broken build re-runs twenty times in a morning.

Step 3: The classifier webhook

The model key must not live in the C++ client. The webhook runs on a free server, holds the key, and exposes one endpoint. In this setup, the free model access and the free server both come from MonkeyCode.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The server is deliberately thin:

# server.py — runs on the free server, holds the model key
from flask import Flask, request, jsonify
import os, urllib.request

app = Flask(__name__)
MODEL_ENDPOINT = os.environ["MODEL_ENDPOINT"]   # from your project settings
MODEL_KEY      = os.environ["MODEL_KEY"]

@app.post("/classify")
def classify():
    payload = request.get_json()
    body = jsonify({
        "error": payload["error"],
        "categories": ["missing_include", "template_deduction",
                       "type_mismatch", "linker_undefined",
                       "abi_mismatch", "other"],
    }).get_data()
    req = urllib.request.Request(MODEL_ENDPOINT, data=body, method="POST")
    req.add_header("Authorization", f"Bearer {MODEL_KEY}")
    with urllib.request.urlopen(req) as resp:
        return resp.read()

The contract is one JSON object in, one JSON object out:

{
  "error": "no matching function for call to 'max(int, double)'",
  "category": "template_deduction",
  "confidence": 0.6,
  "fix": "static_cast<double>(1)"
}

The category list is sent with the prompt so the model is constrained to the taxonomy. An unconstrained model will invent a new category per failure.

Step 4: The recompile gate

This is the step that separates a hypothesis from a result. The bot applies the proposed fix to a copy of the file and recompiles with the same flags:

bool gate(const std::string& source, const std::string& patch,
          const std::string& flags, const ErrorSig& before) {
  apply_patch("triage_copy.cpp", source, patch); // writes patched source
  int rc = std::system(("g++ " + flags +
                        " triage_copy.cpp 2> gate.log").c_str());
  if (rc == 0) return true;                       // promoted
  ErrorSig after = first_error(read_file("gate.log"));
  return after.message != before.message;         // moved the failure: partial
}

The exit code alone is not enough. A fix that deletes the offending line produces a nonzero exit with a different error. The gate records that as a partial result, not a pass. A fix that produces the identical error signature is rejected outright.

The gate is not a circuit breaker. The model is allowed to be wrong; the gate is what absorbs the wrongness.

Step 5: The report

Every failure produces one line of JSON:

{"file":"src/parser.cpp","category":"template_deduction",
 "gate":"promoted","cache":"miss","elapsed_ms":2140}

A promoted result enters the verified cache. A rejected or partial result enters a separate rejection store with a TTL. Verified and rejected hypotheses never share a namespace.

Test plan

The harness is only useful if it can be re-run. Ten snippets, one injected error each, covering the six categories:

for i in tests/snippet_*.cpp; do
  g++ -std=c++17 -fsyntax-only "$i" 2> "logs/$(basename "$i").log"
  ./triage_bot < "logs/$(basename "$i").log"
done

Record the output in a table like this one:

#	injected category	model category	gate verdict	notes
1	missing_include
2	template_deduction
3	type_mismatch
4	linker_undefined
5	abi_mismatch
6	other

I am deliberately not publishing accuracy numbers from a single run. Ten samples is noise, and your compiler version and prompt wording will shift the results. The point of the harness is that you can produce your own numbers in under an hour.

What the harness surfaced

Three failure modes shaped the final design, and each one left a trace in the code.

Full-log drift. Sending the entire log made the model fix the last error instead of the first. Truncating to the first error plus five lines of context stabilized the behavior.

The delete-and-hope fix. The model occasionally "fixed" a missing include by deleting the line that used the missing type. The gate caught it, because the error signature stayed identical.

Cache poisoning. Caching unverified hypotheses made the bot confidently repeat wrong answers. The verified cache and the rejection store exist precisely to prevent that.

Limitations

The classifier is a hypothesis generator, not a diagnosis engine. The compiler is the only oracle, and the gate is the only thing that makes the bot honest.

Free-tier infrastructure is best-effort. If your team needs a latency SLA, this pattern is not for you. The harness also assumes a reproducible build; a flaky or network-dependent build makes the gate meaningless.

MSVC logs have a different shape. The regex handles GCC and Clang; a second pattern is needed for cl.exe.

Who should not use this

• Teams that already have build-failure dashboards and a maintained error taxonomy.
• Projects where the build is not reproducible.
• Anyone who needs guaranteed response times.

Lessons

The model proposes; the compiler disposes. Everything else — the cache, the taxonomy, the JSON contract — is in service of that one sentence.

Cache design mattered more than prompt design. When the scarce resource is a free-tier call, the first question is not "how do I prompt better" but "how do I call less."

A free server is enough for a low-traffic webhook. The bottleneck was the model call, never the server.

The harness above is a complete starting point. If you build one, keep the gate — it is the only part that cannot be fooled.