Case Study: A Free Model Triage of 1,400 C++ Warnings — the First 12 Were Wrong

Compiler warnings are cheap to produce and expensive to read. Last week, one legacy translation unit emitted 1,400 of them in a single build. I built a small C++ tool to group the noise, sent the groups to a free model endpoint through MonkeyCode's free server option, and asked for a triage. The first 12 answers were wrong. The fix was not a better model. It was a wider context window.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The warning flood

The project is a 15-year-old C++17 codebase. One file, session_manager.cpp, pulls in a chain of templates that makes every compiler version angry. The build itself succeeds. The output does not.

1,400 warnings is not a reading problem. It is a sorting problem. Most of them repeat the same three or four root causes, but they are scattered across 60 files and 400 lines of terminal scrollback. A human triage of that volume takes half a day, and the result is usually a list of ten files to look at.

The goal was narrower: produce a one-page triage. Which warning groups are real, which are template noise, and which need a fix this sprint. The constraint was tighter: no source files leave the machine. Only warning lines and a three-line context slice would go to the model.

Step 1: normalize and group

Compiler output is not a data format. Paths are absolute, flags are repeated, and the same -Wunused-variable appears 300 times with different line numbers.

The first artifact is a 60-line C++ filter, warnfold, that reads make output on stdin and buckets warnings by flag:

// warnfold.cpp — group compiler warning lines into stable buckets
#include <iostream>
#include <regex>
#include <string>
#include <unordered_map>
#include <vector>

struct Warning {
  std::string file;
  int line = 0;
  std::string text;
  std::string flag;
};

static const std::regex kLine(
    R"(^(.*?):(\d+):\d+: warning: (.*?) \[(-W[^\]]+)\])");

int main() {
  std::unordered_map<std::string, std::vector<Warning>> groups;
  std::string line;
  while (std::getline(std::cin, line)) {
    std::smatch m;
    if (!std::regex_match(line, m, kLine)) continue;
    groups[m[4]].push_back(Warning{m[1], std::stoi(m[2]), m[3], m[4]});
  }
  for (const auto& [flag, warns] : groups) {
    std::cout << flag << "\t" << warns.size() << "\n";
  }
}

Compile and run:

make 2>&1 | ./warnfold | sort -k2 -nr | head -20

Multiline note: follow-ups are ignored. They explain a warning; they do not define a bucket. The result: 1,400 raw lines became 214 unique (file, line, flag) tuples, then 31 groups. The top five groups covered 73% of the volume. That number is the whole point of the exercise: the model never needs to see the other 27%.

Step 2: build a bounded batch

For each group, I took the first three distinct warning messages and a three-line source context around the first occurrence. The context is the part I initially skipped. That was the mistake.

The payload per group looks like this:

{
  "group": "-Wmaybe-uninitialized",
  "count": 412,
  "samples": [
    {"file": "src/session.cpp", "line": 881, "text": "variable 'it' may be used uninitialized in this function"},
    {"file": "src/session.cpp", "line": 1192, "text": "variable 'it' may be used uninitialized in this function"}
  ],
  "context": "for (auto it = cache.begin(); it != cache.end(); ++it) {\n  if (!it->second.valid) { continue; }\n  results.push_back(*it);\n}"
}

The whole batch was about 2,400 tokens. The raw warning dump would have been roughly 14,000. Grouping is the real optimization; the model is the last mile.

Step 3: the triage prompt

The prompt was deliberately small:

You are triaging C++ compiler warnings. For each group, return:
- risk: high | medium | low
- one-sentence root cause
- one suggested next step

Group: {flag} ({count} occurrences)
Samples: {samples}
Source context: {context}

I called the endpoint through MonkeyCode's free server option from a small script. No model name, no quota math, no benchmark claims: the point is that a free-tier path was enough for a 2,400-token batch.

The first 12 were wrong

The first run had no context field. The model produced 12 summaries, and all 12 were wrong in the same way: it reasoned about the warning text alone and invented the code around it.

Example: -Wmaybe-uninitialized at session.cpp:881. The model said "initialize it before the loop." The variable was a std::optional<iterator> that was intentionally empty until a later branch filled it. The warning was a false positive from an inlined lambda. The model could not see the lambda, so it invented a simpler program than the one that existed.

Worse, the invented fixes were confident. One summary claimed the fix was "add = nullptr" to a variable that was not a pointer. That is the failure mode that matters: not wrong answers, but wrong answers with the structure of right ones.

What changed

I added the three-line context and reran the same 12 groups. Then I held out 40 warnings from the original 214 and compared the model's triage to mine.

• 31 of 40 matched my risk level and suggested step (78%).
• 5 mismatches were risk disagreements: the model called template noise "high risk" because the warning text sounded severe.
• 4 were hallucinated line references: the model cited lines that did not exist in the context slice.

The second run was useful. The first run was worse than nothing, because a confident wrong triage sends you to the wrong file.

Lessons

1. A model summarizes text, not code. If it cannot see the code, it will invent it. Context width is a correctness parameter, not a nice-to-have.
2. Group first, ask later. The 31-group bucket is the artifact worth keeping. The model's answers are disposable.
3. Keep a verification gate. I checked 10% of the triage against my own read. That caught the hallucinated line numbers before they reached anyone else.
4. Free endpoints fail silently. This project did not hit an outage, but the lesson from earlier CI work applies: treat the model call as a best-effort step with a timeout and a fallback, not as a guarantee.

Who should skip this

If your build prints fewer than 100 warnings, read them. The grouping tool pays for itself only when the volume exceeds what one person can scan in a sitting.

If your warning lines or source context can contain secrets — embedded tokens, credentials in string literals — do not send them to any remote endpoint. The three-line context rule does not protect you from a secret on line two.

If you need a fix, not a triage, this workflow under-delivers. The model suggests a next step; it does not patch a 15-year-old template chain.

And if your pipeline cannot tolerate a slow or unavailable free server, put a queue in front of the call. The free server option is a convenience, not a contract.

The artifact that mattered

The model was the part I remembered. The warnfold grouping step was the part that saved the time. If your next build prints more warnings than you can read in one sitting, write the grouping tool first. The model can wait.