GCC Passed, Clang Didn't: A Four-Configuration Compile Matrix for Agent Patches

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The patch compiled on your laptop, so you handed it to CI, and CI returned a wall of template errors from Clang. The agent had only ever compiled with GCC because that was the toolchain its container exposed. A single compile is not a quality signal; it is a sample from one configuration. For agent-generated C++, one sample is almost always the wrong one.

This article builds a compile matrix gate: a small, cost-budgeted set of builds that run before any unit test. The matrix does not guarantee correctness. It does catch a class of regression that unit tests usually miss because they never compile against a different standard library or a strict warning level.

Why a single compile is a weak gate

A unit test checks behavior for the inputs you wrote. A compile check checks your code against one dialect of the language. Agents optimize against the feedback they receive. If the feedback loop only feeds GCC 12's default flags, the agent learns to satisfy GCC 12's grammar, and it has no incentive to respect C++17 or avoid a deprecation that Clang 17 reports.

Two failures show up in practice. The first is silent dialect drift: the patch uses a C++20 feature even though the project targets C++17, and the local build runs with C++20 flags, so everything looks healthy. The second is warning blindness: the patch compiles, but the project treats warnings as errors, and the agent never saw that because -Werror was missing in its environment. Both problems are invisible to a single build configuration, and both are cheap to detect with a matrix.

Designing the four configurations

The matrix deliberately stays small. Four builds capture the largest diversity for the smallest cost, and you can extend it later when your codebase needs a specific compiler version. Each configuration is chosen to answer one question.

Config	Compiler	Standard	Flags	Catches
A	GCC	C++17	-O0	Baseline dialect compatibility
B	Clang	C++20	-O2 -Wall -Wextra -Werror	Strict modern warnings
C	GCC	C++20	-O2 -Wall -Werror	GCC-specific strictness
D	Clang	C++17	-O0 -Werror	Opposite dialect mismatch

The point is not to test every compiler you ship to. The point is to give the agent two independent compilers and two adjacent language standards, so a portability mistake cannot hide inside the single toolchain the agent happens to know. Configuration B and D are intentionally annoying: they promote warnings to errors, which forces the agent to clean the patch instead of leaving a subtle smell.

Budgeting the matrix

Each configuration costs roughly the same as compiling one unit-test binary. Four compiles are still cheaper than a flaky test suite, but if you let the agent run the matrix after every prompt, the cost adds up. So you cap it. The script stops at the first failure, and it also stops if the total number of compiles exceeds a small integer you set.

The cap is your budget. Treat it like a token budget. A patch that fails in the first configuration returns immediately, costs one compile, and the agent can retry with the error text. A patch that passes all four costs four compiles before a single test starts. That asymmetry is what makes the gate useful for iteration loops.

The script

The script is intentionally minimal. It compiles a translation unit, logs each command, and stops on the first failure. Set MAX_COMPILES to your budget, usually six to allow one retry after a failed attempt.

#!/usr/bin/env bash
set -euo pipefail

MAX_COMPILES=${MAX_COMPILES:-6}
COUNT=0
FAILED_CONFIG=""

compile() {
    local name="$1"; shift
    if (( COUNT >= MAX_COMPILES )); then
        echo "Budget exhausted after $COUNT compiles"
        exit 2
    fi
    echo ">>> $name"
    if ! "$@" > "/tmp/compile_${name}.log" 2>&1; then
        echo "FAILED in $name"
        FAILED_CONFIG="$name"
        exit 1
    fi
    (( COUNT++ ))
}

compile gcc17   g++ -std=c++17 -O0              -c sample.cpp
compile clang20 clang++ -std=c++20 -O2 -Wall -Wextra -Werror -c sample.cpp
compile gcc20   g++ -std=c++20 -O2 -Wall -Werror -c sample.cpp
compile clang17 clang++ -std=c++17 -O0 -Werror -c sample.cpp

echo "Compile matrix passed in $COUNT configurations"

You can replace sample.cpp with the translation units the agent touched. On a large codebase, you do not need to build the whole project for this gate; the agent's diff will tell you which files changed, and those are the files the matrix should compile.

Decision table after a failure

When the matrix fails, the error text is only half the story. You still need to decide whether the patch is wrong or the configuration is too strict. This table gives a starting point.

Result	Likely cause	Agent action
Fails only in Clang, passes in GCC	Non-portable construct or implementation-specific assumption	Rewrite using standard C++
Fails only in C++17, passes in C++20	Dialect drift; the patch silently used a newer feature	Replace the new feature with the C++17 equivalent
Fails only when -Werror is on	Warning-level issue such as an unused variable or sign conversion	Clean the warning; do not suppress it
Fails in both compilers with the same error	The patch broke a shared header or changed a signature	Inspect the patch diff and adjust the interface

The table is not a replacement for human judgment. A strict warning may be acceptable in one module and unacceptable in another. The matrix tells you where the disagreement is; you still have to decide what it means.

Running it with free compute

Because the matrix is cheap and disposable, it fits naturally into a free-tier agent loop. With MonkeyCode's free model access and free server option, you can generate a patch with the model and execute this script on the server without touching your paid CI runners. The server gives you a clean environment for each iteration, so a failed compile cannot leave cache debris behind for the next attempt.

The separation matters more than the price. Your main CI should validate behavior, not explore compiler dialects. The matrix is a filter that runs before the expensive parts, and it works better when it lives in a separate, low-trust environment that you can throw away after every patch.

Limitations

The matrix cannot replace tests. Undefined behavior is not a compile error in most configurations, so a patch can pass all four builds and still corrupt memory at runtime. The matrix also cannot detect logic errors: if the function returns the wrong answer, every compiler will happily compile it.

The matrix is compiler-version dependent. A construct accepted by GCC 12 may be rejected by GCC 13 because the standard clarified itself, and that is useful information even when it is not a regression. You need a human to classify the failure before telling the agent to fix a valid pattern, otherwise you are teaching it to chase shadows.

Finally, the four configurations are a heuristic, not a proof. They cover two compilers and two standards, but they will not catch a Windows-specific header or a miscompiled platform intrinsic. If your project targets exotic toolchains, the matrix needs to grow before you trust it.

Who should skip this

You can skip this gate if you control a single compiler and a single standard version forever, or if your patch only touches Python, JavaScript, or another language without a meaningful compile step. You should also skip it if your codebase takes an hour to build a single translation unit, because four full builds might take half a day.

In that case, isolate the files the agent changed and compile those alone. The diff tells you exactly what to feed to the script, and a small translation unit keeps the whole gate under a minute. If you cannot isolate the touched files, the matrix becomes a nightly audit instead of an every-patch gate.

The takeaway

A compile matrix is the cheapest gate that asks whether a patch will build anywhere else. It is not profound, but it catches a different category of agent regression than a test run, and it fails fast enough to keep the feedback loop tight. Start with four configurations and a budget of six compiles, then measure what the gate catches for your codebase.

If your matrix has a configuration that surfaced a real regression that tests missed, leave it in the comments. The most useful configurations are usually the ones that feel unnecessary until they fail.