Build the Patch as a Stranger: A Consumer-Link Gate for C++ Codegen

A packaging job can fail on a Thursday morning while the eval board stays green. In that pattern, the candidate patch repairs normalize() inside one translation unit, and the grader compiles that unit together with the tests. The public header never gains the declaration. Downstream, consumer.cpp includes only the shipped header, links libpatch.a, and stops on an undefined reference.

That miss is structural, not a fluke of one model. In-tree tests often see private headers, extra -I paths, and a link line the author of the harness invented. A stranger's binary does not. The consumer-link gate builds the patch the way a later client would, then classifies the failure before anyone treats a green board as a merge signal.

What the in-tree binary hides

A single test executable can pass for boring reasons. The test file may declare the missing function locally. The build may compile patch.cpp straight into the test binary, so an archive was never produced. A macro set in the test harness may change the header, while a clean client never defines that macro.

Behavior checks still matter. They do not answer whether the patch is installable. Link errors, one-definition-rule collisions, and include-path accidents are silent regressions relative to a grader that never leaves its own tree.

The example below is synthetic. It exists so a team can reproduce the failure class on a laptop before any model is scored. No timing numbers, pass rates, or customer incidents are claimed for it.

Four outcomes worth naming

The gate records one class per golden case. Names stay stable so a later diff of JSON is readable.

1. header_missing means the consumer could not include the public header with the published -I path.
2. link_undefined means the header compiled and the archive did not provide a required symbol.
3. odr_or_duplicate means two translation units disagreed on an inline function, a macro, or a weak symbol.
4. behavior_mismatch means link and start succeeded, and the process exit or stdout disagreed with the golden file.

Timeouts stay in a fifth bucket, hang, so a stalled binary is not scored as a wrong answer. Optimization matrices and assertion censuses belong in other gates. This one only asks whether a stranger can compile, link, and run.

Step 1 — Split the library from the client

Keep the patch sources, the public header, and the consumer in separate directories. The consumer may include patch.hpp and nothing under src/. Private helpers stay out of the client file even when that makes the case look blunt.

CXX ?= c++
CXXFLAGS ?= -std=c++20 -Wall -Wextra -Werror -Iinclude
.PHONY: clean gate

build/libpatch.a: src/patch.cpp include/patch.hpp
    mkdir -p build
    $(CXX) $(CXXFLAGS) -c src/patch.cpp -o build/patch.o
    ar rcs $@ build/patch.o

build/consumer: consumer/consumer.cpp include/patch.hpp build/libpatch.a
    $(CXX) $(CXXFLAGS) consumer/consumer.cpp build/libpatch.a -o $@

gate: build/consumer
    ./build/consumer > build/consumer.out

clean:
    rm -rf build

The archive step is the point. If the model left the definition in a file the test binary used to compile directly, ar never sees that symbol, and the consumer link fails on purpose. A green make gate after that split is a stronger claim than a green in-tree unit test.

A minimal client looks like this. It is a golden case, not a demo to extend with private helpers.

#include "patch.hpp"
#include <iostream>

int main() {
  auto out = demo::normalize("  Ok \n");
  if (out != "ok") return 2;
  std::cout << out << "\n";
  return 0;
}

The matching header must stand alone. A forward declaration stuffed into consumer.cpp would hide the bug the gate exists to catch, so the grader rejects consumer files that declare demo:: symbols themselves. That check is a text scan, cheap and strict, and it runs before make.

Step 2 — Pin the golden contract

Each case is a small JSON file. Paths are relative to the case root. Expected classes describe the seed bug, not every future patch. A correct patch should land on pass.

{
  "id": "normalize-trim",
  "public_header": "include/patch.hpp",
  "archive_sources": ["src/patch.cpp"],
  "consumer": "consumer/consumer.cpp",
  "expect_stdout": "ok\n",
  "expect_exit": 0,
  "forbid_consumer_decls": ["demo::"],
  "compiler_note": "pin c++ --version in the run manifest"
}

Seed a known-bad patch beside it, with the declaration omitted from the header, and expect link_undefined. That seed proves the grader can fail. A harness that has never seen red will happily mark every later accident as pass.

Store the seed under a name that cannot be confused with the fix, such as goldens/normalize-trim.missing-decl.json. Reviewers should be able to run the red case in one command and read the class without opening a dashboard.

Step 3 — Grade the log, not the vibe

The following grader is a proposed workflow, not a report of a measured run. It shells out to make, then maps compiler text to the classes above. Diagnostic strings vary by compiler, so the patterns are explicit and the compiler version is written into the same JSON as the grade.

import json, re, subprocess, sys
from pathlib import Path

PATTERNS = [
    ("header_missing", re.compile(r"fatal error: .*No such file or directory")),
    ("link_undefined", re.compile(r"undefined reference to")),
    ("odr_or_duplicate", re.compile(r"multiple definition of")),
]

def classify(log: str, exit_code: int, stdout: str, golden: dict) -> str:
    for name, pat in PATTERNS:
        if pat.search(log):
            return name
    if exit_code != 0 and "timed out" in log:
        return "hang"
    if stdout != golden["expect_stdout"] or exit_code != golden["expect_exit"]:
        return "behavior_mismatch"
    return "pass"

def main() -> int:
    golden = json.loads(Path(sys.argv[1]).read_text())
    subprocess.run(["make", "clean"], check=False, text=True, capture_output=True)
    proc = subprocess.run(["make", "gate"], check=False, text=True, capture_output=True)
    log = proc.stdout + proc.stderr
    out_path = Path("build/consumer.out")
    stdout = out_path.read_text() if out_path.exists() else ""
    kind = classify(log, proc.returncode, stdout, golden)
    report = {"id": golden["id"], "class": kind, "exit": proc.returncode}
    Path("build").mkdir(exist_ok=True)
    Path("build/grade.json").write_text(json.dumps(report, indent=2))
    print(json.dumps(report))
    return 0 if kind == "pass" else 1

if __name__ == "__main__":
    raise SystemExit(main())

Run it against one case, then against the known-bad seed.

python3 grade_consumer.py goldens/normalize-trim.json
python3 grade_consumer.py goldens/normalize-trim.missing-decl.json
nm -g build/libpatch.a

The second command should print "class": "link_undefined". If it prints pass, the consumer is still compiling the patch source by accident, and the Makefile is wrong. Fix that before scoring any model output. nm is the tie-breaker when the log is noisy: a missing global symbol confirms the archive, not the regex.

Step 4 — Keep the worker poorer than the grader

Candidate patches should be generated and built in a directory that contains include/, src/, consumer/, and the Makefile, and nothing else. Ambient headers from a developer's home directory are a common way for a broken patch to look healthy. Copy the case tree into an empty folder and export a minimal PATH before calling make.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes MonkeyCode as an open-source project and reports free model access plus a free server option. No primary source dated 2026-10-08 was attached for a permanent token ceiling, hardware shape, model list, or duration, so this workflow does not print a quota as a fact. The run manifest stores only what the account response and the build worker showed on that run: a timestamp, the model id string if the API returns one, the compiler version, and the grade JSON.

Use the free server option as the poor worker when it is actually available. Copy the case tree up, run make gate, and pull build/grade.json back. A local laptop remains the reference. Disagreement between the two grade files is a harness bug or an environment bug, not a reason to average the scores.

A practical next step is to diff those two JSON files after one local run and one free-server run, and to treat a mismatch as a failed eval rather than as a successful demo.

How a failing class is reviewed

A link_undefined result is not yet a verdict on the model's reasoning. The reviewer opens the public header and the archive member list. nm -g build/libpatch.a shows whether the symbol was compiled at all. If the symbol exists but is local, the model likely omitted the declaration or hid the definition in an anonymous namespace. If the symbol is absent, the source file was not part of archive_sources, or the definition was wrapped in a macro the consumer never sets.

A header_missing result sends the reviewer to the include path only. Adding -I until the board turns green is the wrong fix. The consumer command in the Makefile is the contract. Change the patch to place the header where that command already looks.

A behavior_mismatch after a clean link goes back to the golden string. Update the golden file only when the contract truly changed, and record the reason in the case notes. Silent edits to expected stdout are how a harness learns to approve regressions.

Decision table

Log fragment	Class	Do not do
No such file or directory on the public header	header_missing	Add the grader's private -I to the consumer
undefined reference to	link_undefined	Compile src/*.cpp into the consumer
multiple definition of	odr_or_duplicate	Delete the second TU to make the board green
Exit 2 or wrong stdout	behavior_mismatch	Rewrite the golden string to match the patch
No exit before the timeout	hang	Score it as a behavioral near-miss

Limitations

This gate does not prove algorithmic correctness beyond the golden stdout and exit code. It does not replace sanitizer builds, cross-optimization checks, or a review of whether the model deleted assertions. Floating-point cases need an explicit tolerance field this sample omits. Compiler diagnostics in languages other than English, or from a compiler that phrases undefined references differently, will fall through to behavior_mismatch unless the patterns are extended.

Header-only libraries gain little from an archive step. Skip the gate there, or replace the archive with a second translation unit that includes only the public header. Teams that already install the package and compile an external smoke client in CI will find this redundant. Patches that are mid-refactor, with an intentionally broken public header, should not be scored as regressions against a stable golden case.

The Python snippet is a sketch. It does not isolate the build in a container, does not bound CPU, and trusts make output. Production use needs a timeout wrapper, a clean PATH, and a pinned compiler recorded in the same JSON as the grade. Anyone who needs a measured quality claim for a named model should run the seed case first and publish the manifest, not a slogan.

Closing

A green in-tree test means the grader's translation unit was satisfied. It does not mean a later client can include the header and link the archive. Split those builds, name the failure, and keep the worker from seeing headers the client will never have. The score that remains is smaller, and harder to fake.