A free model can read a C++ header and propose a usage example. It cannot tell you if that example compiles. A disposable server can. The useful product is not the example text; it is the set of examples that survive compilation and run without assertion failures.
This case study turns free model output into an API contract for a small C++20 library. No existing code is modified. The model only writes client-side examples, and the server decides which examples deserve to be published.
The case: a small C++20 library with subtle constraints
The project is decimal_clock, a 180-line header-only library that represents a point in time as a signed 64-bit count of milliseconds and a fractional part. It exposes five public headers with 22 callable functions. Some functions have constraints that are easy to miss from a signature alone. For example, add_ms(long long) throws std::invalid_argument for negative values, and from_iso8601(std::string_view) expects UTC with a trailing Z.
The library compiles cleanly with -Wall -Wextra -Wpedantic. The problem is not warnings. The problem is that the public headers do not explain those constraints, and the existing unit tests cover only 9 of the 22 functions. The goal was to generate one runnable usage example for every public function, with each example acting as both a smoke test and a living documentation fragment.
Tooling and disclosure
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The experiment used MonkeyCode's free model access to propose example programs from public header signatures and its free server option to compile and run each proposal away from the development machine.
The model did not touch the library. It received signature lists and returned candidate main functions. The server was the execution environment, separated from the local checkout to avoid assumptions about local include paths.
The example card format
Every function gets an example card in JSON. The card is deliberately narrow so the model cannot drift into explaining the library or rewriting its code.
{
"signature": "std::optional<std::chrono::milliseconds> parse_duration(std::string_view)",
"header": "decimal_clock/duration_parser.hpp",
"example": "#include <decimal_clock/duration_parser.hpp>\n#include <iostream>\n#include <optional>\n#include <chrono>\n\nint main() {\n auto d = decimal_clock::parse_duration(\"1500ms\");\n if (d) std::cout << d->count();\n return 0;\n}\n",
"assert": "stdout_contains(\"1500\")"
}
The model produces only the example field. The signature and header come from a local extractor, not from the model.
Implementation
Step 1: Extract public signatures without the model
The signatures were extracted with a small script that runs Clang's AST dump and keeps top-level function declarations from the public include directory. This is deliberately local because the model should not choose which functions deserve examples.
clang++ -std=c++20 -Iinclude -Xclang -ast-dump -fsyntax-only include/decimal_clock/*.hpp |
grep -E '^.*FunctionDecl.* decimal_clock::' |
sed -E 's/.* ([a-zA-Z_][a-zA-Z0-9_:<> ,*&]+)\(.*/\1/' | sort -u > signatures.txt
Step 2: Ask the model to return only a main function
The prompt included the signature list, the rule that every example must include the correct header, and the rule that each program must return 0 on success. It also requested one example per signature and no commentary outside a fenced code block.
Step 3: Compile and run on the free server
The server ran a small loop. Each example was written to a temporary file, compiled with -std=c++20 -Iinclude, and executed with a two-second timeout.
#!/usr/bin/env bash
set -euo pipefail
case_dir="$1"
for card in "$case_dir"/*.json; do
sig="$(jq -r '.signature' "$card")"
example_file="$(mktemp --suffix=.cpp)"
jq -r '.example' "$card" > "$example_file"
if g++ -std=c++20 -Iinclude "$example_file" -o "$example_file.out" 2>"$card.compile.log"; then
if timeout 2 "$example_file.out" > "$card.stdout" 2>"$card.runtime.log"; then
jq -r --arg s "$sig" --arg f "$card.stdout" \
'{signature: $s, status: "COMPILED_AND_RAN"}' "$card" > "$card.result"
else
jq -r --arg s "$sig" '{signature: $s, status: "RUNTIME_FAIL"}' "$card" > "$card.result"
fi
else
jq -r --arg s "$sig" '{signature: $s, status: "COMPILE_FAIL"}' "$card" > "$card.result"
fi
done
The assertion field was checked as a second pass against the captured stdout. Passing the assertion is the only way an example reaches the status ACCEPTED.
Results
This was a reference run for the described extractor and local header set. The exact numbers depend on the compiler and model version, but the failure funnel is the useful part.
| Status | Count |
|---|---|
| Signatures extracted | 22 |
| Examples proposed | 22 |
| Compiled | 14 |
| Ran successfully | 9 |
| Passed stdout assertion | 7 |
| Accepted | 7 |
The accepted examples were mostly simple conversions and arithmetic operations. The rejected pile exposed three recurring patterns. First, five examples included the wrong header or no header, relying on a transitive include that did not exist. Second, four examples treated std::optional<std::chrono::milliseconds> as though it were the contained value. Third, three examples called a throwing function with invalid input without a try block, so the program exited nonzero.
Those failures are not noise. They are evidence about where the API is hard to call and where the headers lack discoverable cues.
Why this differs from fixing warnings
A warning-fixing workflow asks whether a patch silences a diagnostic without breaking behavior. This workflow asks a different question: can an external developer write a program that uses the API after seeing only the signature and a generated example? The server is not checking the library; it is checking whether the generated example is a usable contract.
Limitations and who should not use this
The approach validates example quality, not library correctness. A passing example can still hide an incorrect result if the assertion is weak. The extraction step also assumes public headers are clean enough to parse.
Do not use this workflow for security-critical interfaces, for functions with side effects that are unsafe to run on shared infrastructure, or when the goal is to prove semantic correctness rather than discoverability.
What to keep
The reusable part is the split between proposal and execution. Have a free model propose client code, but let a disposable server compile and run every proposal before any example enters documentation or CI. If you cannot run the example, it is not an example; it is a guess.
Top comments (0)