A free model wrote a fixed-capacity C++ object pool in a single pass. It took three rounds and one human-designed invariant before the pool passed a sanitizer matrix and a 1,000,000-operation property test. The pattern repeated cleanly: the model fixed every mechanical defect quickly, and missed the same logical defect until the data layout changed.
This is that small project, end to end — background, goal, implementation, results, and lessons.
Background
An internal event-loop tool needed a fixed-size object pool. The requirements fit in a short prompt:
- No heap allocation after construction.
- O(1) acquire and release.
- 16-byte alignment for the stored type.
- Capacity 1024, fixed at compile time.
- A single-threaded version first, plus a mutex-guarded variant for a second consumer.
The total code was roughly 150 lines. The risk was low but not zero: a pool that hands the same slot to two callers corrupts the event loop silently, with no crash at the point of failure.
The point of the exercise was not the pool. It was the loop: generate with a free model, verify with cheap automated gates, feed failures back, repeat.
Goal
"Done" was defined before the first prompt. For this project, done meant:
- Zero defects under AddressSanitizer, UndefinedBehaviorSanitizer, and ThreadSanitizer.
- A property test running 1,000,000 random acquire/release operations without violating an invariant.
- A microbenchmark comparing the pool against malloc/free on the same machine.
The harness came first. The model came second. That order was the whole strategy.
Implementation
Step 1: Write the oracle
The property test kept a std::array<bool, 1024> mirror of the pool's state. Every acquire and release updated both the pool and the mirror; any mismatch aborted the run. This mirror is the part the model never saw — it lived entirely in the test.
// oracle.cpp — property test for the pool under test
#include <array>
#include <cassert>
#include <cstdint>
#include <random>
#include <vector>
template <typename Pool>
void run_oracle(Pool& pool, std::uint64_t seed) {
std::array<bool, 1024> live{};
std::vector<void*> outstanding;
std::mt19937 rng(seed);
for (int i = 0; i < 1'000'000; ++i) {
if (outstanding.empty() || (rng() % 3 != 0)) {
void* p = pool.acquire();
if (p == nullptr) {
assert(outstanding.size() == 1024); // only a full pool returns null
} else {
std::size_t idx = pool.index_of(p); // debug-only helper
assert(!live[idx]); // double acquire => fail
live[idx] = true;
outstanding.push_back(p);
}
} else {
std::size_t k = rng() % outstanding.size();
void* p = outstanding[k];
outstanding[k] = outstanding.back();
outstanding.pop_back();
std::size_t idx = pool.index_of(p);
assert(live[idx]); // release of a dead slot => fail
live[idx] = false;
pool.release(p);
}
}
}
Three invariants, all checkable in constant time. The pool's index_of helper existed only under a debug macro, so the production build carried no test code.
Step 2: Generate with the free model
The prompt listed the five requirements and one explicit warning: "Do not allocate memory in acquire or release." The stored type was a 32-byte struct. Generation ran through MonkeyCode's free model access, and the model executed on MonkeyCode's free server option — no local GPU, no environment setup, just a prompt and a reply.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Step 3: Run the matrix
Each candidate compiled with -fsanitize=address,undefined and ran the oracle. The threaded variant also ran under -fsanitize=thread. A separate build ran GCC's -fanalyzer for static warnings. The whole matrix took under two minutes per candidate.
Results
Round 1: four defects, no overlap
The first pool compiled and passed the happy path. The matrix found four distinct defects:
| Check | Defect |
|---|---|
| ASan | Off-by-one in the free-list init loop; the constructor wrote one slot past the array |
| UBSan | Misaligned load; the model dropped alignas(T) and packed the free-list pointer before the data bytes |
| Oracle | Double release; the same pointer pushed onto the free list twice, creating a cycle that handed one address to two callers |
| TSan | The mutex was released before the caller's object was constructed — a race on slot contents |
ASan and UBSan found memory errors. The oracle found the logical error. TSan found the concurrency error. None of them overlapped, which is exactly why the matrix had four layers instead of one.
Round 2: mechanical fixes landed, the design bug survived
The sanitizer logs were fed back to the model, one round per defect class. The off-by-one and the alignment issue were fixed in a single pass. The model also introduced a new defect: after the pool was exhausted once, acquire() returned nullptr forever, because a sentinel value collided with a valid index. The oracle caught it in the first 100 operations.
The double-release defect survived Round 2. The model's fix was to zero the slot's bytes on release — which corrupted live objects, because the free-list pointer and the user data shared the same storage. That is a design problem, not a syntax problem. The model could not see the mirror array, so it kept solving the wrong problem.
Round 3: one human invariant
The human change was small: move the free list out of the slots.
// Final design: free list and liveness live outside the slots.
// storage_ is an aligned byte array; slots never store bookkeeping.
std::array<std::size_t, Capacity> free_list_;
std::array<bool, Capacity> live_{};
std::size_t free_head_ = 0;
T* acquire() {
if (free_head_ == kFree) return nullptr;
std::size_t idx = free_head_;
free_head_ = free_list_[idx];
live_[idx] = true;
return reinterpret_cast<T*>(&storage_[idx]);
}
Slightly more memory, completely unambiguous. After that change, the model reimplemented the logic cleanly on the first try.
Final state: zero defects across ASan, UBSan, TSan, and the oracle at 1,000,000 operations. On the test machine, the pool's acquire/release pair measured 12–15 ns median versus 30–40 ns for malloc/free in the same loop. Treat those numbers as a sanity check, not a product claim; the benchmark method is simple enough to rerun anywhere.
Lessons learned
- Sanitizers do not catch logical defects. A double-release is invisible to ASan until it corrupts something else. The mirror array is what made the defect visible in seconds.
- Models fix mechanical defects fast and design defects slowly. The off-by-one and the alignment were one-pass fixes. The double-release needed a human to change the data layout. Do not expect a model to refactor its own design from a sanitizer log.
- Feed the model the failure, not the conversation. Each round received only the latest sanitizer or oracle output plus the relevant code. Small context, targeted fixes.
- The free server option moved the bottleneck. The loop was generate → test → feed back → repeat. Running the model on MonkeyCode's free server removed the local-GPU question entirely, so the loop ran on a laptop. The bottleneck became the test harness — which is where it should be.
Who should not use this approach
This workflow is not universal. Skip it when:
- You cannot write an oracle. If the invariant cannot be expressed as a check, the model's output is unverifiable, and unverified generated code is a liability.
- The code is security-critical and parses untrusted input. A pool is simple; a parser is not. This gate would need to be far stronger than a property test.
- Your compliance rules forbid external model endpoints. The free server option is a convenience, not a substitute for policy.
- You need a proof, not a test. Property tests find bugs; they do not prove absence.
Closing
The pool shipped and is still running in the event-loop tool. The mirror array lives on in the test suite.
The pattern that worked: harness first, model second, sanitizer matrix third, and a human who knows which invariant the model cannot see. If you run a similar pool through a sanitizer matrix, the double-release case is the one to watch — I would be interested to hear what your harness catches first.
Top comments (0)