The short version
A free model wrote a 50-line C++ fuzzer. I pointed it at a config parser I had maintained for two years. It found a heap-buffer-overflow in 40 seconds. My hand-written unit tests had never caught it because they only fed the parser inputs I thought were valid.
This is not another story about AI writing production code. It is the opposite: AI writing the test that breaks production code. The entire loop ran on MonkeyCode's free model access and free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Background
The parser is a small header-only class that reads a key-value config format. It handles comments, quoted strings, and escaped characters. It has 94% line coverage across 212 unit tests. I wrote it, I reviewed it, I trusted it.
The format looks like this:
# comment
name = "Finley"
path = "/tmp/data"
flag = true
The bug was in the escape handler. A string ending with a backslash before the closing quote caused the parser to read one byte past the buffer. My tests covered "hello\nworld" and "tab\there", but never "abc\" — a string that ends with an escaped quote and then nothing.
Goal
I wanted to test a hypothesis: can a free model write a useful fuzzer, not just a plausible one? The success criterion was simple — the fuzzer must find a real bug in real code within five minutes, or the experiment failed.
Step 1 — the prompt
The prompt was minimal. I did not tell the model what the parser looked like or where the bug was. I only described the input format and the API.
Write a C++17 fuzzer for a config parser with this API:
bool parse(const std::string& input);
The input format is line-based key-value pairs.
Keys are alphanumeric. Values can be quoted strings with escapes,
bare words, or booleans. Lines starting with '#' are comments.
The fuzzer should generate random inputs and call parse().
Use a custom mutator: start from a seed corpus and apply
byte flips, insertions, deletions, and block copies.
Report any input that triggers a crash or an exception.
Compile with -fsanitize=address,undefined.
Step 2 — what the model wrote
The model returned a 50-line fuzzer with a mutate() function and a simple loop. The structure was standard: a seed corpus, a random mutation, a call to parse(), and a check for sanitizer failures.
// fuzzer.cpp — generated, lightly trimmed
#include "parser.hpp"
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <random>
#include <string>
#include <vector>
static std::vector<uint8_t> mutate(const std::vector<uint8_t>& seed,
std::mt19937& rng) {
auto data = seed;
int ops = 1 + rng() % 8;
for (int i = 0; i < ops; i++) {
switch (rng() % 4) {
case 0: // bit flip
if (!data.empty())
data[rng() % data.size()] ^= 1u << (rng() % 8);
break;
case 1: // insert random byte
data.insert(data.begin() + (rng() % (data.size() + 1)),
(uint8_t)(rng() & 0xff));
break;
case 2: // delete a byte
if (!data.empty())
data.erase(data.begin() + (rng() % data.size()));
break;
case 3: // copy a block
if (!data.empty()) {
size_t src = rng() % data.size();
size_t len = 1 + rng() % (data.size() - src);
data.insert(data.begin() + (rng() % (data.size() + 1)),
data.begin() + src, data.begin() + src + len);
}
break;
}
}
return data;
}
int main() {
std::mt19937 rng(0xdeadbeef);
std::vector<std::vector<uint8_t>> corpus = {
{'n', 'a', 'm', 'e', ' ', '=', ' ', '"', 'x', '"'},
{'#', 'c', 'o', 'm', 'm', 'e', 'n', 't'},
{'f', 'l', 'a', 'g', ' ', '=', ' ', 't', 'r', 'u', 'e'},
};
for (int iter = 0; iter < 100000; iter++) {
auto& base = corpus[rng() % corpus.size()];
auto mutated = mutate(base, rng);
std::string input(mutated.begin(), mutated.end());
volatile auto result = parse(input);
(void)result;
}
std::printf("completed 100000 iterations\n");
return 0;
}
The code was not elegant. It did not have coverage-guided instrumentation. It was a dumb mutational fuzzer. That was enough.
Step 3 — building and running on the free server
I compiled the fuzzer against the parser with AddressSanitizer and UndefinedBehaviorSanitizer, then ran it on MonkeyCode's free server option. The server gave the fuzzer a clean, disposable environment — no need to worry about sanitizer artifacts or core dumps polluting the workstation.
g++ -std=c++17 -O1 -g -fsanitize=address,undefined -fno-omit-frame-pointer \
-o fuzzer fuzzer.cpp parser.cpp
./fuzzer
Step 4 — the crash
The fuzzer did not reach 100,000 iterations. It crashed at iteration 1,847 — about 40 seconds in. The sanitizer report pointed at parser.cpp:214, in the escape handler:
ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000012
READ of size 1 at 0x602000000012
#0 in parse_quoted_string parser.cpp:214
#1 in parse_line parser.cpp:181
#2 in parse parser.cpp:150
The crashing input was 11 bytes:
a="\"
A key a, an equals sign, a quoted string containing a single backslash, and a closing quote. The parser's escape handler saw the backslash, consumed the next character as the escaped value, and read past the end of the string when the backslash was the last character before the quote.
Step 5 — why my tests missed it
My unit tests were organized by feature: comments, quoting, escapes, booleans. Every test used an input that was valid according to the spec. The escape tests covered \n, \t, \\, and \". None of them covered a trailing backslash before the closing quote, because I never thought of that as a valid input.
The fuzzer did not think in terms of valid inputs. It started from valid seeds and mutated them into invalid shapes. The mutation that broke the parser was a deletion: it removed the character after the backslash, leaving the escape sequence incomplete. That is exactly the class of input a human test author tends to skip.
The fix
The fix was two lines in the escape handler:
if (i + 1 >= input.size()) {
throw std::runtime_error("unterminated escape sequence");
}
After the fix, the fuzzer ran to 100,000 iterations without a crash. I let it run for another 500,000 iterations across three seeds. No new findings.
What this changed
The experiment reversed the usual AI-code workflow. Instead of asking a model to write code and then testing it, I asked a model to write the test and ran it against code I trusted. The model's fuzzer was not clever, but it did not need to be clever. It needed to be persistent and unbiased, and it was.
The free server mattered here for a practical reason: a fuzzer that can crash is not something you want to run on your main machine. Sanitizer builds are slow, core dumps are messy, and a crash in a parser is usually followed by a debugging session. The free server absorbed all of that.
Limitations
This is one parser, one fuzzer, one bug. The fuzzer was a toy: no coverage guidance, no dictionary, no persistent mode. A real fuzzing harness like libFuzzer or AFL++ would likely find the bug faster, but it would also require more setup. The AI-generated fuzzer worked because the target was small and the input format was simple.
The free server option is shared, best-effort infrastructure. If your fuzzing campaign needs deterministic timing, a specific kernel version, or hours of uninterrupted CPU, run your own runner.
Who should not use this
Teams that need formal coverage guarantees should not rely on a 50-line mutational fuzzer. Projects with complex input grammars — think compilers, network protocols, or binary formats — need structured fuzzing with dictionaries and corpus distillation. And if your codebase has no sanitizer support, this workflow will not find memory bugs; it will just crash the process.
The takeaway
The most valuable thing a free model did in this project was not writing a parser. It was writing a tool that broke a parser I believed in. If you are evaluating AI for your team, try the reverse direction: ask the model to write the test, the fuzzer, or the property check, and point it at your own code. The results are easier to trust, because the code under test is yours.
Top comments (0)