Case Study: A Free Model Rewrote a Config Parser. A Differential Fuzz Gate Found the Crash in 90 Seconds.
The patch passed all 14 unit tests. The differential fuzz gate rejected it in 90 seconds: one heap-buffer-overflow, two semantic divergences. Unit tests encode intent. Fuzzing encodes behavior. This is the story of a small C++ config parser, a free model's rewrite, and the gate that caught what code review missed.
Background: a 450-line parser nobody wanted to touch
minicfg is a C++17 INI-style config parser. It lives inside a build helper and parses [section] headers, key=value lines, # comments, and quoted values with \n, \t, and \\ escapes. About 450 lines, hand-written, single-pass.
The parser had 14 unit tests. All green. Line coverage was reported at 100%. Nobody believed the coverage number meant much, but nobody had proof it was wrong either.
The tokenizer had grown organically. Every feature added a special case. The result worked, but it was hard to read.
Goal: add ${VAR} expansion without breaking anything
The feature request was simple: expand ${HOME} and ${PATH} inside values. The constraint was stricter: the rewrite had to pass the existing 14 tests plus a new differential fuzz gate.
I generated the candidate patch with MonkeyCode's free model access. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The fuzzer didn't read the disclosure, and it didn't care.
The model's plan was reasonable. Replace the hand-rolled tokenizer with a std::string_view-based scanner, then add a separate expand_env() pass. The diff was clean. All 14 tests passed on the first try. That was the trap.
Implementation: a differential fuzz gate
A differential gate needs two implementations of the same contract. The original parser is the reference. The patched parser is the candidate. Feed both the same bytes, compare the parse trees.
Step 1: build both parsers into one binary
Both versions compile into one fuzz target. Each exposes a parse() function; the harness calls both and compares the results.
Step 2: write the harness
// diff_fuzz.cpp — differential harness for minicfg v1 vs v2
#include <cstddef>
#include <cstdint>
#include <map>
#include <string>
struct ParseResult {
bool ok;
std::map<std::string, std::map<std::string, std::string>> sections;
};
ParseResult parse_v1(const std::string& input); // reference
ParseResult parse_v2(const std::string& input); // patched
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
std::string input(reinterpret_cast<const char*>(data), size);
ParseResult a = parse_v1(input);
ParseResult b = parse_v2(input);
if (a.ok != b.ok || (a.ok && a.sections != b.sections)) {
// Divergence: write the input for later inspection, then stop.
__builtin_trap();
}
return 0;
}
The comparison is strict. One parser accepts what the other rejects, or the parsed sections differ — the harness traps. libFuzzer writes the offending input to a file and reports a crash.
Step 3: run the gate
#!/usr/bin/env bash
# fuzz_gate.sh — differential fuzz gate for the minicfg rewrite
set -euo pipefail
BUDGET_SECONDS="${1:-120}"
clang++ -std=c++17 -g -O1 \
-fsanitize=fuzzer,address,undefined \
diff_fuzz.cpp src_v1/parser.cpp src_v2/parser.cpp \
-o build/diff_fuzz
mkdir -p corpus
# Seed with real config files plus known edge tokens.
printf '[build]\ncc = "clang++"\n' > corpus/real.cfg
printf 'key = "a#b"\n' > corpus/hash_in_quote.cfg
printf 'key = "abc\\\n' > corpus/trailing_backslash.cfg
./build/diff_fuzz -max_total_time="$BUDGET_SECONDS" corpus/
The gate ran as a CI job on MonkeyCode's free server option. The 120-second budget didn't block the local machine, and the verdict was reproducible: same seed corpus, same flags, same result.
Results: three findings in 90 seconds
The fuzzer hit the first crash at about 41,000 iterations, roughly 90 seconds in.
Finding 1: heap-buffer-overflow in unescape()
The input was key = "abc\ — a quoted value ending in a backslash, then EOF. The patched scanner's unescape() read s[i + 1] without checking that i + 1 was in bounds. AddressSanitizer aborted immediately.
No unit test had ever generated a trailing backslash. The reference parser handled it because its tokenizer checked the next character before consuming it. The model's rewrite optimized that check away.
Finding 2: # inside quoted values
After fixing the crash, the gate found a semantic divergence. Input: key = "a#b". The reference parser returned a#b. The patched parser returned a.
The model's tokenizer treated # as a comment start everywhere, even inside quotes. The original only did that at the start of a line. The unit tests covered comments and quotes, but never the intersection.
Finding 3: CRLF values
Input: key = "a\r\n". The reference parser stripped the \r. The patched parser kept it. On Linux, where the tests ran, the difference was invisible. On Windows config files, it would have corrupted every value.
The ${VAR} expansion feature itself worked correctly. The tokenizer rewrite was the problem.
Lessons learned
A differential oracle doesn't need to be smarter than the model. It needs to be older. The original parser became the specification. The model optimized for the tests it could see; the gate enforced the behavior it couldn't.
Coverage is not a proxy for edge-case behavior. 100% line coverage said nothing about a trailing backslash. The fuzzer found it by generating bytes, not by reading lines.
Fuzzing budgets are cheap. 90 seconds of wall time found a memory-safety bug and two semantic regressions. The fixes were one bounds check and two condition tweaks.
Free model access is useful for generating candidate patches. The gate is what makes them mergeable. The model produced a clean, readable diff. The gate decided whether the diff was safe.
Who should not use this approach
Differential fuzzing requires a reference implementation. If you are rewriting a parser and the old version is already deleted, you have no oracle. Build a characterization test suite first, then delete the old code.
The gate is also overkill for trivial parsers. If your format fits in 50 lines, the cost of maintaining two builds outweighs the risk.
And if you cannot run sanitizers in CI, the memory-safety half of the gate is blind. ASan was the component that caught the crash.
Closing
The three reproducers went back to the model with the gate's verdict. The second attempt passed: one bounds check restored, two tokenizer conditions aligned with the reference.
// The fix was smaller than the finding:
if (i + 1 < s.size() && s[i] == '\\') {
value.push_back(unescape(s[++i]));
}
Total cost: one fuzz run, three small fixes, zero production incidents.
The next time a model hands you a clean diff, ask what your gate would say. Seed the same harness with your real config files and let the fuzzer do the review.
Top comments (0)