Case Study: The C++20 Migration Compiled Clean. A 40-Line Property Test Found the Ordering Break.

Compilation passing under -std=c++20 -Wall -Wextra -Werror proved nothing. The migration changed the sort order of a 2,400-line library, and only a 40-line property test caught it. The compiler validated the syntax. The test validated the behavior. Only the second one matters.

Background

The library sorts report records for an internal reporting tool. Callers depend on a precise order: priority first, then timestamp, then name. The move to C++20 was driven by a toolchain upgrade and a readability goal — the team wanted CTAD, std::erase_if, and fewer hand-written loops. Nothing in that list should change behavior.

I used MonkeyCode's free model access to draft the migration diff, and its free server option to run the verification loop. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Goal

Three success criteria, in order:

1. Identical behavior on the existing golden fixtures.
2. A clean build with warnings treated as errors.
3. A new property test proving the comparator semantics did not drift.

Criterion 3 was the only one that required writing new code. It was also the only one that failed.

Implementation

The loop had six steps:

1. Snapshot behavior: record the SHA-256 of the existing test outputs.
2. Prompt the model with one hard constraint: mechanical transforms only, no behavior changes, keep every hand-written comparator.
3. Apply the diff and build with -std=c++20 -Wall -Wextra -Werror.
4. Run the existing 14 unit tests on the free server.
5. Add the ordering property test.
6. Run the whole job under ASan and UBSan.

The merged diff was 214 lines. About 87% was mechanical: std::lock_guard<std::mutex> → std::lock_guard, erase-remove → std::erase_if, one std::thread → std::jthread. The first draft also added [[nodiscard]] to a function with three ignored call sites; -Werror rejected it, and the model fixed the call sites with (void) in the same round. That was the compile gate working as intended.

The risky 13% was a single change to Record.

Before:

struct Record {
    int id;
    std::string name;
    int priority;         // 0 = urgent
    long long timestamp;  // ms since epoch

    bool operator<(const Record& other) const {
        if (priority != other.priority) return priority < other.priority;
        if (timestamp != other.timestamp) return timestamp < other.timestamp;
        return name < other.name;
    }
};

After (the model's diff):

struct Record {
    int id;
    std::string name;
    int priority;
    long long timestamp;

    auto operator<=>(const Record&) const = default;
};

The model's reasoning was defensible. A defaulted <=> generates all six comparison operators, and operator< is synthesized from it (cppreference). What it missed: the defaulted comparison walks members in declaration order — id, name, priority, timestamp. The hand-written operator compared priority first. Both are valid strict weak orderings. They are not the same ordering.

Asked why it made the change, the model answered that the defaulted operator was "more modern." That is a style argument, not a behavior argument. Style arguments do not survive contact with a sort order.

The artifact: a pairwise ordering test

The fix is a property test that keeps the legacy comparator alive and checks every pair of records against the new one.

// ordering_property_test.cpp
#include "record.h"
#include <algorithm>
#include <random>
#include <vector>

// Pre-migration comparator, preserved verbatim for the test.
static bool legacy_less(const Record& a, const Record& b) {
    if (a.priority != b.priority) return a.priority < b.priority;
    if (a.timestamp != b.timestamp) return a.timestamp < b.timestamp;
    return a.name < b.name;
}

int main() {
    std::mt19937 rng(20260825);
    std::uniform_int_distribution<int> pri(0, 3);
    std::uniform_int_distribution<long long> ts(0, 1'000'000);

    std::vector<Record> records;
    for (int i = 0; i < 120; ++i) {
        records.push_back({i, "name_" + std::to_string(i % 31),
                           pri(rng), ts(rng)});
    }

    for (int round = 0; round < 50; ++round) {
        std::shuffle(records.begin(), records.end(), rng);
        for (std::size_t i = 0; i < records.size(); ++i) {
            for (std::size_t j = 0; j < records.size(); ++j) {
                const bool legacy = legacy_less(records[i], records[j]);
                const bool modern = records[i] < records[j];
                if (legacy != modern) return 1;  // ordering regression
            }
        }
    }
    return 0;
}

120 records × 120 pairs × 50 rounds = 720,000 pairwise decisions. The test runs in under a second. It failed on round 1.

Results

Step	Result
Build with -Werror	Clean, first CI run
Existing 14 unit tests	14/14 passed
Ordering property test	Failed on round 1
ASan / UBSan	Clean
Golden fixture checksums	Identical after the fix

The first mismatch was instructive. legacy_less(rec42, rec17) returned true, because rec17 had priority 0 and rec42 had priority 2. The synthesized < returned false, because 17 < 42 by id. The sort order had changed silently. No warning, no error, no failing unit test.

The fix was to keep the hand-written operator< and drop the defaulted <=>. Smallest possible diff. After that, the property test passed, the sanitizers stayed clean, and the golden checksums matched.

Lessons

1. Compilation is a syntax gate, not a behavior gate. -Werror catches more, but it cannot see semantics. A defaulted comparison can be valid C++ and wrong behavior at the same time.
2. Defaulted <=> makes member declaration order part of the public contract. If callers depend on a priority-first ordering, the struct must be declared in that order — or the comparator must stay hand-written.
3. The free server did not find the bug. The test did. The server's value was making the extended suite free to run on every iteration. The loop cost nothing extra per iteration, so there was no excuse to skip the gate. That is the correct division of labor: the model proposes, the gate disposes.
4. The prompt constraint did most of the work. "Mechanical transforms only, keep every hand-written comparator" reduced the risky part of the diff to one line. The model followed the constraint; the constraint was the actual safeguard.

Decision table for future migrations

Transform	Risk	Gate that catches a mistake
CTAD (std::lock_guard lock(m))	Low	Compiler
std::erase_if replacement	Low	Existing unit tests
std::thread → std::jthread	Medium	Timeout + TSan
Defaulted <=> on an ordered type	High	Pairwise property test

Limitations

This workflow suits small libraries where behavior can be captured in golden outputs and property tests. It does not suit:

• Codebases without existing tests — the property test needs a legacy comparator to compare against.
• Migrations entangled with API redesign — the golden outputs are moving targets.
• Teams that need a permanent record of every model decision — a diff is not an audit trail.

Free-tier CI is also a constraint. The job must stay within the server's time and resource limits. This library built and tested in about three minutes. A monorepo will not.

If you run this kind of migration loop, MonkeyCode's free tier is enough to sustain it — the property test is what makes it safe. Add the pairwise check before you let any model rewrite a comparator. It is 40 lines, it runs in a second, and it will earn its keep the first time the ordering drifts.