Case Study: A Free Model's C++ Token Bucket Passed 18 Tests. NTP Broke It in One Second.

A token bucket is simple. The clock underneath it is not.

I needed a rate limiter for a telemetry gateway that forwards events to an upstream API capped at 100 requests per second. I asked a free model endpoint to implement the bucket and a unit-test suite. All 18 tests passed. Then a 60-second load test showed second 17 handling 300 requests, followed by two seconds of total starvation.

The root cause was not the bucket logic. It was std::chrono::system_clock.

Background

The gateway batches events and sends them to an upstream that returns 429 when the rate exceeds 100 req/s. Retrying after a 429 makes the problem worse. The limiter has to be local, cheap, and correct under load.

A token bucket is the standard answer: tokens refill at a fixed rate, each request spends one token, and the bucket caps at a burst size. The implementation is about twenty lines. The failure modes live in the details.

Goal

Produce a C++ token bucket that:

1. Allows a sustained rate of 100 req/s with a burst of 100.
2. Never exceeds the configured rate, even when the server clock changes.
3. Works for fractional rates, down to 0.1 req/s.
4. Comes with tests that prove all of the above.

The constraint: the implementation and tests would be generated by a free model endpoint, and the whole verification loop would run on a free server. I would not hand-write the first version. I would only audit it.

Setup: the toolchain

I used MonkeyCode's free model access to generate the code, and its free server option to run the compile-test-soak loop. MonkeyCode is an open-source project; its free tier currently includes 10 million tokens and a free server option for jobs like this one. The README documents the current terms, and the numbers here are from one run on 2026-08-24, not a benchmark of the product.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The gate was simple: no generated code reaches the gateway until it passes unit tests, a load test, and a soak test. That gate is the point of this case study.

Step 1: Generate the first version

The prompt was one paragraph: "Implement a token bucket in C++17 with try_acquire(). Rate is tokens per second, capacity is the burst. Use a header-only class. Include unit tests."

The model returned this:

// token_bucket.hpp — v1, generated
#include <chrono>
#include <algorithm>

class TokenBucket {
public:
    TokenBucket(double rate_per_sec, double capacity)
        : rate_(rate_per_sec), capacity_(capacity),
          tokens_(capacity), last_(std::chrono::system_clock::now()) {}

    bool try_acquire() {
        auto now = std::chrono::system_clock::now();
        double elapsed = std::chrono::duration<double>(now - last_).count();
        tokens_ = std::min(capacity_, tokens_ + elapsed * rate_);
        last_ = now;
        if (tokens_ >= 1.0) {
            tokens_ -= 1.0;
            return true;
        }
        return false;
    }

private:
    double rate_;
    double capacity_;
    double tokens_;
    std::chrono::system_clock::time_point last_;
};

It looks correct. That is the trap.

Step 2: Unit tests pass

The model also generated 18 unit tests: burst consumption, refill after sleep, no negative tokens, capacity cap. I compiled with g++ -std=c++17 -Wall -Wextra and ran them.

[==========] 18 tests from 4 test suites ran.
[  PASSED  ] 18 tests.

All green. A free model wrote a correct-looking limiter, and the tests agreed. I still did not trust it. Time-based code fails on time, not on logic.

Step 3: The load test harness

I wrote a harness that measures what actually happens, second by second. It runs the bucket for 60 seconds, counts requests per second, and reports the total. The harness uses steady_clock for measurement, independent of whatever clock the bucket uses internally.

// load_test.cpp
#include "token_bucket.hpp"
#include <chrono>
#include <cstdio>
#include <thread>

int main() {
    TokenBucket bucket(100.0, 100.0);
    const int kSeconds = 60;
    int per_second[kSeconds] = {0};
    auto start = std::chrono::steady_clock::now();
    int total = 0;

    while (true) {
        auto now = std::chrono::steady_clock::now();
        int sec = static_cast<int>(
            std::chrono::duration_cast<std::chrono::seconds>(now - start).count());
        if (sec >= kSeconds) break;
        if (bucket.try_acquire()) {
            per_second[sec]++;
            total++;
        } else {
            std::this_thread::sleep_for(std::chrono::milliseconds(1));
        }
    }

    for (int i = 0; i < kSeconds; i++)
        std::printf("second %2d: %4d\n", i, per_second[i]);
    std::printf("total: %d (expected ~%d)\n", total, 100 * kSeconds);
}

Build and run on the free server:

g++ -std=c++17 -O2 load_test.cpp -o load_test
./load_test

Results: what the harness found

The output was not a flat 100 per second.

second 15:  101
second 16:   99
second 17:  300
second 18:    0
second 19:    0
second 20:  102
...
total: 5998 (expected ~6000)

Second 17 handled 300 requests. Then the bucket starved for two seconds. The total was close to expected, but the shape was wrong: a burst three times the configured limit, followed by silence. For an upstream that 429s at 100 req/s, that burst is a failure.

Step 4: The audit

The logic was fine. The clock was not. system_clock is wall time. On a shared server, NTP can step it forward or backward by seconds. A forward step makes elapsed large, so the bucket grants a huge refill instantly. A backward step makes elapsed negative, so the bucket denies everything until the debt is repaid.

The fix is to use a monotonic clock and integer micro-tokens, so fractional rates do not depend on floating-point comparisons:

// token_bucket.hpp — v2, after the audit
#include <chrono>
#include <cstdint>
#include <algorithm>

class TokenBucket {
public:
    TokenBucket(double rate_per_sec, double capacity)
        : rate_per_us_(rate_per_sec / 1'000'000.0),
          capacity_us_(static_cast<int64_t>(capacity * 1'000'000.0)),
          tokens_us_(capacity_us_),
          last_(std::chrono::steady_clock::now()) {}

    bool try_acquire() {
        auto now = std::chrono::steady_clock::now();
        auto elapsed_us = std::chrono::duration_cast<std::chrono::microseconds>(
            now - last_).count();
        last_ = now;
        tokens_us_ = std::min(capacity_us_,
            tokens_us_ + static_cast<int64_t>(elapsed_us * rate_per_us_));
        if (tokens_us_ >= 1'000'000) {
            tokens_us_ -= 1'000'000;
            return true;
        }
        return false;
    }

private:
    double rate_per_us_;
    int64_t capacity_us_;
    int64_t tokens_us_;
    std::chrono::steady_clock::time_point last_;
};

Clock choice is a decision, not a default:

Clock	Monotonic	Safe against NTP steps	Use for rate limiting
system_clock	No	No	No
steady_clock	Yes	Yes	Yes
high_resolution_clock	Usually	Usually	Check your platform

Step 5: The soak test found the second bug

The load test passed after the fix: max 104 per second, min 96, total 5,987. Good enough for a 100 req/s cap.

Then I ran a 24-hour soak at 0.1 req/s, simulating a slow upstream. The v1 code with tokens_ >= 1.0 granted 8,639 tokens instead of 8,640. The double comparison lost one token to rounding at the boundary. The integer version granted exactly 8,640.

Unit tests never caught either bug. The first needed a real clock and NTP. The second needed a fractional rate and 24 hours.

Lessons learned

1. Green unit tests on generated code prove the logic, not the environment. Time-based code must be load-tested with a monotonic measurement clock.
2. system_clock is for timestamps. steady_clock is for measuring. Mixing them in a rate limiter turns an NTP sync into a traffic burst.
3. Floating-point token counts hide boundary errors. Integer micro-tokens make the math exact.
4. Free infrastructure changes the economics of verification. The model call, the compile, the 60-second load test, and the 24-hour soak all cost $0. The gate is what costs attention.

Limitations

This approach is not for everyone. If your limiter guards money movement, safety systems, or anything where a 3x burst is catastrophic, you need formal reasoning and production-scale load testing, not a soak test on a free box. The free server option is appropriate for CI, test harnesses, and long soak runs. It is not a substitute for production infrastructure.

Reproduce it

The harness above is complete. Run it against v1, watch second 17, apply the clock fix, run it again, then run the 24-hour fractional-rate soak. The whole loop is a few commands. The README for MonkeyCode's open-source project has the current terms for the free model access and the free server option.