Case Study: ThreadSanitizer Had Final Say on a Free Model's C++ Cache

The problem

A thread-safe LRU cache is a compact component. That makes it attractive to generate with an AI model: one header, one implementation file, one afternoon of review. It also makes it dangerous. In C++, an implementation can pass simple single-thread tests and still violate the C++ memory model under concurrent access, or silently degrade to O(n) lookup when the eviction path is naive.

This case study uses one small project to test a mechanical review workflow. The model produces the implementation; a short sequence of binary checkpoints decides whether the code can merge.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Goal and setup

The aim was not to produce the fastest cache. The aim was to build a review sequence that requires evidence at every step.

I gave a free model a fixed C++20 header and asked it to implement only the .cpp file. I used MonkeyCode's free model access for generation and its free server option for a disposable Linux runner that performed the checks outside my local machine.

The component under test was a bounded LRU cache with this contract:

#pragma once
#include <cstddef>
#include <optional>
#include <string>

template <typename K, typename V>
class ThreadSafeLruCache {
 public:
  explicit ThreadSafeLruCache(std::size_t capacity);

  // Returns the value if present, otherwise std::nullopt.
  std::optional<V> get(const K& key);

  // Inserts or updates a key. Refreshes recency. Evicts LRU if over capacity.
  void put(const K& key, const V& value);

  std::size_t size() const;
  std::size_t capacity() const;

  ThreadSafeLruCache(const ThreadSafeLruCache&) = delete;
  ThreadSafeLruCache& operator=(const ThreadSafeLruCache&) = delete;
};

The model was not allowed to change this header. That constraint matters. A clever model can fix a defect by weakening the API, and a reviewer can miss the change. Keeping the header pinned turns the implementation into a binary problem: the code either satisfies the contract and the tooling, or it does not.

Checkpoint 1: compile with warnings as errors

The first checkpoint is uninteresting on purpose. Nobody should spend time reading generated code that does not compile cleanly under a modern warning set.

g++ -std=c++20 -Wall -Wextra -Werror -Wconversion \
    -pthread -c cache_impl.cpp -o cache_impl.o

For template-heavy code, the implementation file often needs an explicit instantiation to be useful from a test harness. I required the model to add one line at the bottom of the .cpp file:

template class ThreadSafeLruCache<int, std::string>;

This forced the compiler to instantiate the full class and surface template errors that might otherwise hide until link time.

Checkpoint 2: behavioral tests with assertions

The second checkpoint checks the public behavior without concurrency. The test is deliberately small and deterministic.

#include "cache.hpp"
#include <cassert>
#include <string>

int main() {
  ThreadSafeLruCache<int, std::string> cache(2);

  assert(!cache.get(1).has_value());

  cache.put(1, "one");
  cache.put(2, "two");
  cache.put(3, "three");  // capacity 2: evict key 1.

  assert(!cache.get(1).has_value());
  assert(cache.get(2).value() == "two");
  assert(cache.get(3).value() == "three");

  cache.get(2);           // Refresh key 2.
  cache.put(4, "four");   // Evict key 3, not key 2.

  assert(cache.get(2).value() == "two");
  assert(!cache.get(3).has_value());
  assert(cache.get(4).value() == "four");
  assert(cache.size() == 2);
  assert(cache.capacity() == 2);
}

The assertion sequence catches the two most common failures: a cache that does not evict at all, and a cache that evicts the wrong item because it updated recency on put only and not on get.

Checkpoint 3: ThreadSanitizer under contention

Correct single-thread behavior says nothing about data races. The third checkpoint builds with ThreadSanitizer and runs a small concurrent stress test.

#include "cache.hpp"
#include <thread>
#include <vector>

void hammer(ThreadSafeLruCache<int, int>& cache, int seed) {
  for (int i = 0; i < 50'000; ++i) {
    int key = (seed + i) % 512;
    if (i % 3 == 0) {
      cache.put(key, i);
    } else {
      (void)cache.get(key);
    }
  }
}

int main() {
  ThreadSafeLruCache<int, int> cache(64);
  std::vector<std::thread> threads;
  for (int t = 0; t < 4; ++t) {
    threads.emplace_back(hammer, std::ref(cache), t);
  }
  for (auto& th : threads) {
    th.join();
  }
  return 0;
}

g++ -std=c++20 -fsanitize=thread -O1 -g -pthread \
    cache_stress.cpp cache_impl.cpp -o cache_tsan
TSAN_OPTIONS=halt_on_error=1 ./cache_tsan

If the implementation uses one mutex around both the list and the map, the test exits zero. If it uses two locks or a lock-free structure without a valid synchronization story, TSan stops with a race report. The checkpoint does not trust the model's claim that the cache is thread-safe; it requires an exit code.

Checkpoint 4: a complexity watchdog, not a benchmark

The last checkpoint is deliberately coarse. A cache that passes TSan can still become linear when eviction scans a large structure under load. Rather than collect noisy timing numbers on a shared free server, the fourth checkpoint uses a generous watchdog.

timeout 5s ./cache_large

The cache_large binary performs 100,000 mixed operations against a 10,000-entry cache. The five-second limit is set high enough to avoid flaky failures, but low enough to stop an obvious O(n) eviction path from passing on the disposable runner.

This is not a benchmark. It cannot prove O(1) behavior. It only rejects the worst complexity regressions before a human spends time reviewing the code.

Results and decision table

The review sequence reduces the model output to four pass/fail signals:

Checkpoint	Pass condition	Failure means
Compile	Zero warnings or errors under C++20 and -Werror	Code is not ready for review
Behavior	All assertions in the deterministic test pass	Cache contract is wrong
Concurrency	TSan exits zero under 4-thread stress	Data race or synchronization bug
Complexity	100k operations finish under 5s	Pathological eviction or lookup path

A merge requires all four checkpoints. A failed checkpoint returns the output to the model with the exact signal, not with a paragraph of opinion. That is the whole point: the checkpoint output is the review feedback.

Lessons learned

The most important variable was the pinned header. Once the model could not modify the API to dodge a requirement, the remaining work became binary. The second most important variable was running the checks on a disposable server. Local toolchain differences are one of the fastest ways to waste time on generated C++.

The third lesson is that a watchdog is not a benchmark. Calling it a benchmark would imply precision the free server cannot guarantee. The checkpoint is allowed to be coarse because its job is rejection of obvious bad cases, not certification of good ones.

Limitations and who should not use this

This workflow is useful for small, well-specified C++ components where the acceptance criteria are testable. It is not a replacement for a careful review of data structure choices or for long-running stress tests in production. If you need lock-free guarantees, crash consistency, or strict real-time latency, this sequence is insufficient.

The article does not report wall-clock numbers because a single disposable server sample is too noisy. If you repeat the project, treat only the pass/fail signals as stable evidence; collect performance data separately on controlled hardware.

The checkpoint design is the transferable artifact. The cache is just the sample problem.