The Crash Tagger Kept Passing the JSON Contract Until It Didn't: A C++ Drift Ledger for Free Model Labels

At 02:40 the triage queue stopped draining. The worker process was healthy. The Redis length grew. The HTTP client saw 200 OK. The JSON decoder returned objects. But the bucket assignments were wrong. A heap corruption report landed in the network timeout bucket. The on-call engineer opened the payload. The label looked plausible. The JSON was valid. That was the trap.

The service used a free model endpoint to turn raw crash stacks into a small set of triage labels. The model produced clean JSON every time. The schema stayed stable. The values did not. One day heap_use_after_free became network_timeout. Another day stack_overflow became plugin_crash. There was no parsing error. There was no timeout. Retries made it worse. Each attempt returned another valid, equally confident answer.

This article walks through a C++ sidecar that treats a model label as a proposal, not a result. The sidecar applies a deterministic contract first. It records disagreement in a drift ledger. It routes unstable labels to quarantine. MonkeyCode's free model access and free server option appear in the pipeline as the proposal source and a shadow voter.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Start with the failure, not the prompt

The first mistake was to treat the model as a classifier with a stable vocabulary. A free model endpoint is useful for proposing labels from messy input. It is not a deterministic grammar. JSON validity is not label correctness. A response with "label": "network_timeout" and "confidence": 0.96 still passes every serialization check. The bug is invisible at the transport layer.

The sidecar therefore keeps a small contract table. It is not a replacement for the model. It covers only known crash signatures. It is deliberately boring. A rule fires only when a normalized stack contains an exact needle. The goal is not coverage. The goal is a stable reference point for detecting drift.

#include <algorithm>
#include <cctype>
#include <optional>
#include <string>
#include <string_view>
#include <vector>

std::string Normalize(std::string_view trace) {
  std::string out;
  out.reserve(trace.size());
  bool last_space = false;
  for (char c : trace) {
    unsigned char uc = static_cast<unsigned char>(c);
    if (std::isspace(uc) || !std::isprint(uc)) {
      if (!last_space) out.push_back(' ');
      last_space = true;
      continue;
    }
    out.push_back(static_cast<char>(std::tolower(uc)));
    last_space = false;
  }
  return out;
}

struct CrashSignature {
  const char* bucket;
  const char* needle;
};

const std::vector<CrashSignature> CONTRACT = {
  {"heap_use_after_free", "heap-use-after-free"},
  {"stack_overflow", "stack-overflow"},
  {"network_timeout", "etimedout"},
  {"null_deref", "null pointer dereference"},
};

std::optional<std::string> ContractVote(std::string_view trace) {
  const std::string normalized = Normalize(trace);
  for (const auto& rule : CONTRACT) {
    if (normalized.find(rule.needle) == std::string::npos) continue;
    return std::string(rule.bucket);
  }
  return std::nullopt;
}

This code is intentionally small. It owns only the labels the team can verify by hand. It does not need to understand every crash. It needs to catch the moment the model starts calling a known signature by the wrong name.

Record drift without punishing new labels

The contract gate is not a classifier. It is an oracle for a few labels the team can verify by hand. When the contract returns a label and the model returns a different one, the sidecar does not retry. It increments a ledger cell for the contract bucket. When the model agrees, the cell records agreement. The ledger uses a sliding window so an old incident does not poison the current rate forever.

#include <chrono>
#include <deque>
#include <map>
#include <string>

struct DriftEntry {
  std::chrono::steady_clock::time_point at;
  bool agreed;
};

class DriftLedger {
 public:
  explicit DriftLedger(std::chrono::seconds window) : window_(window) {}

  void Record(const std::string& bucket, bool agreed) {
    auto now = std::chrono::steady_clock::now();
    auto& q = cells_[bucket];
    q.push_back({now, agreed});
    while (!q.empty() && now - q.front().at > window_) {
      q.pop_front();
    }
  }

  double DriftRate(const std::string& bucket) const {
    auto it = cells_.find(bucket);
    if (it == cells_.end() || it->second.empty()) return 0.0;
    int drift = 0;
    for (const auto& entry : it->second) {
      if (!entry.agreed) ++drift;
    }
    return static_cast<double>(drift) / it->second.size();
  }

 private:
  std::chrono::seconds window_;
  std::map<std::string, std::deque<DriftEntry>> cells_;
};

The ledger is reactive. It does not block the model after one bad answer. It lets the rate become visible. The routing layer below decides what to do with a disagreement.

Route with the contract as the authority

The helper below receives a normalized trace, a model proposal, a shadow proposal, and the ledger. The contract always wins when it has an opinion. If there is no contract vote, the sidecar falls back to agreement between the two free-model voters.

struct ModelProposal {
  std::string label;
  double confidence = 0.0;
};

std::string Route(const std::string& trace,
                  const ModelProposal& model,
                  const ModelProposal& shadow,
                  DriftLedger& ledger) {
  const auto contract_vote = ContractVote(trace);

  if (contract_vote) {
    const bool model_agreed = model.label == *contract_vote;
    ledger.Record(*contract_vote, model_agreed);
    if (!model_agreed) return "quarantine";
    return *contract_vote;
  }

  const double min_confidence = 0.85;
  const bool model_confident = model.confidence >= min_confidence;
  const bool shadow_confident = shadow.confidence >= min_confidence;
  const bool labels_match = model.label == shadow.label;

  if (labels_match && model_confident && shadow_confident) {
    ledger.Record(model.label, true);
    return model.label;
  }

  return "manual_review";
}

The free server option is useful as a shadow voter when the contract abstains. It runs the same normalized stack through a second service instance. The sidecar treats the shadow as a second opinion, not as ground truth. If the contract and the model disagree, the shadow does not break the tie. The contract wins. If the contract has no opinion and both free endpoints agree with enough confidence, the label can route. If they disagree, the case goes to manual review.

A reproducible test for the routing decision

A test plan should start with four simple cases. The table below shows the expected route before any network call is made.

Input trace snippet	Contract vote	Model label	Expected route
ERROR: heap-use-after-free in arena.cc	heap_use_after_free	network_timeout	quarantine
connect failed: ETIMEDOUT	network_timeout	network_timeout	network_timeout
unknown vendor stack frame	none	plugin_crash	manual_review
unknown vendor stack frame	none	plugin_crash shadow plugin_crash	plugin_crash

The first row is the bug that started the incident. The contract catches it without retrying. The second row confirms the happy path. The third and fourth rows show the behavior when the deterministic oracle has no opinion.

What the free server does not solve

The shadow voter is not an independent ground truth. Both the model and the shadow may share the same training bias. They may also share the same upstream failure. Two wrong confident answers do not become one right answer just because they match. The only non-negotiable check in this design is the contract table.

MonkeyCode's free model access reduces the cost of running a shadow voter. It does not remove the need for a deterministic gate. The sidecar still needs a quarantine path. The ledger still needs a human to watch the drift rate for known buckets.

Limitations

The contract table is narrow. Add a rule only after a human reviews a real crash signature. Too many rushed rules create false positives and erode trust. The confidence threshold is local policy, not a product guarantee. Tune it against labeled data before treating it as a stable number. The drift ledger is in-memory. A process restart loses the window. Persist it if the routing decision depends on history across restarts. No benchmark numbers appear here because endpoint limits were not measured for this article.

Who should not use this approach

Teams with hard latency SLOs should not add a second network vote casually. Systems where a wrong label creates immediate, non-reversible action need a human approval step before routing. Teams that cannot review a quarantine queue will only move the failure to a quieter place. Safety-critical, health, finance, and compliance workloads need controls much stronger than a contract gate and a drift ledger.

The clean JSON response was never the safety rail. The contract gate was. A free model can propose labels cheaply. A free server can add a second opinion. But the sidecar earns routing trust only when a deterministic rule confirms the known cases and the drift ledger shows the model agreeing over time. Start with a small contract table before scaling the model.