A Free Model Endpoint Returned Numeric JSON That Was Valid and Wrong

The refund ledger failed on a Tuesday morning. The schema validation passed. The model endpoint had returned clean JSON. Every field had the right type. The account balance was still off by 37 cents.

That gap did not come from a missing field. It came from a number that was syntactically valid and semantically wrong. The C++ service trusted the JSON type system. It should have trusted a decimal contract instead.

The case

The engineer owned a small advisory pipeline. It asked a free model endpoint to classify refund records and return an amount suggestion. The output was never allowed to post directly to the ledger. It fed a review queue. The service validated the response with a standard JSON schema. It checked that amount was present and numeric. Then it parsed the value into a double and added it to a daily total.

The bug surfaced only in a month-end reconciliation. The daily totals were close. They were not exact. One response contained 12.399999999999. Another contained 12.40. The schema accepted both. The ledger could not.

The endpoint was not down. It did not time out. It did not return an empty body. It returned valid JSON with a floating point value that was not safe for money.

The failure mode

A JSON schema validates structure. It does not validate arithmetic. A number with 14 decimal places is still a number. A string such as 12.40 can be valid in a different schema. A negative amount can pass a type check. A value that is ten orders of magnitude too large can also pass.

The same problem appears with scientific notation, leading zeros, missing decimal places, and values that round differently across compilers. Free model endpoints make the problem more frequent because they are less predictable than a fixed parser. That is not a reason to avoid them. It is a reason to put tighter constraints before the value touches business logic.

The team used the operator-supplied free model access from MonkeyCode to repeat the failure. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The article does not claim a specific model name, quota, uptime, or performance level. The harness was plain C++ and did not depend on the provider.

The contract pipeline

The fix was not a retry. It was a five-stage pipeline.

1. Freeze the raw payload before parsing.
2. Extract the suggested amount as a plain token.
3. Parse the token into integer cents.
4. Apply a numeric contract.
5. Compare against a deterministic baseline in shadow mode.

Only after all five stages does the suggestion reach the review queue.

The raw payload matters because free model output can change between reads. Storing the exact bytes makes the failure reproducible. The extraction step matters because a JSON number is not the same as a decimal amount. The baseline matters because a contract can still pass a plausible but wrong value.

The C++ guard

The example below is trimmed for readability. The harness uses a JSON parser to extract the amount token first. The parser below rejects exponent notation and comma separators on purpose. It accepts only plain decimal amounts with at most two fractional digits.

#include <cstddef>
#include <cstdint>
#include <optional>
#include <string>
#include <string_view>

struct RawPayload {
  std::string full_json;
  std::string amount_token;
  std::string source;
  std::uint64_t attempt_id = 0;
};

std::optional<std::int64_t> parseCents(std::string_view token) {
  bool negative = false;
  std::size_t pos = 0;

  if (pos < token.size() && (token[pos] == '+' || token[pos] == '-')) {
    negative = token[pos] == '-';
    ++pos;
  }

  std::int64_t whole = 0;
  std::int64_t frac = 0;
  int frac_digits = 0;
  bool seen_dot = false;

  for (; pos < token.size(); ++pos) {
    const char c = token[pos];
    if (c == '.') {
      if (seen_dot) return std::nullopt;
      seen_dot = true;
      continue;
    }
    if (c < '0' || c > '9') return std::nullopt;
    if (seen_dot) {
      if (frac_digits >= 2) return std::nullopt;
      frac = frac * 10 + (c - '0');
      ++frac_digits;
    } else {
      whole = whole * 10 + (c - '0');
    }
  }

  while (frac_digits < 2) {
    frac *= 10;
    ++frac_digits;
  }

  const std::int64_t cents = whole * 100 + frac;
  return negative ? -cents : cents;
}

Overflow checks are omitted to keep the example short.

Then the numeric contract blocks impossible values.

struct NumericContract {
  std::int64_t min_cents = 1;
  std::int64_t max_cents = 999999999;
  std::int64_t max_shadow_drift_cents = 100;
};

struct ContractDecision {
  bool accepted = false;
  std::string reason;
};

ContractDecision checkContract(
    std::optional<std::int64_t> maybe_cents) {
  if (!maybe_cents) {
    return ContractDecision{false, "not a plain decimal amount"};
  }

  const NumericContract contract;
  const std::int64_t cents = *maybe_cents;

  if (cents < contract.min_cents || cents > contract.max_cents) {
    return ContractDecision{false, "outside expected amount range"};
  }

  return ContractDecision{true, ""};
}

That check catches obvious errors. It does not catch a number that is inside the range but still wrong. A shadow comparison closes part of that gap.

bool withinShadowDrift(std::int64_t live_cents,
                       std::int64_t baseline_cents) {
  const auto delta = live_cents > baseline_cents
                         ? live_cents - baseline_cents
                         : baseline_cents - live_cents;
  return delta <= 100;
}

enum class Stage {
  Raw,
  Parsed,
  ContractOk,
  ShadowOk,
  Applied,
  Rejected
};

Stage promote(const RawPayload& payload,
              std::optional<std::int64_t> baseline) {
  const auto maybe_cents = parseCents(payload.amount_token);
  if (!maybe_cents) return Stage::Rejected;

  const auto decision = checkContract(maybe_cents);
  if (!decision.accepted) return Stage::Rejected;

  if (baseline && !withinShadowDrift(*maybe_cents, *baseline)) {
    return Stage::Rejected;
  }

  return Stage::Applied;
}

The free server option hosted this shadow checker in the team's experiment. The same code ran locally as a pre-commit hook. The maintainers did not treat the free server as the system of record. They treated it as a disposable test surface.

Guard table

Guard	What it stops	What it misses
JSON schema	Missing keys, wrong types	Meaningless values
Plain decimal parser	Scientific notation, too many decimals	Out-of-range amounts
Numeric contract	Negative, zero, enormous values	Plausible but wrong values
Shadow drift	Live endpoint drifting from baseline	Shared bias between endpoints
Human review	Blind adoption failures	Slow, subjective decisions

None of the layers is enough alone. The stack works because the later layers protect the earlier ones.

What this does not solve

This approach is not a payments library. It does not make a free model endpoint safe for final posting, regulated calculations, or irreversible external actions. It also does not make model output deterministic. A free endpoint can still produce valid JSON that fails the contract. It can still return a value that passes every numeric check. The contract only shrinks the error surface.

The example also depends on extracting the correct token before parsing. JSON with duplicate keys, nested values, or unusual number formatting still needs a separate parser strategy. The default double path should be removed for monetary values even if the endpoint usually returns clean input.

The pattern fits an advisory path where a wrong number is visible before it causes damage. It does not fit a system where the model can spend money, close a ledger, or trigger a customer charge without a human gate.

The next improvement is not a bigger model or a faster retry. It is a smaller trust boundary. Validate the shape. Validate the arithmetic. Then let the model advise.