The 2 AM Incident: When a Free Model Endpoint Returned 200 and Lied

The alert fired at 2:14 AM. Error rate: 0%. Latency: normal. HTTP status: 200. Everything looked fine. The users were still wrong.

A support ticket arrived at 2:31 AM. "The summary is cut off mid-sentence." Not an error. A truncation. The model returned a complete response. The response was incomplete.

This is the failure mode nobody tests for. Free model endpoints do not always fail loudly. Sometimes they fail politely. This post is the incident report, the fix, and the monitoring pattern that caught it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. The incident below used a generic free endpoint.

The timeline

2:14 AM — A cron job summarizes support tickets. It posts results to a Slack channel. The first truncated summary appears.

2:31 AM — A user reports the truncation. The team checks dashboards. All green.

2:47 AM — A second user reports the same symptom. Different ticket. Same truncation point.

3:02 AM — An engineer manually calls the same endpoint with the same prompt. Gets a full response. The bug reproduces only in production.

Why the dashboards lied

The service checked three things:

1. HTTP status code
2. Response time
3. JSON validity

All three passed. The response was valid JSON. The status was 200. The latency was normal.

The problem was content-level. The model returned a summary field that ended mid-sentence. No error code. No flag. Just a period that never came.

The endpoint had a hidden output token limit. Long inputs consumed the budget. The response was silently cut at the limit.

The reproduction

The bug appeared only with long inputs. Short prompts worked. The team built a minimal repro:

const longInput = "A".repeat(4000);
const res = await fetch(MODEL_URL, {
  method: "POST",
  headers: { "content-type": "application/json" },
  body: JSON.stringify({ prompt: longInput }),
});
const data = await res.json();
console.log(data.summary.length); // 512, exactly

The output length was always 512. A hard cap. The endpoint never mentioned it.

The fix: golden response monitoring

A golden response is a known input with a known expected output shape. Run it periodically. Compare the shape, not just the status code.

The team added a five-minute cron job:

// golden-check.js
const GOLDEN_INPUT = "Summarize this: " + "A".repeat(4000);

async function check() {
  const res = await fetch(MODEL_URL, {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ prompt: GOLDEN_INPUT }),
  });

  const data = await res.json();

  // The real check: content shape, not HTTP status
  const complete = data.summary && data.summary.trim().endsWith(".");
  const lengthOk = data.summary && data.summary.length > 100;

  if (!complete || !lengthOk) {
    console.error("GOLDEN CHECK FAILED", {
      status: res.status,
      length: data.summary?.length,
    });
    process.exit(1);
  }

  console.log("golden check passed");
}

check();

The check failed within ten minutes. The team had a live alert before the next user complaint.

The monitoring pattern

Three layers, in order of increasing cost:

1. HTTP-level — status code, latency. Catches hard failures. Misses silent ones.
2. Schema-level — validate JSON structure. Catches malformed responses. Misses truncated-but-valid ones.
3. Content-level — golden response with shape assertions. Catches silent degradation.

Most teams stop at layer one. The incident happened because nobody built layer three.

The deployment checklist

For any free model endpoint, before trusting it in production:

1. Send a long input. Measure the output length.
2. Send the same input twice. Compare outputs.
3. Run a golden check for 24 hours. Record failures.
4. Set an alert on the golden check, not just on HTTP errors.

Limitations of this approach

Golden responses are not perfect. They catch regressions on one input shape. They do not catch model drift on other inputs. They add load to a rate-limited endpoint. Run them sparingly.

This pattern also assumes the output has a predictable shape. Free-form chat responses are harder to assert. Use it where structure exists: summaries, classifications, extractions.

The takeaway

A 200 status is a transport-level promise. It says the bytes arrived. It says nothing about the meaning of those bytes.

Your monitoring should match the failure modes you actually fear. If a truncated summary is acceptable, skip the golden check. If it is not, build it now. Before 2 AM.

For teams that want a quick start, MonkeyCode's free model access can serve as the test target for this exact golden-check pattern. The code above runs unchanged against any OpenAI-compatible endpoint.