Dear Past Me: A Model 200 Is Not a Database Commit

Dear past me,

You wired a remote coding model into a webhook. The first response looked almost like valid JSON. You shipped the handler right before lunch anyway.

By evening the table held four ghost rows. The remote hop still answered with HTTP 200. Your catch block kept retrying the same broken shape.

That pattern wasted a working day. The model was not the root cause. Your output contract was missing.

The failure you can reproduce without folklore

Status 200 only proves the socket completed. It does not prove the body matches a schema. A trailing comma still returns 200. A markdown fence still returns 200.

Retries without keys multiply remote work. Four retries become four separate jobs. Four jobs write four partial records.

An unversioned prompt cannot be replayed later. Logs that say "summarize user" are not evidence. You cannot diff a template literal from Tuesday.

Cheap generation does not cheapen contracts. Architecture still starts at the response boundary. Pin that boundary before you pin a stack.

Three mistakes that actually cost the day

These are protocol errors, not vibe checks. None of them require a new model. All three show up on free remote hops.

Mistake 1: You treated HTTP 200 as a commit

Your handler looked like the sketch below. Treat it as a labeled bad example.

// BAD EXAMPLE — do not ship this
async function handleJob(req, res) {
  const text = await callRemoteModel(req.body.prompt);
  const data = JSON.parse(text); // throws on almost-JSON
  await db.insert(data);
  res.status(200).end();
}

JSON.parse is not a contract. It accepts any object shape. It also explodes on fenced markdown.

Mistake 2: You retried generation as a brand-new job

The catch block pushed a fresh queue item. That item had no idempotency key. The remote hop saw four independent POSTs.

Shared free hops hate retry storms. Connection slots are not infinite. Your backlog then looked like provider downtime.

Mistake 3: You never pinned the prompt as an artifact

The prompt sat inside a template literal. You changed one adjective after lunch. Afternoon failures could not replay morning calls.

Temperature, token cap, and prompt text are one unit. Split them and debugging becomes folklore. Version the unit on disk.

The workflow to install before the next webhook

Follow these numbered steps in order. Do not skip the tests at the end. Label this as a method, not a production war story.

Step 1: Freeze a prompt version on disk

Store the prompt as a file. Do not keep it in application code. Name the file with a version suffix.

prompts/user-summary.v3.txt

Return ONLY JSON. No markdown. No preface.
Schema:
{"user_id": string, "summary": string, "risk": "low"|"med"|"high"}
User id: {{user_id}}
Notes: {{notes}}

Load it with an explicit version constant. Fail boot if the file is missing.

import { readFileSync } from 'node:fs';

export const PROMPT_VERSION = 'user-summary.v3';
export const PROMPT_TEMPLATE = readFileSync(
  new URL(`../prompts/${PROMPT_VERSION}.txt`, import.meta.url),
  'utf8'
);

Step 2: Define a JSON Schema and reject extras

Use a schema validator on every model body. Additional properties must be errors. Missing required fields must be errors.

export const summarySchema = {
  type: 'object',
  additionalProperties: false,
  required: ['user_id', 'summary', 'risk'],
  properties: {
    user_id: { type: 'string', minLength: 1, maxLength: 64 },
    summary: { type: 'string', minLength: 1, maxLength: 500 },
    risk: { type: 'string', enum: ['low', 'med', 'high'] }
  }
};

Step 3: Strip fences, then validate, then persist

Remote models wrap JSON in fences. Strip first. Validate second. Write last. Never reverse that order.

import Ajv from 'ajv';

const ajv = new Ajv({ allErrors: true, removeAdditional: false });
const validate = ajv.compile(summarySchema);

export function parseModelJson(raw) {
  const stripped = String(raw)
    .replace(/^```
{% endraw %}
(?:json)?\s*/i, '')
    .replace(/\s*
{% raw %}
```$/i, '')
    .trim();
  const start = stripped.indexOf('{');
  const end = stripped.lastIndexOf('}');
  if (start < 0 || end <= start) {
    throw new Error('MODEL_BODY_NOT_JSON');
  }
  const parsed = JSON.parse(stripped.slice(start, end + 1));
  if (!validate(parsed)) {
    const err = new Error('MODEL_SCHEMA_MISMATCH');
    err.details = validate.errors;
    throw err;
  }
  return parsed;
}

Step 4: Put an idempotency key on every remote call

Hash the prompt version, the inputs, and the job id. Persist that key before you call the hop. Also send it as a header when the hop accepts one.

import { createHash } from 'node:crypto';

export function idempotencyKey({ jobId, promptVersion, userId }) {
  return createHash('sha256')
    .update([jobId, promptVersion, userId].join(':'))
    .digest('hex');
}

Store terminal states only: accepted, invalid, failed. Do not retry invalid. Schema failures are not transient faults.

Step 5: Bound the hop with timeout and single-flight

A free remote server is a shared network hop. Timeouts belong in the contract. Duplicate in-flight keys must join one promise.

const inflight = new Map();

export async function callRemoteModelOnce({ url, key, body, ms = 12_000 }) {
  if (inflight.has(key)) return inflight.get(key);

  const run = (async () => {
    const res = await fetch(url, {
      method: 'POST',
      headers: {
        'content-type': 'application/json',
        'idempotency-key': key
      },
      body: JSON.stringify(body),
      signal: AbortSignal.timeout(ms)
    });
    if (!res.ok) {
      throw new Error(`MODEL_HTTP_${res.status}`);
    }
    return res.text();
  })();

  inflight.set(key, run);
  try {
    return await run;
  } finally {
    inflight.delete(key);
  }
}

Twelve seconds is an example timeout only. Measure your own p95 before copying it. Do not publish that number as a benchmark.

Step 6: Assemble the handler in one direction

Read the version. Render the prompt. Call once. Parse. Persist. Record the key. Invalid bodies must not re-enter the queue.

export async function handleSummaryJob(job, deps) {
  const key = idempotencyKey({
    jobId: job.id,
    promptVersion: PROMPT_VERSION,
    userId: job.userId
  });

  const prior = await deps.store.get(key);
  if (prior?.status === 'accepted' || prior?.status === 'invalid') {
    return prior;
  }

  const prompt = PROMPT_TEMPLATE
    .replaceAll('{{user_id}}', job.userId)
    .replaceAll('{{notes}}', job.notes);

  let raw;
  try {
    raw = await callRemoteModelOnce({
      url: deps.modelUrl,
      key,
      body: { prompt, version: PROMPT_VERSION }
    });
  } catch (err) {
    await deps.store.put(key, { status: 'failed', error: String(err) });
    throw err;
  }

  try {
    const data = parseModelJson(raw);
    await deps.db.insertSummary(data);
    const accepted = { status: 'accepted', data, promptVersion: PROMPT_VERSION };
    await deps.store.put(key, accepted);
    return accepted;
  } catch (err) {
    const invalid = {
      status: 'invalid',
      error: String(err),
      rawPreview: String(raw).slice(0, 200)
    };
    await deps.store.put(key, invalid);
    return invalid;
  }
}

Notice invalid does not rethrow into retry logic. That single branch is the whole point. Ghost rows come from retrying typed failures.

A stub server that returns 200 plus junk

Unit tests should not call a live model. Point integration checks at a local stub first. The stub must return HTTP 200 with a bad body.

// stub-server.mjs — labeled example, not a hosted service
import http from 'node:http';

const server = http.createServer((req, res) => {
  if (req.method !== 'POST') {
    res.writeHead(405);
    res.end();
    return;
  }
  res.writeHead(200, { 'content-type': 'text/plain' });
  res.end('Sure. ```

json\n{"user_id":"u1","summary":"ok","risk":"hot"}\n

```');
});

server.listen(8787, '127.0.0.1');
console.log('stub listening on 127.0.0.1:8787');

Start it with node stub-server.mjs. Then point deps.modelUrl at http://127.0.0.1:8787. Confirm the handler stores invalid and inserts zero rows.

A test plan you can run without faith

Label these as executable tests. They are not production metrics. Run them before any live hop.

import test from 'node:test';
import assert from 'node:assert/strict';
import { parseModelJson, idempotencyKey } from './contract.js';

test('fenced almost-json is rejected, not inserted', () => {
  const raw = '```

json\n{"user_id":"u1","summary":"ok","risk":"hot"}\n

```';
  assert.throws(() => parseModelJson(raw), /MODEL_SCHEMA_MISMATCH/);
});

test('idempotency key is stable for the same inputs', () => {
  const a = idempotencyKey({
    jobId: 'j1',
    promptVersion: 'user-summary.v3',
    userId: 'u1'
  });
  const b = idempotencyKey({
    jobId: 'j1',
    promptVersion: 'user-summary.v3',
    userId: 'u1'
  });
  assert.equal(a, b);
});

test('prompt version change produces a new key', () => {
  const a = idempotencyKey({
    jobId: 'j1',
    promptVersion: 'user-summary.v3',
    userId: 'u1'
  });
  const b = idempotencyKey({
    jobId: 'j1',
    promptVersion: 'user-summary.v4',
    userId: 'u1'
  });
  assert.notEqual(a, b);
});

Run them with node --test. Add one integration test against the stub. Assert the database insert count stays at zero.

Decision table for retries

Print this table beside the worker. Argue from the table, not from panic.

Observation	Treat as	Retry?	Persist domain row?
HTTP 429 or timeout	transient	yes, with backoff	no
HTTP 200 + schema miss	invalid	no	error row only
HTTP 200 + valid schema	accepted	no	yes
HTTP 401 or 403	config	no	no
Duplicate idempotency key	replay	no	return stored

Transient faults may retry with jitter. Invalid bodies must not. Config errors need a human, not a loop.

Where a free remote hop fits

A local stub cannot catch every fence and prefix. You still need one live URL for integration checks. Keep that URL behind environment config.

MonkeyCode offers free model access and a free server option you can point this client at while rehearsing the contract. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Do not treat that hop as a database. Keep the schema, the key, and the timeout in your repo. If the hop changes, local tests still fail first.

Read current product docs before assuming any limit. Do not copy quotas from a screenshot. Do not pin a model name this article never measured.

Limitations, stated plainly

This workflow does not make a model truthful. It only makes failures typed. A hallucinated user_id can still pass the schema.

It does not replace a queue with backoff. It only stops schema errors from becoming storms. You still need a worker and a dead-letter path.

It does not pin a specific model. Free remote hops can rotate without notice. Pin your prompt version and your schema instead.

Single-flight lives in one process. Two Node instances still need a shared store. Use Redis or SQLite once you run multiple workers.

The 200-character rawPreview is for logs. It is not a full forensic archive. Redact notes before you store previews.

Who should not use this approach

Do not use this as a medical, legal, or payments authority. A JSON schema is not a compliance control.

Do not use this if output is unbounded code execution. That is a sandbox problem, not a parser problem.

Do not use this if you cannot store idempotency keys. Without a store, the table above is theater.

Skip it if you only generate throwaway drafts in a REPL. The ceremony is for webhooks, workers, and writers.

What to do before the next deploy

1. Move the prompt into a versioned file.
2. Add the schema test with a fenced bad body.
3. Refuse retries on invalid.
4. Log promptVersion beside every stored row.

Then replay one failing job by key. If you cannot replay it, you are still in yesterday. Contracts belong in git. Status codes do not.

If you need a remote URL after the stub passes, check current MonkeyCode docs for the free server option and verify limits there.