A Retry Is a Duplicate Waiting to Happen: Idempotency for Free Model Endpoints

Retries fix 429s, but they also create duplicates. A client times out while the server already processed the request. The retry sends the same prompt again. You burn quota twice. You get two different answers.

This is the silent tax of free endpoints. The fix is an idempotency key. One key per logical request. Retries reuse the key, and the server replays the first response. No second inference. No double quota.

This tutorial builds a zero-dependency proxy. It adds idempotency to any OpenAI-compatible endpoint. The setup uses MonkeyCode's free model access as the upstream. The same folder deploys to MonkeyCode's free server option. The pattern works with any endpoint you control.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What idempotency means for LLM calls

An idempotency key is a client-generated string. It identifies one logical request. The server stores the first response under that key. A second request with the same key returns the stored response.

Two rules matter.

1. The key must be stable across retries. Generate it once per logical request. Never generate a new key per attempt.
2. The key must be unique per logical request. Two different prompts must never share a key.

A hash of the normalized body works well. Normalize means stable JSON key order. Strip timestamps and random fields. The hash then survives retries.

Step 0 — Prerequisites

You need four things.

• Node.js 18 or newer, with global fetch
• curl for verification
• A free model endpoint URL and key
• A free server that can host a Node process

MonkeyCode covers the last two. The proxy itself is host-agnostic.

Step 1 — The proxy

Save this file as idem-proxy.mjs.

// idem-proxy.mjs — zero-dependency idempotency proxy
import { createHash } from "node:crypto";
import { readFile, writeFile, mkdir } from "node:fs/promises";
import { createServer } from "node:http";

const UPSTREAM = process.env.UPSTREAM_URL;
const API_KEY = process.env.UPSTREAM_KEY;
const STORE_DIR = process.env.STORE_DIR || "./.idem-store";
const TTL_MS = Number(process.env.TTL_MS || 3600_000);
const TIMEOUT_MS = Number(process.env.TIMEOUT_MS || 60_000);

await mkdir(STORE_DIR, { recursive: true });

function sortKeys(value) {
  if (Array.isArray(value)) return value.map(sortKeys);
  if (value && typeof value === "object") {
    return Object.keys(value)
      .sort()
      .reduce((acc, key) => {
        acc[key] = sortKeys(value[key]);
        return acc;
      }, {});
  }
  return value;
}

function keyFor(body) {
  const normalized = JSON.stringify(sortKeys(body));
  return createHash("sha256").update(normalized).digest("hex");
}

async function readCache(key) {
  try {
    const raw = await readFile(`${STORE_DIR}/${key}.json`, "utf8");
    const entry = JSON.parse(raw);
    if (Date.now() - entry.createdAt > TTL_MS) return null;
    return entry.response;
  } catch {
    return null;
  }
}

async function writeCache(key, response) {
  const entry = { createdAt: Date.now(), response };
  await writeFile(`${STORE_DIR}/${key}.json`, JSON.stringify(entry));
}

const server = createServer(async (req, res) => {
  if (req.method !== "POST" || req.url !== "/v1/chat/completions") {
    res.writeHead(404).end("not found");
    return;
  }

  let raw = "";
  for await (const chunk of req) raw += chunk;

  let body;
  try {
    body = JSON.parse(raw);
  } catch {
    res.writeHead(400).end("invalid json");
    return;
  }

  if (body.stream === true) {
    res.writeHead(400).end("streaming is not supported by this proxy");
    return;
  }

  const key = keyFor(body);
  const cached = await readCache(key);

  if (cached) {
    res.writeHead(200, {
      "content-type": "application/json",
      "x-idem-cache": "HIT",
    });
    res.end(JSON.stringify(cached));
    return;
  }

  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), TIMEOUT_MS);

  let upstreamRes;
  try {
    upstreamRes = await fetch(UPSTREAM, {
      method: "POST",
      headers: {
        "content-type": "application/json",
        authorization: `Bearer ${API_KEY}`,
      },
      body: JSON.stringify(body),
      signal: controller.signal,
    });
  } catch {
    clearTimeout(timer);
    res.writeHead(502).end(JSON.stringify({ error: { message: "upstream unreachable" } }));
    return;
  }
  clearTimeout(timer);

  const text = await upstreamRes.text();
  let payload;
  try {
    payload = JSON.parse(text);
  } catch {
    res.writeHead(502).end(JSON.stringify({ error: { message: "upstream returned non-JSON" } }));
    return;
  }

  if (upstreamRes.ok) {
    try {
      await writeCache(key, payload);
    } catch {
      // a cache failure must not fail the response
    }
  }

  res.writeHead(upstreamRes.status, {
    "content-type": "application/json",
    "x-idem-cache": "MISS",
  });
  res.end(JSON.stringify(payload));
});

server.listen(Number(process.env.PORT || 3000), () => {
  console.log(`idem proxy listening on :${process.env.PORT || 3000}`);
});

Verify the file parses.

node --check idem-proxy.mjs

No output means valid syntax.

Step 2 — Verify cache logic against a mock

Do not burn real quota yet. Run a mock upstream on port 4000.

node -e 'require("http").createServer((req,res)=>{res.writeHead(200,{"content-type":"application/json"});res.end(JSON.stringify({id:"mock-1",choices:[{message:{content:"fixed reply"}}]}))}).listen(4000)'

Start the proxy against the mock.

UPSTREAM_URL=http://localhost:4000 UPSTREAM_KEY=test node idem-proxy.mjs

Send the same request twice, and check the cache header each time.

curl -s -X POST localhost:3000/v1/chat/completions \
  -H "content-type: application/json" \
  -d '{"model":"mock","messages":[{"role":"user","content":"ping"}]}' \
  -D - -o /dev/null | grep -i x-idem-cache

curl -s -X POST localhost:3000/v1/chat/completions \
  -H "content-type: application/json" \
  -d '{"model":"mock","messages":[{"role":"user","content":"ping"}]}' \
  -D - -o /dev/null | grep -i x-idem-cache

The first call returns x-idem-cache: MISS. The second returns x-idem-cache: HIT. The cache logic works.

Step 3 — Point at the real endpoint

Stop the proxy. Restart it with the real upstream.

UPSTREAM_URL=https://your-free-endpoint.example/v1/chat/completions \
UPSTREAM_KEY=your_key \
node idem-proxy.mjs

Replace the URL and key with your MonkeyCode free model credentials. Keep the same request body shape.

Step 4 — Prove idempotency with timing

Send one request twice, and measure both calls.

curl -s -X POST localhost:3000/v1/chat/completions \
  -H "content-type: application/json" \
  -d '{"model":"your-model","messages":[{"role":"user","content":"Explain idempotency in one sentence"}]}' \
  -w "\ntime: %{time_total}s\n" -D - -o /tmp/first.json | grep -Ei "x-idem-cache|time:"

curl -s -X POST localhost:3000/v1/chat/completions \
  -H "content-type: application/json" \
  -d '{"model":"your-model","messages":[{"role":"user","content":"Explain idempotency in one sentence"}]}' \
  -w "\ntime: %{time_total}s\n" -D - -o /tmp/second.json | grep -Ei "x-idem-cache|time:"

Expect MISS on the first call. Expect HIT on the second. The second response should be much faster. Compare the two bodies.

diff /tmp/first.json /tmp/second.json

No diff means a perfect replay. Your retry now costs zero extra quota.

Step 5 — Deploy to a free server

The proxy has zero dependencies. Copy the folder to your free server. Set the same environment variables, and start the process.

MonkeyCode's free server option accepts this folder as-is. If you use another host, the steps are identical.

Verify the public URL.

curl -s -X POST https://your-deployed-proxy.example/v1/chat/completions \
  -H "content-type: application/json" \
  -d '{"model":"your-model","messages":[{"role":"user","content":"ping"}]}' \
  -D - -o /dev/null | grep -i x-idem-cache

The first remote call returns MISS.

Step 6 — Verify retry behavior remotely

Run the same curl twice in a row. The second call must return HIT.

This is the production proof. A client timeout now produces a replay, not a duplicate inference.

When the key should change

The hash changes when the body changes. That is usually correct. Some cases need explicit control.

Situation	Key strategy
Retry after a client timeout	Reuse the same key
New user turn in a chat	New body, new key automatically
Temperature above 0, want fresh output	New key per request
Temperature at 0, deterministic task	Reuse the key to save quota
Client adds a timestamp field	Strip it before hashing

Limitations

The file store is not distributed. Two instances have two stores. Duplicates can still happen across instances.

The store is lost on restart. Free servers restart often. The first request after a restart is a MISS. Correctness survives. Quota savings do not.

Streaming is rejected. A stream: true body returns 400. Streaming clients need a replay buffer, not a cache.

The cache stores raw responses. Do not cache sensitive data unless you control the store.

The TTL defaults to one hour. Long-running conversations can exceed it.

The hash covers the whole body. A retry with a different temperature is a different request by design.

Who should not use this

Skip this proxy if you run multiple instances without shared storage. Skip it if every call must be creative and fresh. Skip it if your client streams. Skip it if you retry once a month. The extra hop is not worth it.

This pattern targets one specific failure: retries after a lost response. If that failure hurts your quota, this proxy removes the pain.

Steal the test in Step 4. It proves idempotency in ten seconds. The proxy is just the vehicle.