Migration Diary: Rebuild the Cost Ledger You Lose When the Meter Disappears

The short version: a paid AI vendor ships a billing dashboard, and a free endpoint ships a rate-limit error and very little else. So the first artifact you build before a cutover is not a prompt suite or a load test, but your own per-call usage ledger, because the meter you lose is the exact instrument you need to decide which workloads stay on free capacity and which ones earn a paid fallback.

MonkeyCode is an open-source project that, per its operator, offers free model access and a free server option; the free token allowance was advertised as 10 million tokens at the time of writing. Treat every quota number as perishable and confirm the current figure on the project page before you size a workload around it. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

1. Start with an inventory of what the meter was giving you

Before you cancel anything, open the old billing page one last time and write down the four things it actually provided. You got per-call cost attribution, which told you that one feature burned most of the budget. You got a monthly trend line, which was your only early warning that a retry loop had started doubling traffic. You got quota headroom signals, and you got an invoice you could hand to someone for audit purposes.

Three of those four are reconstructible from your own request logs, and the fourth exists on the free side too. Nothing here requires getting clever, but it does require deciding the record shape before your first free request lands, because retrofitting identifiers into a running pipeline is miserable.

2. Define a ledger record you can actually reconcile

Keep the record flat, append-only, and free of prompt text. A single JSON line per model call is enough to answer the questions that matter at the end of a week, and the omission of prompt bodies keeps the file safe to copy between machines.

{"ts":"2026-10-20T09:12:03Z","run_id":"cutover-7","route":"free","feature":"summarize","prompt_tokens":812,"completion_tokens":144,"latency_ms":2310,"status":200,"fallback":false}

The feature field is the one people skip and then regret, because without it you can only report totals. The fallback boolean lets you count the calls that quietly escaped to a paid route, and that count is the real cost of the migration. The run_id header lets you join the ledger to your existing traces whenever something looks wrong.

3. Put a recording proxy in front of every model call

You want one hop between your application and any model endpoint, and that hop writes the ledger. The reference sketch below is a Node/Express proxy you should adapt and test against your own stack; it is not a hardened gateway, and it deliberately never inspects or stores message content.

// ledger-proxy.mjs — reference sketch, adapt and test before relying on it
import express from "express";
import { appendFile } from "node:fs/promises";

const app = express();
app.use(express.json({ limit: "2mb" }));

const UPSTREAM = process.env.UPSTREAM_BASE_URL;
const KEY = process.env.UPSTREAM_API_KEY;
const LEDGER = process.env.LEDGER_PATH ?? "./ledger.jsonl";

app.post("/v1/chat", async (req, res) => {
  const started = Date.now();
  const route = req.get("x-route") ?? "free";
  const feature = req.get("x-feature") ?? "unknown";
  let status = 200, usage = {}, fallback = false;

  try {
    const r = await fetch(`${UPSTREAM}/chat/completions`, {
      method: "POST",
      headers: {
        "content-type": "application/json",
        authorization: `Bearer ${KEY}`,
      },
      body: JSON.stringify(req.body),
    });
    status = r.status;
    const data = await r.json();
    usage = data.usage ?? {};
    res.status(r.status).json(data);
  } catch {
    status = 502;
    fallback = true;
    res.status(502).json({ error: "upstream_unavailable" });
  } finally {
    await appendFile(LEDGER, JSON.stringify({
      ts: new Date().toISOString(),
      run_id: req.get("x-run-id") ?? "adhoc",
      route,
      feature,
      prompt_tokens: usage.prompt_tokens ?? null,
      completion_tokens: usage.completion_tokens ?? null,
      latency_ms: Date.now() - started,
      status,
      fallback,
    }) + "\n");
  }
});

app.get("/healthz", (_req, res) => res.json({ ok: true }));
app.listen(process.env.PORT ?? 8787);

Note the ?? null on the token fields. Some endpoints return a usage object and some do not, so a null count is honest information rather than a bug. When usage is missing you can estimate locally with a tokenizer and store the estimate in a separate field, but never mix estimates and reported counts in one column.

4. Run the proxy on the free server option

This is where free capacity pays for itself twice, because the proxy is small, boring, and does not need a GPU. Start it, point your application's base URL at the proxy, and verify with one request before you touch any production configuration.

npm init -y && npm i express
PORT=8787 UPSTREAM_BASE_URL="$MONKEYCODE_BASE_URL" UPSTREAM_API_KEY="$MONKEYCODE_KEY" node ledger-proxy.mjs

curl -s localhost:8787/v1/chat \
  -H 'content-type: application/json' \
  -H 'x-feature: summarize' \
  -H 'x-run-id: smoke-1' \
  -d '{"messages":[{"role":"user","content":"ping"}]}' | head -c 300

Then handle the boring failure modes: run it under a supervisor so a restart is automatic, ship the JSONL file off the host on a timer so a wiped disk cannot erase your history, and keep an eye on whether the free server's disk and memory fit both the proxy and whatever tooling you colocate.

5. Reconcile weekly and route on evidence

Once you have a week of records, the ledger stops being overhead and starts being a decision tool. This jq one-liner gives you per-feature call volume, token totals, and a rough p95 latency that is good enough for triage.

jq -s 'group_by(.feature) | map({
  feature: .[0].feature,
  calls: length,
  prompt: (map(.prompt_tokens // 0) | add),
  completion: (map(.completion_tokens // 0) | add),
  errors: (map(select(.status >= 500)) | length),
  p95_ms: (map(.latency_ms) | sort | .[((length * 0.95) | floor)]) 
})' ledger.jsonl

Feed the output into a routing table like this one, and revisit it monthly rather than per incident.

Workload	Default route	Escalate when	Why
Batch summarization	free	error rate over 5% for an hour	latency is irrelevant, throughput matters
Code review comments	free, then paid on failure	two consecutive upstream errors	developer already waits on CI
Multi-step tool calling	paid until proven	never, until replay tests pass	partial tool state is expensive to unwind
Latency-critical chat	paid	never	user-visible p95 is a product constraint
Regulated or PII payloads	self-hosted	never	third-party routing needs a review

6. Write down the leftovers before you cancel the old plan

The leftovers are the parts of the old workflow that were never about the model in the first place. Prompt caches keyed to a provider's caching semantics usually do not survive a cutover. Evaluation baselines pinned to a specific model version stop being comparable the moment the model behind the endpoint changes without notice. Billing alerts that referenced a spend threshold have no meaning when spend reads zero.

Also inventory the human leftovers: IDE plugins tied to a seat license, shared dashboards bookmarked by teammates, and the retry policy you tuned against a provider's specific error codes. Each of those needs an owner and a replacement date, or it silently becomes tribal knowledge that leaves when the last person who configured it does.

Limitations and who should not do this

A proxy is a single point of failure, so you need it supervised, and you need a documented switch that bypasses it during an incident. Local token counts are unreliable whenever the upstream omits usage, which means your reconciliation is approximate rather than exact. Free capacity changes shape over time, so any routing table built on today's numbers needs a scheduled review.

Skip this approach entirely if you carry regulated data that cannot transit an intermediary, or if you owe users a latency SLO with contractual teeth. Teams that need a vendor SLA and a support escalation path should stay on a paid contract and use the ledger as a cost report rather than a cutover instrument.

If you want a low-stakes place to run step 3, the free model access and the free server option are enough to host the proxy and absorb the calls while you collect that first week of records. Build the ledger first, read it honestly, and let the numbers — not the price tag — decide what stays free.

Corrections welcome, especially if your endpoint returns usage in a shape the sketch above mishandles.