Measure the Escalation Rate. A Local-First Router That Never Earns Back Is Just Latency.

A local-first router only pays off if tier 1 absorbs most of your traffic. If your local model answers 20% of prompts and escalates the rest, you didn't build a cost saver. You built a queue.

So here's the order I'd do it in: build the meter before you build the router. Then let the meter tell you whether the cheap tier deserves to exist.

This is a from-zero tutorial. Every stage ends with a verification step, and the whole thing runs against a fake provider for free before you point it at anything real.

The problem, in one number

Everyone routes local-first because it sounds frugal. Few people measure the escalation rate — the share of prompts that fell through tier 1 and had to hit a remote model anyway.

That number decides everything. High escalation rate? You pay the latency twice and the tokens once. Low escalation rate? The cheap tier is doing real work and you should protect it.

Step 1 — Freeze a replay corpus

You cannot tune a router against prompts you make up on the spot. Freeze real ones.

I take 20–40 prompts from actual logs, strip anything sensitive, and write one JSON object per line:

{"id":"c001","prompt":"Summarize this incident timeline and name the top risk."}
{"id":"c002","prompt":"Turn this changelog into three release notes."}

Verify: wc -l corpus.jsonl matches the number of prompts you think you froze, and every line parses with node -e 'require("fs").readFileSync("corpus.jsonl","utf8").split("\n").filter(Boolean).forEach(l=>JSON.parse(l))'. Boring check. Do it anyway — a broken corpus makes every later number a lie.

Step 2 — Decide what "accepted" means, in code

This is the part people skip, and it's the reason their dashboards are useless. You need a deterministic proxy for "was this answer usable?"

Mine is a schema check plus a refusal check. It is a proxy, and it will accept things a human would throw away. Say that out loud in the code comment so nobody treats it as ground truth.

// accept.mjs — a deterministic proxy, not a judge.
const REQUIRED = ["summary", "risk", "next_step"];

export function accept(text) {
  if (typeof text !== "string" || text.length < 40) return { accepted: false, why: "too_short" };

  let obj;
  try { obj = JSON.parse(text); }
  catch { return { accepted: false, why: "not_json" }; }

  const missing = REQUIRED.filter((k) => !(k in obj));
  if (missing.length) return { accepted: false, why: `missing:${missing.join(",")}` };

  // Truncated output often still parses. Length + required keys is a cheap tripwire.
  if (/i cannot|as an ai/i.test(text)) return { accepted: false, why: "refusal" };
  return { accepted: true, why: "schema_ok" };
}

Verify: feed it one good sample and one truncated sample and confirm you get schema_ok and not_json. If truncation passes your check, your escalation meter is measuring nothing.

Step 3 — One ledger row per attempt, grouped by correlation id

Every model call writes a row. Every row carries the same correlation_id as the other tiers tried for that prompt. That's the join key your meter groups on.

// ledger.mjs — append-only JSONL. One row per attempt, no exceptions.
import { appendFileSync, existsSync, readFileSync } from "node:fs";

const LEDGER = process.env.LEDGER ?? "runs/ledger.jsonl";

export function record(entry) {
  // { ts, correlation_id, tier, model_ref, tokens_in, tokens_out, finish_reason, outcome, accepted }
  appendFileSync(LEDGER, JSON.stringify(entry) + "\n");
}

export function readAll() {
  if (!existsSync(LEDGER)) return [];
  return readFileSync(LEDGER, "utf8").split("\n").filter(Boolean).map((l) => JSON.parse(l));
}

One rule matters here: the local tier logs accepted too. If tier 1 succeeded, that's a row with tier:"local", accepted:true and zero remote tokens. If you only log remote calls, your denominator disappears.

Verify: run one prompt end to end, then cat runs/ledger.jsonl | jq -c . and confirm every row for that prompt shares one correlation_id.

Step 4 — The meter, with a kill threshold

Now the interesting file. It reads the ledger and answers a single question: is tier 1 absorbing work, or adding latency?

// meter.mjs — node meter.mjs --ledger runs/ledger.jsonl --kill-threshold 0.5
import { readFileSync } from "node:fs";

const args = process.argv.slice(2);
const arg = (name, fallback) => {
  const i = args.indexOf(name);
  return i === -1 ? fallback : args[i + 1];
};

const LEDGER = arg("--ledger", "runs/ledger.jsonl");
const KILL = Number(arg("--kill-threshold", 0.5));

const rows = readFileSync(LEDGER, "utf8").split("\n").filter(Boolean).map((l) => JSON.parse(l));

const runs = new Map();
for (const r of rows) {
  if (!runs.has(r.correlation_id)) runs.set(r.correlation_id, []);
  runs.get(r.correlation_id).push(r);
}

let tier1 = 0, escalated = 0, accepted = 0, remoteTokens = 0;

for (const attempts of runs.values()) {
  const first = attempts[0];
  const last = attempts[attempts.length - 1];
  if (first.tier === "local") tier1 += 1;
  if (attempts.length > 1) escalated += 1;
  if (last.accepted) {
    accepted += 1;
    remoteTokens += attempts
      .filter((a) => a.tier !== "local")
      .reduce((n, a) => n + (a.tokens_in ?? 0) + (a.tokens_out ?? 0), 0);
  }
}

const rate = tier1 ? escalated / tier1 : 0;
console.table({
  tier1_attempts: tier1,
  escalated,
  escalation_rate: rate.toFixed(3),
  accepted_answers: accepted,
  remote_tokens_per_accepted: accepted ? (remoteTokens / accepted).toFixed(1) : "n/a",
});

if (rate > KILL) {
  console.error(`escalation rate ${rate.toFixed(3)} > ${KILL}: tier 1 is not earning its latency`);
  process.exit(1);
}

Verify: seed a fake ledger with one local-only success and two escalations. You should see escalation_rate 0.667 and exit code 1. That exit code is what you wire into CI, and it's the whole point — the router fails its own performance review.

The criteria I'd start with, and expect to change once I have real numbers:

7-day escalation rate	Read	Action
under 0.20	Tier 1 carries the load	Keep it, and shrink the remote budget
0.20 – 0.50	Mixed	Group failures by outcome before touching prompts
over 0.50	Tier 1 mostly costs latency	Demote it to cache-only, or delete it
over 0.80 with rising tokens per accepted answer	You're paying twice	Kill tier 1 this week

The middle row is where most teams actually sit. Don't guess your way out of it — group by outcome first. Rate limits, timeouts, and schema failures are three different diseases.

Step 5 — Swap in the real tiers

The plumbing is proven against a fake provider, so now you change two things and nothing else: the tier-2 base URL and key. Mint a route to a prompt.

This is where MonkeyCode's free model access fits in my own setup: it's the tier-2 endpoint, so it's the thing my escalation rate is measured against. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What I can honestly state is limited to what the operator supplied: there is free model access, and there is a free server option. The operator-supplied figure for the free allotment is 10M tokens. I have not independently verified that number, quotas move, and you should confirm the current terms from the project's own documentation before you plan around it. This post is not a source of truth for pricing.

Verify: run the identical corpus against the real tier and diff the ledger outcome sequence against the mock run. Same number of rows, same ordering, or you learned something real about the remote endpoint that no blog post would have told you.

Step 6 — Run the meter on the free server, on a schedule

A one-off escalation rate is an anecdote. A daily one is a trend.

The operator lists a free server option, so I put the meter there and let a scheduled job append one summary row per day: date, escalation rate, tokens per accepted answer. I have not benchmarked that server — sizing, cold starts, and how long the option stays available are unknown to me. Treat them as unmeasured until you measure them yourself.

Verify: after three days you should have three summary rows. If the schedule silently stopped, tokens per accepted answer won't move and you'll notice.

What this does not prove

• The acceptance proxy is a schema check. It accepts answers a human would reject, so your escalation rate is a floor, not a ceiling.
• appendFileSync on a shared ledger is fine for a single process and unsafe for concurrent writers. Add a lock or ship to a log sink before you parallelize.
• Nothing here shows the remote tier is fast, permanent, or unlimited. Free tiers are not SLAs, and this design assumes the remote side can disappear.
• The mock's token counting is irrelevant by design — only provider-reported usage belongs in the ledger, and providers sometimes omit it. Log null and treat it as unknown rather than as zero.

Who should not do this

Skip all of it if you make ten calls a day — just call the model and move on. Skip it if you need a latency SLA, because a router with two hops is slower than one. And skip it if the prompts contain regulated data, since tier 2 is a third party by definition.

But if you already route local-first and you have never printed the escalation rate, that's the bug. Build the meter tonight, run it against a fake provider, and let the exit code decide whether your cheap tier stays.

If you run it and tier 1 gets killed by your own threshold, I want to hear which outcome did it.