Sort the Payload Before the Server

Sensitivity decides the host. Latency only decides the runner-up.

Most local-first arguments start with a stopwatch. That is the wrong first instrument. One leaked API key costs more than a year of slow requests.

Invert the order. Classify every field in the payload first. Then measure latency, then choose a host.

Three classes and one rule

Public data can go anywhere. Derived data needs a contract. Secrets never leave the machine.

Define the classes as integers, so comparisons stay cheap and testable.

// classify.mjs
export const CLASS = { PUBLIC: 0, DERIVED: 1, SECRET: 2 };

const RULES = [
  [/^(api[_-]?key|access[_-]?token|secret|password|authorization|cookie)$/i, CLASS.SECRET],
  [/^(email|phone|ssn|address|card|iban|passport)$/i, CLASS.SECRET],
  [/^(user_?id|account_?id|tenant|session|ip)$/i, CLASS.DERIVED],
];

export function classifyField(name) {
  for (const [re, cls] of RULES) if (re.test(name)) return cls;
  return CLASS.PUBLIC;
}

export function maxClass(value) {
  if (value === null || typeof value !== 'object') return CLASS.PUBLIC;
  let worst = CLASS.PUBLIC;
  const entries = Array.isArray(value) ? value.entries() : Object.entries(value);
  for (const [key, child] of entries) {
    const own = Array.isArray(value) ? CLASS.PUBLIC : classifyField(key);
    const deep = maxClass(child);
    worst = Math.max(worst, own, deep);
  }
  return worst;
}

Name matching is a heuristic, not a proof. It catches the boring majority of leaks. Keep a manual override list for the rest.

The routing rule is one line of policy. Anything above the outbound ceiling runs locally. That ceiling belongs in config, not in a reviewer's memory.

Canary values make leaks visible

A passing test that never sees a secret proves nothing. Plant a unique string in every secret field instead. Then grep every outbound body for it.

// egress.test.mjs
import test from 'node:test';
import assert from 'node:assert/strict';
import { CLASS, maxClass } from './classify.mjs';

const CANARY = 'CANARY-6f2a9c-LEAK';
const outbound = [];

const transport = async (_url, init = {}) => {
  outbound.push(String(init.body ?? ''));
  return new Response('{"ok":true}', { headers: { 'content-type': 'application/json' } });
};

async function route(payload, ceiling = CLASS.DERIVED) {
  const body = JSON.stringify(payload);
  if (maxClass(payload) > ceiling) return { where: 'local', text: 'local-stub' };
  const res = await transport('https://cloud.invalid/v1/chat', { method: 'POST', body });
  return { where: 'cloud', text: await res.text() };
}

test('secret payload stays local', async () => {
  outbound.length = 0;
  const result = await route({ api_key: CANARY, prompt: 'summarize this ticket' });
  assert.equal(result.where, 'local');
  assert.equal(outbound.length, 0);
});

test('clean payload may leave', async () => {
  outbound.length = 0;
  const result = await route({ prompt: 'rewrite this changelog entry' });
  assert.equal(result.where, 'cloud');
  assert.equal(outbound.length, 1);
});

test('no canary ever crosses the wire', () => {
  assert.ok(outbound.every((body) => !body.includes(CANARY)));
});

Run it with plain Node.

node --test egress.test.mjs

Wire the same canary into one real payload in staging. Fill a test tenant with the sentinel string. Then read proxy logs, not application logs. Proxies see what the app forgot it sent.

When the third test fails, you found a real leak for one afternoon of work. That is the whole value of the exercise.

Measure second, not first

Once classification is done, timing becomes simple. Send ten public payloads to both hosts. Record p50 and p95, never the mean.

const times = [];
for (const p of payloads) {
  const t0 = performance.now();
  await route(p);
  times.push(performance.now() - t0);
}
times.sort((a, b) => a - b);
const q = (x) => times[Math.min(times.length - 1, Math.floor(x * times.length))];
console.log({ p50: q(0.5).toFixed(0), p95: q(0.95).toFixed(0) });

Ten samples show a shape, not a verdict. Cold starts and queue depth dominate tiny runs. Warm up once, keep the payloads fixed, and repeat the loop.

This ordering kills most local-first arguments early. Latency rarely changes the host for secret data. It only settles public data.

The decision table

Payload class	Offline needed	Runs on	Reason
PUBLIC	no	free or paid cloud	cheapest per token, no containment cost
PUBLIC	yes	small local model	availability beats quality here
DERIVED	no	cloud, redacted or under contract	policy review, not a firewall
DERIVED	yes	local	a contract cannot cover an outage
SECRET	any	local only	no endpoint qualifies

Read the table top to bottom. The rows never argue with each other.

Offline is a property of the session, not of the model. A local model that fetches a remote tokenizer is not offline. Test that path with the network unplugged.

Where a free server earns a slot

The cloud column needs a sandbox before it needs a contract. MonkeyCode offers free model access and a free server option. The operator currently advertises a 10 million token allotment; quotas and terms change, so read the live page before sizing any work.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use it for the PUBLIC row and the redacted half of DERIVED. Point the transport stub at it and rerun the canary tests there. Free capacity is good for proving a route works. It is not a reason to relax the ceiling.

Never send a SECRET field to a free endpoint. No free tier changes that rule.

The free server wins when the payload is already public. It also wins when your team has no GPU and needs a second data point. It loses when the payload carries credentials, or when you need a throughput guarantee in writing.

Limitations and who should skip this

Field-name classification is shallow. Renamed keys slip through. Pair it with an egress allowlist at the proxy.

The canary test only covers paths you wrote yourself. Third-party SDKs can send data you never see. Audit their network calls on a separate track.

Skip this approach if your data cannot leave a VPC at all. A policy boundary beats a routing trick every time. Skip it too if you need certified isolation; check the operator's terms and your own compliance rules first.

Start with the classifier. It costs one file and one afternoon. Run the canary before you benchmark anything.

If you want to exercise the cloud column without a billing conversation, the free tier is a reasonable sandbox for public payloads.