Free Tokens Are for Demos, Not CI

At 11:40pm a solo founder has a demo at nine. The feature works on the laptop. Then the test suite starts calling the model on every push.

Two hundred runs later, the free allotment is a memory. The build is green and the demo is dark.

Loops eat quotas. Retries eat quotas. Nothing eats a quota faster than a test that quietly touches a network.

The rule here is blunt. Tests replay recorded fixtures. Demos call the model. Free access funds the second pass and never the first.

One seam, three implementations

The first job is a provider interface. Every call site talks to that interface and nothing else.

// provider.ts — one seam, three implementations
export type Msg = { role: "user" | "assistant"; content: string };

export interface Model {
  name: string;
  generate(messages: Msg[]): Promise<string>;
}

A recorded provider reads a JSONL file. Each line holds a prompt key and the output that a real call returned.

// recorded.ts — replays what a real call returned
import { readFileSync } from "node:fs";
import type { Model, Msg } from "./provider.ts";

type Fixture = { key: string; output: string };

const fixtures: Fixture[] = readFileSync("fixtures/replies.jsonl", "utf8")
  .trim()
  .split("\n")
  .map((line) => JSON.parse(line));

const key = (messages: Msg[]) =>
  messages.map((m) => `${m.role}:${m.content}`).join("|");

export const recorded: Model = {
  name: "recorded",
  async generate(messages) {
    const hit = fixtures.find((f) => f.key === key(messages));
    if (!hit) throw new Error(`no fixture for ${key(messages).slice(0, 60)}`);
    return hit.output;
  },
};

A missing fixture fails loudly. That failure is the point. Prompt drift becomes a red test instead of a silent bill.

Keys built from full prompt text are brittle. Hash the prompt and keep a version prefix in the fixture file. The examples below are illustrative; rename the types to match the repository.

A tripwire that catches stray calls

Fixtures only help if nothing bypasses them. A small test replaces fetch and throws on contact.

// tripwire.test.ts — no network in CI, ever
import { afterEach, beforeEach, expect, test, vi } from "vitest";
import { recorded } from "./recorded.ts";

const original = globalThis.fetch;

beforeEach(() => {
  globalThis.fetch = vi.fn(() => {
    throw new Error("network call in tests: use the recorded provider");
  }) as unknown as typeof fetch;
});

afterEach(() => {
  globalThis.fetch = original;
});

test("pipeline runs on the recorded provider", async () => {
  const out = await runPipeline({ model: recorded, input: "summarise readme" });
  expect(out.length).toBeGreaterThan(0);
});

The tripwire catches the import that slipped past review. It also catches the retry helper that phones home on a timeout.

Recording the pass that costs nothing after

MonkeyCode is an open source project that offers free model access and a free server option. The announced free allotment is 10M tokens, so verify the current terms on the project page before planning around them. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Free access buys the recording pass. One scripted run captures real replies into fixtures/replies.jsonl. Every later run replays them at zero cost.

# record.sh — one real pass, then never again in CI
set -euo pipefail
mkdir -p fixtures
npm run record -- --out fixtures/replies.jsonl --cases tests/cases.jsonl
wc -l fixtures/replies.jsonl

Keep the recorder in scripts/, out of the test runner. CI should hold no token at all, so a leaked key cannot be spent from a fork.

Count characters before you trust the number

A ledger beats intuition. Append one line per real call, then sum it with jq.

// ledger.ts — append-only, one line per real call
import { appendFileSync } from "node:fs";

export function logCall(provider: string, promptChars: number, outChars: number) {
  const row = { at: new Date().toISOString(), provider, promptChars, outChars };
  appendFileSync("ledger.jsonl", JSON.stringify(row) + "\n");
}

#!/usr/bin/env bash
# budget.sh — check the day's spend before the day ends it
set -euo pipefail
LEDGER=${1:-ledger.jsonl}

jq -s '
  { calls: length,
    chars: (map(.promptChars + .outChars) | add // 0),
    approx_tokens: ((map(.promptChars + .outChars) | add // 0) / 4 | floor) }
' "$LEDGER"

The divide-by-four figure is a rough estimate, not a billing statement. Treat it as a smoke alarm. The real check lives in the provider console.

The free server gets the same honesty

A free server is a small box with firm edges. It rewards apps that boot with no secrets at all.

#!/usr/bin/env bash
# preflight.sh — prove the app boots without hidden secrets
set -euo pipefail
PORT=${PORT:-8080}

env -i PORT="$PORT" NODE_ENV=production ./node_modules/.bin/tsx src/server.ts &
pid=$!
trap 'kill $pid 2>/dev/null || true' EXIT

for _ in $(seq 1 30); do
  if curl -fsS "http://127.0.0.1:${PORT}/health" >/dev/null; then
    echo "ok: booted with an empty environment"
    exit 0
  fi
  sleep 1
done

echo "fail: the app needs something that is not in the repo"
exit 1

Run it before every deploy. The day a local .env leaks into the deploy path, the preflight goes red instead of the demo.

Route traffic with one health check and one timeout. Small boxes cold start slowly, so measure boot time on the free tier itself.

Where this approach breaks

Fixtures age when prompts change. Budget an hour each week to re-record the cases that moved.

Recorders hide model drift. A reply captured in October may not match November behaviour, and a replay will never say so. A capped nightly job against the live model covers that gap.

Character counts drift from billed tokens, especially with long system prompts. The ledger is a tripwire, not an invoice.

Free allotments and free servers change. Nothing here is a contract, and no free tier carries a demo-night SLA.

Who should skip it

Teams that need per-token billing accuracy should instrument the provider directly. Products that need sub-second latency from a large model should pay for capacity. Teams whose tests must assert live model behaviour should run a separate capped suite, not this one.

Indie builders shipping this week fit well. The pattern costs an afternoon and removes the worst failure mode: a working product that cannot afford to demo itself.

The repository is public. Open it, record one call, and see what the free path actually buys.