The Cold-Start Budget Packet: A Take-Home Where the Free Server Is Allowed to Vanish

A take-home that never simulates a host restart measures almost nothing about cost discipline. The packet below asks one question: after the machine disappears mid-run, does the candidate's harness still know what it spent, and does it refuse to spend more than it can account for? Everything else in the exercise, including the model calls themselves, is scaffolding around that single property.

This version is written for interviewers who hand out agent or LLM-harness tasks, and for candidates who want to know what a strong submission looks like before they open an editor. It assumes free-tier model access and a free server are available to whoever runs the packet, which is exactly the situation where hard budgets and restart behaviour stop being theoretical.

Why the constraint belongs in the interview

A paid API key with a generous ceiling hides the two decisions that separate a careful harness from a fragile one. The first decision is where the spend record is written relative to the network call, and the second is what happens to that record when the process dies between them. Candidates who have only worked against unmetered internal gateways tend to answer both questions by accident rather than by design.

Free tiers make the failure legible without asking anyone to burn money. They also introduce realistic instability: a shared host can be recycled, and a client that has never checkpointed will silently re-pay for every call it already made. The packet turns that instability into a graded requirement instead of an unpleasant surprise in week two.

The prompt handed to the candidate

The candidate receives a repository containing three JSON tasks, a stub transport that returns canned text, and a README. The written brief is deliberately short and numbered, because long briefs let candidates negotiate with ambiguity instead of designing for it.

1. Implement runTask(task, transport) so that the total token spend never exceeds the cap in MAX_TOKENS.
2. Persist enough state that a second process invocation reuses earlier results instead of re-calling the transport.
3. Make an interrupted call visible as spent budget after restart, even though it produced no usable output.
4. Fail loudly with a stable error code when the cap cannot accommodate the next call.
5. Ship a test that kills the transport mid-call and proves points 2 and 3 with assertions, not prose.

The candidate may run this against any provider that offers free model access, or against the included stub if no network is reachable. A free server option is acceptable as the execution host, provided the harness survives that host being restarted; the brief says explicitly that the host is assumed to be disposable.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The rubric, out of twenty points

The scoring table is published to the candidate up front, because hidden rubrics reward guessing and punish engineering. Reviewers score each row independently and record the file and line that justify the score.

Points	Requirement	What earns full credit
5	Budget correctness	Spend is summed from a durable log, and the cap check happens before the call
5	Restart behaviour	Re-running after a kill reuses committed output and does not re-call
4	Interrupted-call accounting	An unmatched reservation counts as spent after restart
3	Error taxonomy	Distinct codes for budget_exhausted, transport_failed, and invalid_task
3	Test quality	A test kills the transport and asserts both reuse and burned spend

Two submission styles score badly on this table even when the demo looks good. An in-memory counter passes a single happy-path run and fails row one the moment the process restarts. A try/catch around everything passes row three only in appearance, because a swallowed error turns an unknown spend into a silent one.

A reference solution: reserve, commit, resume

The shape below is the implementation an interviewer can keep private and compare against. It uses one append-only JSONL file, writes a reservation before the call, and writes a commit after it, so an interruption leaves a witness rather than a gap. The transport is injected, which keeps the packet runnable offline.

// budget-runner.mjs — Node 20+, zero dependencies
import { createHash } from 'node:crypto';
import { appendFileSync, existsSync, readFileSync } from 'node:fs';

const LEDGER = process.env.LEDGER ?? 'ledger.jsonl';
const CAP = Number(process.env.MAX_TOKENS ?? 20000);

const taskKey = (task) =>
  `${task.id}:${createHash('sha256').update(task.prompt).digest('hex').slice(0, 12)}`;

function readLedger(path) {
  if (!existsSync(path)) return { spent: 0, committed: new Map(), rows: [] };
  const rows = readFileSync(path, 'utf8').split('\n').filter(Boolean).map((l) => JSON.parse(l));
  const committed = new Map(); // committed.get(key) -> output
  const open = new Map();      // reservation id -> tokens
  let spent = 0;
  for (const row of rows) {
    if (row.type === 'reserve') open.set(row.id, row.tokens);
    if (row.type === 'commit') {
      open.delete(row.id);
      committed.set(row.key, row.output);
      spent += row.tokens;
    }
  }
  for (const tokens of open.values()) spent += tokens; // burned by an interrupted call
  return { spent, committed, rows };
}

export async function runTask(task, transport) {
  const key = taskKey(task);
  const { spent, committed, rows } = readLedger(LEDGER);
  if (committed.has(key)) return { output: committed.get(key), source: 'ledger' };

  const estimate = transport.estimate(task.prompt);
  if (spent + estimate > CAP) {
    throw new Error(`budget_exhausted spent=${spent} need=${estimate} cap=${CAP}`);
  }

  const id = `${key}#${rows.filter((r) => r.type === 'reserve').length + 1}`;
  appendFileSync(LEDGER, JSON.stringify({ type: 'reserve', id, key, tokens: estimate }) + '\n');

  const res = await transport.call(task.prompt); // the process may die here

  appendFileSync(
    LEDGER,
    JSON.stringify({ type: 'commit', id, key, tokens: res.tokens, output: res.text }) + '\n'
  );
  return { output: res.text, source: 'call' };
}

The reservation is an estimate and the commit carries the measured count, which keeps the accounting conservative without pretending the client can predict usage exactly. A candidate who explains that trade-off in the README, rather than silently picking one number, should score higher on row four than one who hard-codes a constant.

The restart test that decides the review

A single test file separates submissions that reasoned about interruption from submissions that reasoned about the happy path. The stub throws on the first call, simulating a recycled host, and the assertions cover reuse, burned spend, and the cap refusal that follows.

// restart.test.mjs — run with: node --test
import test from 'node:test';
import assert from 'node:assert/strict';
import { rmSync } from 'node:fs';
import { runTask } from './budget-runner.mjs';

const stub = (crashOnCall) => {
  let calls = 0;
  return {
    estimate: () => 100,
    async call() {
      calls += 1;
      if (calls === crashOnCall) throw new Error('transport_failed');
      return { text: `answer-${calls}`, tokens: 100 };
    },
  };
};

test('a killed call stays visible in the budget', async () => {
  process.env.LEDGER = 'test-ledger.jsonl';
  process.env.MAX_TOKENS = '250';
  rmSync('test-ledger.jsonl', { force: true });

  await assert.rejects(() => runTask({ id: 't1', prompt: 'summarise the diff' }, stub(1)));

  const retry = await runTask({ id: 't1', prompt: 'summarise the diff' }, stub(99));
  assert.equal(retry.source, 'call');

  await assert.rejects(
    () => runTask({ id: 't2', prompt: 'draft the changelog' }, stub(99)),
    /budget_exhausted/
  );
});

Run node --test restart.test.mjs and read the ledger afterwards. A correct submission shows three rows: an unmatched reservation, a commit, and a refusal, with the refusal caused by the burned reservation rather than by an arbitrary counter. A broken submission shows a clean commit and no memory of the first attempt, which is the exact bug that turns a free tier into a surprise invoice on a paid one.

Failure modes that recur in review

• Counters in memory. The budget resets with the process, so a restart loop spends without limit while every individual run looks compliant.
• Reserving after the call. The cheapest possible mistake to make and the most expensive to debug, because the only record of an interrupted call is its absence.
• Index-keyed caches. Results stored by array position silently mismatch once tasks are reordered or filtered, and the test still passes.
• Blanket error handling. Catching every exception and continuing converts a known failure into an unknown spend, which no ledger can repair later.
• Backoff against a cap. Retrying a budget refusal with exponential delay wastes wall time without ever making the call admissible.
• Live dependencies in CI. Pointing tests at a shared host makes the suite flaky and hides whether the candidate's own restart logic works.

Each of these maps to a rubric row, which is why the table is worth publishing: the candidate can self-review against the same list the reviewer uses.

Limitations and who should not use this packet

A single-writer JSONL ledger is not concurrency-safe, so the packet is unsuitable for teams that want to grade multi-process throughput or distributed scheduling. Token estimation is approximate by nature, and a strict cap will occasionally refuse a call that would have fit, which is an acceptable trade for determinism in an interview.

Organisations that require private data residency, contractual SLAs, or audited retention should run the transport stub only and keep candidate code entirely local. Anyone evaluating prompt-engineering taste rather than harness engineering should pick a different exercise, because this one grades bookkeeping and recovery, not phrasing.

Finally, treat every availability claim as perishable. Free allowances, host lifetimes, and API surface change without notice, so the README should instruct candidates to confirm the current terms in the provider's documentation on the day they run the packet, and should state plainly that the exercise does not depend on any of those numbers staying fixed.

Where the free tier actually helps

The reason this packet is runnable by anyone is boring and useful: MonkeyCode is an open-source project that offers free model access and a free server option, so an interviewer can hand a candidate a path to real calls without provisioning a paid key first. That matters for the restart test, because a disposable host is precisely the environment the rubric describes, and instructions to "assume the machine can vanish" stop being hypothetical.

The honest boundary is that free access is a convenience for exercises and prototypes, not a capacity plan. If your evaluation needs guaranteed uptime, fixed model versions, or data-handling commitments, the packet should still use the stub and the real provider choice should be made separately.

If you want to try the exercise end to end, the free model access and free server option are enough to get one candidate through the full loop, and the ledger will tell you more about their engineering than the demo ever will.