The Cold-Start Take-Home: Grade Whether an Agent Survives an Ephemeral Free Server

An agent take-home that only grades the happy path quietly rewards designs that cannot survive a restart, a redeploy, or a suspended container. The cold-start take-home adds exactly one adversarial condition to an otherwise ordinary task: the process is killed at an arbitrary point and must finish correctly after coming back. That single constraint exposes state placement, idempotency, and recovery honesty faster than any code review of the prompt layer.

The reason this belongs in an interview loop now is practical rather than fashionable. Free and low-cost hosting tiers commonly suspend idle processes, restart containers without warning, or ship without durable local disk, so "it worked on my machine" has become the default failure mode for junior agent work. Candidates who can reason about ephemeral compute produce demos that survive a reviewer clicking the link on the following morning.

Why an ephemeral server is the sharper constraint

Most take-homes test whether a model call can be wired to a tool, which is a fifteen-minute exercise with a decent SDK. The interesting engineering question sits one layer down, where the agent has to record what it already did and prove it did not do it twice. A restart is the cheapest available probe of that layer, and it needs no paid infrastructure to reproduce.

Three assumptions drive the whole exercise, and each should be stated in the prompt rather than implied:

• Local disk may not persist between restarts, so anything load-bearing must live in an external store or be reconstructible.
• The process may be killed mid-step, so every side effect needs a stable identity that makes a retry a no-op.
• The reviewer may open the demo hours after the candidate last touched it, so cold-start behaviour is the normal case and not the edge case.

The prompt to hand candidates

The task below is deliberately small. Its purpose is to produce a gradeable artifact, not to consume a weekend.

Build a minimal agent with two tools: fetch_price(sku) returning a canned value, and charge_ledger(run_id, step_id, amount) writing one row to storage. Expose a single HTTP endpoint that runs a three-step plan behind it. The service must be deployed on a free hosting option and must complete correctly if the process is killed at any point during the run. Submit the repository, a short README, and the output of your restart test.

Two details in that wording matter more than they look. Naming run_id and step_id in the tool signature hands the candidate the shape of an idempotency key without handing them the design. Requiring the restart test output forces evidence instead of narration, which is where most submissions quietly fall apart.

The rubric

Each dimension is scored 0 to 2, and the score is intended to be defensible from the repository alone rather than from an interview conversation.

Dimension	0	1	2
State placement	In-memory dict or local file treated as durable	External store, but path or connection hardcoded to a dev machine	External store with configuration, and no assumption of local persistence
Idempotency	No key, retries duplicate rows	Key exists but is time-based or random per attempt	Key derived from run and step, enforced by a uniqueness constraint
Recovery path	Restart loses the run entirely	Restart re-executes from the beginning and silently duplicates	Resume reads committed progress and replays only uncommitted steps
Evidence	Claimed in prose	Manual log pasted without a reproducible command	Scripted kill-and-restart check printing the ledger diff
Honest scoping	Presents untested paths as working	Notes gaps only when asked	README lists what was verified, how, and what was not

A submission can score well without any clever orchestration. The rubric rewards the boring version that commits progress and tolerates a duplicate call, which is precisely the instinct a production team wants to hire.

Sample solution sketch

The following is illustrative and was not executed for this article. It shows the shape of a passing answer, not a library to copy.

# illustrative only: unexecuted sketch of resume-safe step execution
import sqlite3, os

DB = os.environ["LEDGER_PATH"]  # must not point at ephemeral local disk

def init():
    with sqlite3.connect(DB) as c:
        c.execute("""CREATE TABLE IF NOT EXISTS ledger (
            run_id TEXT NOT NULL,
            step_id TEXT NOT NULL,
            amount INTEGER NOT NULL,
            PRIMARY KEY (run_id, step_id))""")

def charge(run_id: str, step_id: str, amount: int) -> bool:
    """Returns False when the step was already committed."""
    with sqlite3.connect(DB) as c:
        try:
            c.execute("INSERT INTO ledger VALUES (?,?,?)", (run_id, step_id, amount))
            return True
        except sqlite3.IntegrityError:
            return False

def completed_steps(run_id: str) -> set[str]:
    with sqlite3.connect(DB) as c:
        return {r[0] for r in c.execute(
            "SELECT step_id FROM ledger WHERE run_id=?", (run_id,))}

The plan executor then reads completed_steps(run_id) before each step and skips anything already present. The PRIMARY KEY does the real work: a duplicate attempt raises IntegrityError inside the same transaction that would have written the row, so there is no window in which a partial write survives. The sketch assumes a SQLite file on a volume that actually persists; on hosts without one, the same pattern moves to a managed database, and that substitution is exactly the judgement the rubric is grading.

Running the restart probe

The candidate's evidence should be a command a reviewer can re-run, not a screenshot. A minimal probe kills the worker mid-run and then compares the ledger against the expected step count.

# illustrative probe: one kill, one restart, one invariant check
RUN_ID="demo-$(date +%s)"
curl -sS "$BASE_URL/run?run_id=$RUN_ID" &
PID=$!
sleep 1 && kill -9 $PID 2>/dev/null || true

curl -sS "$BASE_URL/run?run_id=$RUN_ID"
sqlite3 "$LEDGER_PATH" \
  "SELECT COUNT(*), COUNT(DISTINCT step_id) FROM ledger WHERE run_id='$RUN_ID';"

The invariant is simple enough to check by eye: the two counts must match, and the total must equal the number of steps in the plan. A submission that prints 3|3 after a kill at the midpoint has demonstrated something a code review cannot, while a submission that prints 4|3 has demonstrated the exact bug the task exists to find. Timing sensitivity means the sleep value should be tuned by the candidate and documented, since a kill that lands before any work starts proves nothing.

Common failure modes

Six patterns account for most weak submissions, and each maps to a rubric row rather than a taste judgement.

1. SQLite on the container filesystem. The schema is correct and the constraint is correct, but the file disappears on redeploy, so the resume path never fires in practice.
2. Idempotency derived from the clock. A key built from time.time() is unique on every attempt, which converts a retry into a second charge.
3. Checkpointing after the whole plan. The run is atomic and therefore restart-safe in theory, but any interruption discards work and the reviewer sees a timeout instead of progress.
4. Resume by re-running everything. Correctness is preserved only if every tool is idempotent, which is asserted rather than enforced in storage.
5. Evidence by assertion. The README states the service "handles restarts" with no command, no log, and no invariant to check.
6. Cowboy deployment. The demo runs on a laptop with an ngrok tunnel and the free-hosting step is described as future work.

The sixth failure is worth calling out separately in feedback, because it usually reflects time pressure rather than misunderstanding. Reviewers who want to grade deployment skill should say explicitly that an unreachable demo scores zero on that dimension, so the expectation is not a surprise.

Where free model access and a free server change the exercise

MonkeyCode, an open-source AI coding project, advertises free model access together with a free server option; the project's materials describe a free token allowance, and the current figure should be confirmed on its official site, since published limits change. For this take-home, the relevant effect is not model quality but the removal of two excuses that distort grading. Candidates no longer need a paid API key to produce a working agent, and they no longer need a credit card to put it somewhere a reviewer can reach.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That changes the interviewer's job in a specific way. Because deployment is now cheap, "I could not host it" stops being an acceptable reason to skip the restart test, and the rubric can reasonably require evidence from a live instance. The corollary is that the rubric must not encode any particular host's limits, because a free tier that suspends after an idle period today may behave differently next quarter. Grade the recovery design, not the platform.

If a team wants to try this loop on real candidates, the sane first step is to run the prompt against a known-good reference implementation before writing the scoring sheet, so the rubric reflects observed behaviour rather than hoped-for behaviour.

Limits and who should not use this format

• Long-running or GPU-bound work is a poor fit. A task whose unit of progress is a multi-minute job will fight the restart cadence of most free hosting options and produce noise instead of signal.
• Model-quality evaluation is out of scope. This take-home grades durability and honesty, not prompt craft, retrieval quality, or output elegance.
• Teams without a stable reference implementation should not use it. If the interviewers cannot produce a passing solution themselves, the rubric will drift toward whatever the first candidate happened to submit.
• Candidates new to backend work need a hint. Stating the three assumptions in the prompt keeps the exercise about engineering judgement rather than guessing the interviewer's hidden agenda.

A take-home is a compressed model of the job, and most real agent work fails at boundaries rather than at prompts. Grading one restart, one invariant, and one honest README tells an interviewer more about how a candidate will behave in production than a polished demo ever will.