Cold-Start Audit: What I Measure Before Pointing an Agent at a Free Server

Last Friday I had zero budget and one goal. Ship a small agent by lunch. I had free model access and a free server, both from MonkeyCode.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Lunch came and went. I spent the day debugging an environment I never measured.

This is a letter to setup-me. Three ordering mistakes cost me a full day. None of them were about the model itself.

The three mistakes, in the order I made them

1. I wired the agent loop before probing the endpoint.
2. I treated the free box as idle and predictable.
3. I had no snapshot, so every retry paid full cold-start cost.

Each mistake has a cheap fix. Each fix takes under ten minutes.

Step 1 — Probe the endpoint before you write the loop

A free tier is a shared resource. Latency and failures will vary by hour. You cannot plan around numbers you never collected.

Run 20 sequential calls. Record success rate, p50, max, and error codes.

#!/usr/bin/env python3
# probe.py - Python 3.9+, stdlib only. Unverified template: I have not run this
# against the current endpoint. Set env vars from the project's current docs.
import json, os, statistics, time, urllib.error, urllib.request

BASE_URL = os.environ["MC_BASE_URL"].rstrip("/")   # do not hardcode; URLs move
MODEL = os.environ["MC_MODEL"]                     # the model id you actually have
KEY = os.environ.get("MC_API_KEY", "")
CALLS = int(os.environ.get("CALLS", "20"))

def one_call(prompt):
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": 16,
    }).encode()
    headers = {"Content-Type": "application/json"}
    if KEY:
        headers["Authorization"] = "Bearer " + KEY
    req = urllib.request.Request(BASE_URL + "/chat/completions", data=body, headers=headers)
    start = time.monotonic()
    try:
        with urllib.request.urlopen(req, timeout=90) as r:
            r.read()
        return time.monotonic() - start, True, ""
    except urllib.error.HTTPError as e:
        return time.monotonic() - start, False, "http_" + str(e.code)
    except Exception as e:
        return time.monotonic() - start, False, type(e).__name__

rows = []
for i in range(CALLS):
    dt, ok, err = one_call("reply with the single word: ready")
    rows.append({"i": i, "s": round(dt, 2), "ok": ok, "err": err})
    time.sleep(1)  # stay gentle; respect documented rate limits

good = [r["s"] for r in rows if r["ok"]]
print(json.dumps({
    "summary": {
        "calls": CALLS,
        "ok": len(good),
        "fail": CALLS - len(good),
        "p50_s": round(statistics.median(good), 2) if good else None,
        "max_s": max(good) if good else None,
        "errors": sorted({r["err"] for r in rows if not r["ok"]}),
    },
    "rows": rows,
}, indent=2))

The first call is usually the slowest. Treat call 0 as warm-up and read p50 from the rest.

Read the summary before you read the model output. Three numbers decide the architecture.

• fail > 0: add retries and a dead-letter file before the loop exists.
• max_s far above p50_s: never assume steady latency inside a loop.
• http_429: back off and stop. Do not retry around a rate limit.

That last one matters. Retrying around a 429 is how a free allowance disappears.

Step 2 — Baseline the free server before the first prompt

A free box may share CPU. Fresh disk may be slower than steady-state disk. Measure now, not after a timeout.

#!/usr/bin/env bash
# baseline.sh - Linux sandbox. /proc and /usr/bin/time do not exist on macOS.
set -euo pipefail
stamp="$(date -u +%Y%m%dT%H%M%SZ)"
out="baseline-${stamp}.json"

cpu_s=$( { /usr/bin/time -f '%e' python3 -c 'i=0
for _ in range(3000000): i += 1' ; } 2>&1 | tail -1 )

write_s=$( { /usr/bin/time -f '%e' dd if=/dev/zero of=./.probe bs=1M count=256 conv=fsync ; } 2>&1 | tail -1 )
rm -f ./.probe

cat > "$out" <<JSON
{
  "stamp": "${stamp}",
  "nproc": $(nproc 2>/dev/null || echo null),
  "mem_total_kb": $(awk '/MemTotal/{print $2}' /proc/meminfo 2>/dev/null || echo null),
  "disk_avail_kb": $(df -Pk . | awk 'NR==2{print $4}'),
  "loop_3m_seconds": ${cpu_s},
  "write_256mb_seconds": ${write_s}
}
JSON

jq . "$out" > /dev/null && echo "valid: $out"

Keep the receipt. Re-run the same script at a different hour. A two-second swing on the loop is noise. A tenfold swing means the box is busy.

Compare the receipt, never your memory of it. Three receipts from three hours beat one lucky sample.

Step 3 — Snapshot first, then log every run as one line

The day-killer was retry cost. Every failed run rebuilt its environment from scratch. Free the loop from dependency roulette: build the working state once, snapshot it, reuse it.

Then append one JSONL line per run. The ledger answers "what changed?" without a debugging session.

run_id="$(date -u +%Y%m%dT%H%M%SZ)"
jq -cn \
  --arg id "$run_id" \
  --arg sha "$(git rev-parse --short HEAD 2>/dev/null || echo no-git)" \
  --argjson wall "$SECONDS" \
  --arg status "${1:-ok}" \
  '{run:$id, commit:$sha, wall_s:$wall, status:$status}' >> runs.jsonl

Three fields are enough to start. Add token counts and exit codes when you have them. Do not add free-text notes.

Where the free tier actually fits

Free model access plus a free server is a strong fit for measured, low-volume work. It is a weak fit for anything you cannot observe.

Situation	Fits the free tier?	Reason
Under ~30 calls, human watching	Yes	Cold-start cost is tolerable
Latency-insensitive batch job	Yes	Queue time rarely matters
Teaching, demos, screenshots	Yes	Reproducible and cheap
Long unattended agent loops	No	Failure cost compounds
Hard p95 latency contract	No	Shared capacity varies
Regulated or secret data	No	Verify your own compliance first

This table is a starting position, not a verdict. Measure, then edit it.

Limitations, said plainly

The scripts above are templates. I have not executed them against the current endpoint. The output shape is real; the numbers are yours to collect.

Availability, quotas, and model lists change without notice. Confirm current limits on the project page before you design around them. Do not plan capacity from a blog post, mine included.

The approach also adds friction. Two probes and a ledger cost about fifteen minutes. If your task is a single prompt you read once, skip all of it.

Skip this workflow entirely if you need guaranteed throughput today. Skip it if your data cannot leave your own network, or if you cannot read the language the endpoint errors come back in.

What I would tell setup-me

Probe before you wire. Baseline before you prompt. Snapshot before you retry.

A day of debugging bought three small scripts. The scripts take ten minutes to run, and they still pay for themselves.

If you want to try the audit without paying for the first step, the free model access and free server option lower the entry cost. Start with probe.py, and read the summary before you read the model.