Stopwatch the Cold Start, Not the Model

The first conclusion I want to hand you is uncomfortable: in a brand-new AI-assisted workspace, the model is almost never your bottleneck. The cold start is, and it stays invisible because nobody writes timestamps down. Ask ten developers where their first fifteen minutes went and nine will blame the model, which is a polite way of saying they never measured. I now treat onboarding as a latency problem with a p99 budget, and the change that moved the number most was not a smarter model at all.

So let me show you the instrument first and the opinions second, because an argument about free tiers without a stopwatch is just vibes with a changelog attached. The script below is deliberately boring: it records a millisecond timestamp before and after each step you already run, then prints the deltas. Nothing in it is specific to any vendor, which is exactly why it survives the next time you switch tools.

#!/usr/bin/env bash
# scripts/cold-start.sh — timestamp every step of a fresh workspace bootstrap.
set -euo pipefail

LOG="${LOG:-cold-start.tsv}"
: > "$LOG"

now_ms() { python3 -c 'import time; print(int(time.time() * 1000))'; }
mark()   { printf '%s\t%s\n' "$(now_ms)" "$1" >> "$LOG"; }

mark "boot"

mark "install:start";  ./scripts/install-toolchain.sh;      mark "install:end"
mark "deps:start";     npm ci --no-audit --no-fund;          mark "deps:end"
mark "auth:start";     ./scripts/check-creds.sh;             mark "auth:end"
mark "model:start";    ./scripts/smoke-task.sh > /tmp/smoke.patch; mark "model:end"

mark "apply:start"
git apply --check /tmp/smoke.patch && git apply /tmp/smoke.patch
mark "apply:end"

The report is six lines of awk, and I keep it in the repo next to the script so the numbers are comparable across weeks. Reading a table beats remembering a feeling, especially when the feeling is "it seemed slow".

awk -F'\t' '
  NR==1 { start=$1; prev=$1; printf "%9s %9s  %s\n", "delta_ms", "total_ms", "step"; next }
  { printf "%9d %9d  %s\n", $1-prev, $1-start, $2; prev=$1 }
' cold-start.tsv

Now run it three times, with the container thrown away between runs, because a warm cache will happily lie to you. I use a throwaway image precisely so that "it works on my machine" cannot creep into the measurement.

for i in 1 2 3; do
  docker run --rm -v "$PWD:/w" -w /w node:22-slim ./scripts/cold-start.sh
  mv cold-start.tsv "cold-$i.tsv"
 done

The variance between those three runs matters more than the absolute number, and I learned that the hard way. If run one and run three differ by more than about thirty percent on the same step, you are not measuring your setup, you are measuring the network. That distinction changes what you fix, so please do not skip the repetition.

Here is how I bucket the deltas, and these thresholds are opinions you should tune rather than laws. The point of a table like this is to stop the argument before it starts, because "slow" is not actionable and "install took 412 seconds" is.

Bucket	Typical symptom	Act when	First move
install	toolchain download, version drift	over ~4 min	pin versions, cache the layer
deps	npm ci re-resolving a lockfile	over ~2 min	commit the lockfile, use ci
auth	silent 401 retries, wrong profile	any failure	fail loudly, never retry blind
model	first round trip, no output	over ~60 s	send a trivial prompt, not the real task
apply	patch rejected, whitespace noise	any failure	shrink the smoke task to one file

The one fix that mattered for me was not faster inference; it was turning the cold start into a single warm step that either succeeds or fails in under a minute. I put a warm target in the Makefile, and now every new machine, container, or teammate runs exactly one command before touching the real work.

warm:
    ./scripts/install-toolchain.sh --pinned
    npm ci --no-audit --no-fund
    ./scripts/smoke-task.sh > /dev/null   # cheap proof that creds and endpoint answer
    ./scripts/cold-start.sh --report      # keeps the number honest over time

Why does a smoke task beat a real task as the first move? Because a real task mixes three unknowns at once — your prompt, the endpoint, and the repository state — and you cannot debug three unknowns together. A tiny task that touches one file separates them, and it costs almost nothing when the environment is already broken. I have wasted entire afternoons on prompts that were fine while the credentials were quietly stale, and I would like you to skip that afternoon.

When the smoke patch never lands, the debugging order is fixed and short in my notes. First check that the endpoint answers a trivial request without touching the repo, then check that the repo is clean enough to accept a patch, then check the diff size. Ninety percent of the time the failure is in step one or two and the prompt was never the problem at all.

This is the workflow where MonkeyCode's free model access and free server option are genuinely relevant, and I want to be precise about what that means. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes the project as open source with a free tier that includes model access and a server option, and the free allowance has been quoted to me as ten million tokens; quotas move, so verify the current figure on the project's own page before you plan around it. What I actually used it for is unglamorous and that is the point: I ran the timed loop above against a hosted workspace so the install and dependency buckets were not my laptop's problem. Because the source is public, you can read what the server does with your prompt before you send anything you would not paste into a ticket.

Who should not use this approach? Anyone who needs a contractual SLA, anyone whose cold start is dominated by a proprietary build cache the free tier cannot see, and anyone working in a regulated environment where the answer to "where did the prompt go" must be a document rather than a repository. A free tier is a fine place to measure friction and a poor place to promise uptime, and confusing those two is how teams end up with a very confident incident report.

So before you benchmark another model, time your own first fifteen minutes and write the numbers into a file that other people can read. My guess is that your biggest bucket is not the one you have been blaming, and you will only find out if something is counting. If you want a place to run the stopwatch without provisioning anything, the free workspace is a reasonable first target — just bring the timer with you.