The Model Is Not the Machine: Four Free-Tier Myths

Picture a Friday afternoon: an agent writes a plausible fix, a free server runs the test suite, every check is green, and you close the laptop feeling done. On Monday the same commit fails in CI, and the failure is not in the diff at all — it is in the machine that produced the green. That distance between "it worked once on a free tier" and "it will work again" is where most free-tier disappointment lives, and it is measurable the moment you stop treating the tier as one thing.

MonkeyCode is a convenient place to run that measurement because the project offers free model access and a free server option, so both resources can be exercised before anyone signs anything. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The probe below is vendor-neutral on purpose — it records facts about any box and any model endpoint, so you can rerun it against whatever tier you are actually considering.

Myth one says that free model access and a free server are one product. They fail independently: you can have tokens left and no working environment, or a healthy box and a rate-limited model, and the symptom looks identical from the outside. The corrected model is two meters instead of one dial, which means every free-tier verdict needs two separate readings before you write it down. If a run fails, ask which meter moved.

Myth two says a green run proves the runtime matched your laptop. It proves the runtime was green, and nothing else, because locale, timezone, core count, memory ceiling and filesystem type all drift between a laptop and a shared free box. A test that sorts strings, formats dates, or writes to a temp path is exactly the kind of test that passes on one and fails on the other. You cannot argue this away; you can only record it.

Myth three says free only changes speed. In practice free changes failure modes, and failure modes are what your retry logic has to survive: a stream that stops mid-sentence, a cold start that outlives your client timeout, a burst of 429s, or a home directory that forgets yesterday. Slower output is annoying for an hour; an unclassified truncation can cost you a day of debugging the wrong layer. Budget for the shape of the failure, not just the latency.

Myth four says a wrong answer means a bad tier. Most "bad tier" verdicts decompose into four layers — model, transport, environment, task specification — and only one of them is the vendor's problem. If the prompt never pinned a library version, a plausible wrong answer is a specification bug wearing a model costume. Classify first, then judge, otherwise you will churn tiers without ever fixing the thing that broke.

Here is the artifact: a probe that prints the environment facts deciding whether a run transfers. It is POSIX-ish bash, needs no dependencies beyond coreutils and curl, and produces diffable text so you can compare two boxes line by line.

#!/usr/bin/env bash
# tier_probe.sh - print the facts that decide whether a free-tier run transfers.
set -uo pipefail

marker="${PROBE_MARKER:-$HOME/.tier-probe-marker}"

kv() { echo "$1 = $2"; }

kv generated_at "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
kv kernel       "$(uname -sr)"
kv arch         "$(uname -m)"
kv cpus         "$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 1)"
kv memory_kb    "$(awk '/MemTotal/{print $2}' /proc/meminfo 2>/dev/null || echo 0)"
kv timezone     "$(date +%Z)"
kv locale       "${LANG:-unset}"
kv python       "$(python3 -V 2>&1 | head -1)"
kv root_fs      "$(stat -f -c %T / 2>/dev/null || echo unknown)"
kv home_fs      "$(stat -f -c %T "$HOME" 2>/dev/null || echo unknown)"
kv marker_seen  "$(cat "$marker" 2>/dev/null || echo absent)"

Run it once on the free server and once on the machine where the tests passed, then diff the two files; every line that differs is a candidate explanation for a Monday failure.

chmod +x tier_probe.sh
./tier_probe.sh > probe-free-box.txt
# same script, same shell, on your own machine:
./tier_probe.sh > probe-laptop.txt
diff -u probe-laptop.txt probe-free-box.txt

Persistence is the interesting line, so write a marker in session one and read it back in session two, which takes about thirty seconds and settles the ephemeral-disk question without guessing.

date -u +%Y-%m-%dT%H:%M:%SZ > "${PROBE_MARKER:-$HOME/.tier-probe-marker}"
./tier_probe.sh | grep marker_seen
# reconnect later, rerun, compare

Measure transport separately with curl's own timers, because DNS, TLS and total time fail for different reasons and a single "it was slow" number hides which one you hit.

curl -sS -o /dev/null -w 'dns=%{time_namelookup} connect=%{time_connect} tls=%{time_appconnect} total=%{time_total} code=%{http_code}' https://example.com; echo

The model side needs the same discipline, so repeat one small task three times and record exit code, byte count and finish reason rather than reading the prose. The loop below is a template I have not executed for this article — swap the runner for whatever your free model access exposes — and its point is to make variance visible before it becomes a mystery.

# Template only, unexecuted here.
for i in 1 2 3; do
  ./run_task.sh 'Return the SHA-256 of the string monkeycode-probe' > "out-$i.txt" 2> "err-$i.txt"
  echo "run=$i exit=$? bytes=$(wc -c < "out-$i.txt")"
done

Once you have the readings, classify the failure before assigning blame, because the table below is the difference between a five-minute fix and a new vendor.

Symptom	Layer	First check
Output stops mid-sentence	model or stream	finish reason, context limit
Session two cannot see session one files	environment	home_fs, marker_seen
Sorting or date test flips	environment	locale, timezone
Bursts of 429 or 502	transport	retry with backoff, count them
Confident wrong API call	task spec	pin the version in the prompt

Then decide whether the workflow belongs on a free tier at all, since honesty about fit saves more time than any benchmark you could run.

Situation	Free tier fit
Spike, throwaway branch, environment probe	good fit
Uptime commitment or paid SLA	poor fit
Reproducible CI where the image can be pinned	workable, pin everything
Customer data with unclear processing terms	stop and read the terms

The limitations are worth stating plainly. One probe run is a snapshot, not a distribution, so repeat it in a different hour and on a different day before drawing a trend; quotas and model line-ups on free tiers change without notice, which is why this article names none of them and you should re-read the current limits instead of trusting a blog post. The script also assumes a Unix-like shell and degrades silently on systems without stat -f -c or nproc, which is acceptable for a diagnostic and unacceptable for a gate. If you need hard guarantees tomorrow morning, a paid environment with a pinned image is the honest answer, and no probe changes that.

What changes after this exercise is not your opinion of any vendor but the shape of your notes. Instead of "the free server was flaky", you end up with a diff showing four cores instead of eight, an ephemeral home, and a locale mismatch — three facts, each with an obvious fix or a clear reason to move the job elsewhere. That is a far better artefact to bring to a team than a feeling.

If you want a low-stakes way to try the workflow, run the probe on MonkeyCode's free server and push a single small task through its free model access, then compare the two outputs before committing a real project to it.