Is the Free Box Staging? A Developer FAQ

You finally wired a free model endpoint. Then you got a remote box for free. Did that pairing secretly become staging? I keep hearing that claim in review threads. It sounds thrifty. It is usually wrong.

This is a FAQ, not a launch post. I am writing it because the combo keeps failing in the same places. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode only as one place that offers free model access and a free server option. The checks below still apply if you delete that name.

Why this FAQ exists

Free tokens and free compute are different budgets. Mixing them hides cost until a retry storm. Your laptop is not a quota. A shared box is not a freeze file. Want a cleaner mental model? Read the five questions, then run the smoke job.

I labeled every script as a proposal. I did not attach fake timings. I did not name models. I did not invent quotas. If a claim needs a vendor number, I left it out.

Q1: Does free compute make the job reproducible?

The claim. "It ran on the free box, so we can rerun it." People treat the server like a checked-in artifact. They skip hashes. They skip lockfiles. They skip the prompt pack digest.

What actually breaks. The box image drifts. The client library drifts. The system prompt in a dashboard drifts. Your job exits zero. The text is not the same job.

Corrected model. Reproducibility is a freeze, not a hostname. Capture the prompt pack hash. Capture the client version. Capture the endpoint identifier as a string you control. Capture the working directory digest. The server is only the place the freeze ran.

# proposed freeze, unexecuted on your machine
python3 - <<'PY'
from hashlib import sha256
from pathlib import Path
p = Path("prompts")
digest = sha256()
for f in sorted(p.glob("*.txt")):
    digest.update(f.name.encode())
    digest.update(b"\0")
    digest.update(f.read_bytes())
print(digest.hexdigest())
PY

Ask yourself one question. Could a stranger replay this without Slack? If not, it was not a freeze.

Q2: Is a free server just a bigger localhost?

The claim. "Same Python, same script, same result." Developers scp a notebook and call it parity. They ignore clock skew. They ignore DNS. They ignore outbound allowlists.

What actually breaks. Localhost talks to your VPN. The box talks to a public resolver. Timeouts fire in different layers. Your local run hits the model. The box hits a proxy page. Both look like HTTP 200 if you only print ok.

Corrected model. Treat local and remote as two environments. Compare artifacts, not vibes. Store status code, content type, and a short body hash. Store whether the JSON parsed. Do not store secrets in those artifacts.

# proposed probe, label the env in the filename
curl -sS -D - -o /tmp/body.bin \
  -H "Authorization: Bearer $MODEL_TOKEN" \
  -H "Content-Type: application/json" \
  --max-time 30 \
  "$MODEL_URL" \
  --data @pack/one.json | tee "artifacts/${ENV:-local}-headers.txt"
sha256sum /tmp/body.bin > "artifacts/${ENV:-local}-body.sha256"

Did the remote header file mention a different server name? Good. That is evidence. Do not paper over it with a retry loop.

Q3: If tokens are free, do retries stay free?

The claim. "We can hammer it. It costs nothing." This one shows up in agent loops. A tool fails. The agent retries. The box stays up. Nobody counts calls.

What actually breaks. Free still has a ceiling you do not control. Rate limits, queueing, and abuse trips still exist. Your job looks hung. It is waiting. Logs fill the disk. The next experiment starves.

Corrected model. Budget calls like money even when the invoice is zero. Give every job a hard cap. Give every tool a retry budget of one or two. Fail loud. Write the cap into the report.

# proposed call budget, not a production client
from dataclasses import dataclass

@dataclass
class Budget:
    max_calls: int
    used: int = 0

    def charge(self) -> None:
        if self.used >= self.max_calls:
            raise RuntimeError("call budget exhausted")
        self.used += 1

Would you ship this cap if the model were billed? Then ship it now. Free is not a reason to delete governors.

Q4: Can experiments share the free box without isolation?

The claim. "It is just my playground." Two branches land on the same home directory. Two .env files overwrite each other. One cron job kills the other process.

What actually breaks. Port clashes. Cache directories. Hugging the same filename latest.json. You debug the wrong run for an hour. Then you blame the model.

Corrected model. One job, one workdir, one env file, one artifact dir. Name them after a git sha and a clock you control. Never use latest as a gate.

# proposed layout
JOB_ID="$(git rev-parse --short HEAD)-$(date -u +%Y%m%dT%H%M%SZ)"
ROOT="$HOME/jobs/$JOB_ID"
mkdir -p "$ROOT/artifacts" "$ROOT/prompts"
cp .env.example "$ROOT/.env"
# edit $ROOT/.env; do not reuse $HOME/.env

Is latest still in your path? Delete the symlink. Read a dated directory instead.

Q5: Does a green remote run mean you have staging?

The claim. "The free server passed, promote it." This is the quiet one. It borrows language from CI. It skips the contract you actually need.

What actually breaks. Staging means traffic shape, secrets policy, rollback, and owners. A free box usually has none of those. A green smoke job means the pipe is not on fire. It does not mean customers can enter.

Corrected model. Split three layers. Smoke means the client, network, and schema parser moved. Eval means a frozen pack scored against a rubric you wrote. Staging means production constraints without production blast radius. The free box can host smoke. It should not wear the staging badge.

Ask the promotion question out loud. Who gets paged if this hostname dies? If the answer is nobody, it is not staging.

A proposed remote smoke job

Here is the artifact. It is a small harness. It does not grade prose quality. It answers one thing. Did local and remote fail the same mechanical checks?

Create three files. Keep them tiny on purpose.

pack/cases.jsonl holds one object per line. Each line needs id, prompt, and must_include as a string list. No giant corpus. No leaderboard.

{"id":"t1","prompt":"Reply with the word pong only.","must_include":["pong"]}
{"id":"t2","prompt":"Return JSON with key ok set to true.","must_include":["ok"]}

smoke_job.py is the runner. Treat it as a proposal. Plug in your own HTTP client. Do not paste tokens into the script.

#!/usr/bin/env python3
"""Proposed smoke job. Unexecuted example. Not a benchmark."""
import hashlib, json, os, sys, time, urllib.request

BUDGET = int(os.environ.get("SMOKE_BUDGET", "4"))
URL = os.environ["MODEL_URL"]
TOKEN = os.environ["MODEL_TOKEN"]
ENV = os.environ.get("ENV", "local")

def post(prompt: str) -> tuple[int, bytes, float]:
    body = json.dumps({"prompt": prompt}).encode()
    req = urllib.request.Request(
        URL, data=body, method="POST",
        headers={
            "Authorization": f"Bearer {TOKEN}",
            "Content-Type": "application/json",
        },
    )
    t0 = time.monotonic()
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            raw = resp.read()
            code = resp.status
    except Exception as exc:
        raw = str(exc).encode()
        code = 0
    dt = time.monotonic() - t0
    return code, raw, dt

def main() -> int:
    used = 0
    rows = []
    with open("pack/cases.jsonl") as fh:
        cases = [json.loads(line) for line in fh if line.strip()]
    for case in cases:
        if used >= BUDGET:
            rows.append({"id": case["id"], "error": "budget"})
            continue
        used += 1
        code, raw, dt = post(case["prompt"])
        text = raw.decode("utf-8", "replace")
        missing = [s for s in case["must_include"] if s not in text.lower()]
        rows.append({
            "id": case["id"],
            "http": code,
            "ms_bucket": int(dt * 1000) // 250 * 250,
            "body_sha": hashlib.sha256(raw).hexdigest()[:12],
            "missing": missing,
        })
    out = f"artifacts/{ENV}-smoke.json"
    os.makedirs("artifacts", exist_ok=True)
    with open(out, "w") as fh:
        json.dump({"env": ENV, "used": used, "rows": rows}, fh, indent=2)
    print(out)
    bad = [r for r in rows if r.get("http") != 200 or r.get("missing")]
    return 1 if bad else 0

if __name__ == "__main__":
    sys.exit(main())

Run it twice. Once at home. Once on the free box.

export MODEL_URL TOKEN  # from a local secret manager, never from chat
ENV=local  python3 smoke_job.py
# on the remote box, after syncing the same git sha
ENV=remote python3 smoke_job.py
python3 - <<'PY'
import json
local = json.load(open("artifacts/local-smoke.json"))
remote = json.load(open("artifacts/remote-smoke.json"))
print("used", local["used"], remote["used"])
for a, b in zip(local["rows"], remote["rows"]):
    print(a["id"], "http", a.get("http"), b.get("http"),
          "missing", a.get("missing"), b.get("missing"))
PY

Read the diff as a checklist. Matching HTTP and matching missing is a smoke pass. Matching body hashes is extra. Divergent hashes with the same missing list still need a human. Do not auto-promote on hash equality. Models can drift inside a passing string check.

Decision table

Use this table before anyone says "ship it."

• Claim: hostname equality means job equality. Test: compare freeze hashes. Pass: same pack, client, endpoint id. Fail: any field blank.
• Claim: local green predicts remote green. Test: run smoke_job.py in both envs. Pass: same HTTP class and same missing. Fail: one env budget-trips.
• Claim: retries are harmless. Test: log used versus SMOKE_BUDGET. Pass: used stays under cap. Fail: silent extra calls.
• Claim: shared disk is fine. Test: job path contains git sha plus UTC. Pass: no latest symlink. Fail: two jobs write one file.
• Claim: remote green is staging. Test: name an on-call owner and a rollback. Pass: both exist in writing. Fail: the box is a hobby login.

Print the table in the PR. If a row is fail, the merge is not a model problem. It is an environment problem.

What this does not prove

This workflow does not measure quality. It does not rank models. It does not replace evals. It does not prove latency. Buckets of 250 ms are only for spotting timeouts, not for p50 bragging. It does not prove the free server will exist next month. I am not claiming permanence, hardware, or a quota.

It also does not prove security. A free box still holds tokens. Rotate them. Do not scp .env through chat logs. Do not reuse the same token in a demo and a job.

Who should skip this

Skip it if you already have a real staging cluster. Skip it if legal blocks any remote vendor box. Skip it if your prompts are regulated data. Skip it if you cannot write a freeze file. Skip it if the team wants a leaderboard. This harness will disappoint that team on purpose.

Also skip it if you need GPU proof. I did not describe GPUs. I cannot. A smoke job that posts JSON does not certify accelerators.

The mental model I want you to keep

Free models buy you attempts. Free servers buy you a second place to attempt. Neither buys you a freeze. Neither buys you isolation. Neither buys you staging. Put the governors in first. Then enjoy the price.

If you already have both pieces, run the two-env smoke job once. Compare the JSON. Then decide whether the free box is a lab bench or a story you are telling yourself.