Is a Free Endpoint a Lab? A Runtime FAQ

Can a free endpoint still freeze your night? I keep getting that question in review threads. People still call the box a lab. Then they skip every runtime rule they would keep in prod.

I used to shrug at the same claim. A hung child process taught me the cost. This post is an FAQ, not a tour. Each answer has a claim, a check, and a better model.

You can run every check on a laptop. You can also park it on a spare host. I will not invent quotas, model names, or latency charts.

The shrug I still hear

"It's free. If it burns, we shrug."

That one line smuggles five mistakes at once. Do you want those mistakes named before the FAQ?

• Wall-clock still exists.
• Status codes still bind you.
• A remote kernel still has neighbors.
• Prompt files still hold secrets.
• A killed PID still is not undo.

I walk each mistake as a question. Bring a terminal.

Q1. If the model is free, may I skip timeouts?

Claim: Timeouts only belong to paid production traffic.

What that claim means: The job is a toy, so let it run.

Evidence you can collect tonight:

curl -m 8 -o /tmp/out.json -w "%{http_code} %{time_total}\n" \
  "$MODEL_URL"

Did that curl command stop near eight seconds? Then you have a real control. If your client has no deadline, you do not have a test.

Ask yourself one thing. What holds the file descriptor when the model stalls?

Corrected mental model: Free access changes the invoice, not physics.

A stuck TCP session still occupies a slot. Your runner still has a max job time. I always set the wall-clock budget first. The token counting can come after that.

Q2. Are extra retries just being careful?

Claim: Retry until it answers. The call is free.

What that claim means: A 429 feels like weather.

Evidence:

Classify the stop reason. Do not classify your mood.

# proposed check — label it as unexecuted
echo "$HTTP_CODE" | grep -E '^(429|503|000)$'

000 from curl often means a timeout. 429 means a stated budget. 503 means upstream pain.

Blind retries amplify load on a shared endpoint. They also poison your own notes. You cannot tell luck from a policy.

Corrected mental model: Retries spend a budget you do not see.

Cap the attempts. Add jitter. Record the class that stopped the run. Thorough is not the same as unbounded.

Would you retry a locked mutex forever? Then why retry a 429 forever?

Q3. Is a free server just localhost with SSH?

Claim: Same Python, same job, same night.

What that claim means: SSH is a long USB cable.

Evidence:

hostname
nproc
df -h /
ulimit -n
ps -o pid,etime,cmd --sort=-etime | head

Read the process list slowly. Who else is here? localhost runs your coffee script. A free server shares a kernel with neighbors.

CPU, disk, and open-file limits are not souvenirs. They are the real machine.

Corrected mental model: Treat the box as a rented kitchen.

Bring a timer. Wipe the counters. Do not assume your laptop's quiet idle.

I still SSH in. I just stop calling it local.

Q4. Can I dump every prompt because disk is cheap?

Claim: Keep full text. Debug needs everything.

What that claim means: Monday-me will want the raw string.

Evidence:

du -sh ./runs
find ./runs -name "*.prompt" | wc -l
ls -ld ./runs

Now answer two questions. Who can read this directory? When do these files die?

Prompts carry names, tokens, and customer wording. Free compute does not waive retention. Cheap disk is not a legal theory.

Corrected mental model: Log a hash plus a redacted stub.

Keep raw text behind an explicit flag. Default that flag to off. Your future incident channel will thank you.

Q5. If I kill the process, did the job undo?

Claim: kill -9 is a rollback button.

What that claim means: No process means no side effects.

Evidence:

Look for partial files and repeat POSTs.

ls -l ./receipts ./runs
# proposed: grep for duplicate request ids

If you have no receipt, you cannot answer. You only have a feeling.

Corrected mental model: Process death is not a transaction.

You need a small receipt with a terminal state. I use started, wall, budget, upstream, and ok. Without that line, retry is a coin flip.

The artifact: a receipt wrapper

This is a proposed local script. I am not reporting a live vendor run.

It enforces four rules only.

1. A wall-clock budget around the child.
2. A hard cap on tries.
3. Isolation via subprocess.
4. A JSON receipt beside the run.

#!/usr/bin/env python3
"""Proposed receipt wrapper. Unexecuted example. Not a benchmark."""
from __future__ import annotations

import json
import os
import subprocess
import sys
import time
import uuid
from pathlib import Path

BUDGET_S = int(os.environ.get("JOB_BUDGET_S", "20"))
MAX_TRIES = int(os.environ.get("JOB_MAX_TRIES", "2"))
RECEIPT_DIR = Path(os.environ.get("JOB_RECEIPT_DIR", "./receipts"))
CMD = os.environ.get("JOB_CMD", 'python3 -c "print(0)"')


def classify(returncode: int, err: str) -> str:
    text = (err or "").lower()
    if returncode in (124, -9):
        return "wall"
    if "429" in text:
        return "budget"
    if "timed out" in text or returncode == 124:
        return "wall"
    if returncode != 0:
        return "upstream"
    return "ok"


def run_once(remain: float) -> tuple[int, str, str]:
    if remain <= 0:
        return 124, "", "budget hit before spawn"
    try:
        proc = subprocess.run(
            CMD,
            shell=True,
            capture_output=True,
            text=True,
            timeout=remain,
        )
        return proc.returncode, proc.stdout, proc.stderr
    except subprocess.TimeoutExpired as exc:
        return 124, exc.stdout or "", "timed out"


def main() -> int:
    RECEIPT_DIR.mkdir(parents=True, exist_ok=True)
    run_id = str(uuid.uuid4())
    path = RECEIPT_DIR / f"{run_id}.json"
    started = time.time()
    receipt = {
        "id": run_id,
        "state": "started",
        "tries": 0,
        "cmd": CMD,
        "budget_s": BUDGET_S,
    }
    path.write_text(json.dumps(receipt, indent=2))

    state = "started"
    for attempt in range(1, MAX_TRIES + 1):
        remain = BUDGET_S - (time.time() - started)
        code, out, err = run_once(remain)
        state = classify(code, err)
        receipt.update(
            {
                "state": state,
                "tries": attempt,
                "returncode": code,
                "stderr_tail": (err or "")[-400:],
                "elapsed_s": round(time.time() - started, 3),
            }
        )
        path.write_text(json.dumps(receipt, indent=2))
        if state == "ok":
            (RECEIPT_DIR / f"{run_id}.out").write_text(out or "")
            return 0
        if state == "budget":
            break
        time.sleep(min(2 ** attempt, 8))

    print(f"receipt={path} state={state}", file=sys.stderr)
    return 1


if __name__ == "__main__":
    raise SystemExit(main())

Save it as receipt_job.py. Then run a dry command first.

export JOB_BUDGET_S=20
export JOB_MAX_TRIES=2
export JOB_CMD='python3 -c "import time; time.sleep(3); print(42)"'
python3 receipt_job.py
cat receipts/*.json

Want a failure class? Force a timeout.

export JOB_BUDGET_S=1
export JOB_CMD='python3 -c "import time; time.sleep(30)"'
python3 receipt_job.py

You should see "state": "wall". If you do not, the wrapper is lying. Fix the wrapper before you blame the model.

How I read a receipt

I do not read chat logs as status. I read the file.

State	Meaning	Next action
started	Child never wrote a terminal line	Do not retry blindly
wall	Budget hit	Shrink work or split the job
budget	429-class stop	Back off. Do not hammer.
upstream	Non-zero, not a budget	One retry, then stop
ok	Child exited zero	Read the .out file

Print the table next to the job in CI. Humans argue. Files do not.

A tiny grep helps during a bad evening.

python3 -c "import json,glob; import pathlib as p;\n[print(json.loads(pathlib:=open(f).read())['state'], f) for f in glob.glob('receipts/*.json')]"

Need that without a one-liner mess? Use a short loop.

for f in receipts/*.json; do
  python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['state'], sys.argv[1])" "$f"
done

If that line is started, you have a crash during spawn. That is a wrapper bug, not a model bug.

Where a free model and a free server fit

I want the hung job off my laptop. Browser tabs already steal the fans.

MonkeyCode's free model access and free server option fit that parking pattern. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I am not claiming a quota, a GPU SKU, or a forever window. I am claiming a place to run the wrapper. Any SSH host with python3 can host the same receipt file.

The product is optional. The receipt is not.

Who should not use this

Skip this approach in a few cases.

• You are sending production user traffic through a free box.
• You need a signed audit log, not a JSON file.
• You need an SLA from the endpoint.
• You cannot set JOB_BUDGET_S because the work is unbounded by design.
• You must keep raw prompts for regulated review, and this default-off log will fight you.

If you need those things, build a real queue. Do not dress a receipt as a platform.

Limitations I will not hand-wave

The wrapper does not talk distributed locks. A second replica can still double-post.

shell=True is convenient and dangerous. Pass a list if your command reads untrusted text.

Timeouts on subprocess.run do not kill grandchildren by default. Long child pipelines still need a process group.

The classifier greps stderr for 429. Some clients hide that number. Then you mislabel upstream.

This FAQ still does not score model quality. A receipt of ok means the child exited. It does not mean the answer was right.

I also do not know your network MTU. I do not know your NAT idle timer. I do not know the vendor's next policy change. Recheck primary docs when you pin a client.

Close

So is a free endpoint really a lab? Only if the lab has a clock, a cap, and a receipt.

What claim are you still repeating in standup? Timeouts are optional? Retries are free? SSH equals local?

Disk equals memory? Kill equals rollback? Pick one claim. Run one check. Write one receipt.

If you already have a spare box, start there. Run receipt_job.py on a job that hung last week. That is the only ask I have.