Four Myths About Free Model Endpoints, and the Four-Check Probe That Broke Them

I run a small digest job. It summarizes a handful of repos every morning. It calls a free model endpoint and runs on a free server. Last month it broke twice.

Not with a crash. It kept returning 200s. The summaries just got quietly worse. So I stopped trusting my own assumptions and wrote a probe instead.

Here is what the probe checks, and the myths it killed.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used its free model access and its free server option to run this workflow. Treat both availability claims as operator-supplied facts, not as a contract or an SLA.

Myth 1: "The free endpoint is one model"

We paste a model name from the docs and assume a fixed target. Why would we doubt it? The string is right there in our config.

But you have not checked what comes back. A routing layer, a cache, or a version bump can sit between you and the weights. I cannot tell you what your endpoint does. I can tell you my assumptions were never evidence.

So log the echo. Every call, every time.

import json, os, sqlite3, time, urllib.request

BASE = os.environ["MONKEYCODE_BASE_URL"].rstrip("/")
KEY  = os.environ["MONKEYCODE_API_KEY"]
WANT = os.environ.get("EXPECTED_MODEL", "")

con = sqlite3.connect("calls.db")
con.execute("CREATE TABLE IF NOT EXISTS calls ("
            "ts REAL, http INT, model TEXT, finish TEXT, ms INT)")

def post(prompt: str):
    body = json.dumps({
        "model": WANT or None,
        "messages": [{"role": "user", "content": prompt}],
    }).encode()
    req = urllib.request.Request(
        BASE + "/chat/completions",
        data=body,
        headers={"Authorization": "Bearer " + KEY,
                 "Content-Type": "application/json"},
    )
    t0 = time.time()
    with urllib.request.urlopen(req, timeout=60) as r:
        payload = json.loads(r.read())
        ms = int((time.time() - t0) * 1000)
        ch = payload["choices"][0]
        con.execute("INSERT INTO calls VALUES (?,?,?,?,?)",
                    (time.time(), r.status, payload.get("model"),
                     ch.get("finish_reason"), ms))
        con.commit()
        return ch

Two caveats. The request shape above assumes an OpenAI-style API. Adapt field names to whatever your account's docs describe. And the model name you send may not be the model you get.

Now query your own log instead of your memory:

SELECT model, finish, COUNT(*) FROM calls GROUP BY 1, 2;

If that table shows one row, good. If it shows three, your "pinned" dependency was never pinned.

Myth 2: "If it runs on the free server, it runs anywhere"

This one bit me hard. A script passed on the free server. It failed in CI forty minutes later. Nothing about the code changed.

The environment did. Different libc, different Python patch level, different timezone. Free servers are fine machines. They are still not your laptop.

So take a fingerprint. It takes ten seconds to write.

import json, platform, subprocess, sys

def run(*a):
    try:
        return subprocess.run(a, capture_output=True, text=True,
                              timeout=5).stdout.strip()
    except Exception:
        return None

flags = None
try:
    line = [l for l in open("/proc/cpuinfo") if l.startswith("flags")][0]
    flags = len(line.split(":", 1)[1].split())
except Exception:
    pass

print(json.dumps({
    "python": sys.version.split()[0],
    "impl": platform.python_implementation(),
    "machine": platform.machine(),
    "libc": platform.libc_ver(),
    "openssl": run("openssl", "version"),
    "git": run("git", "--version"),
    "tz": run("date", "+%Z"),
    "cpu_flag_count": flags,
}, sort_keys=True, indent=2))

Run it on both machines. Then diff the files.

python3 fingerprint.py > fp_free.json   # on the free server
python3 fingerprint.py > fp_ci.json     # on your CI runner
diff -u fp_free.json fp_ci.json || echo "drift found"

Any line in that diff is a future bug. Pin the ones you care about in your build, not in your head.

Myth 3: "Valid JSON means a complete answer"

I hear this one a lot. The parser accepted it, so the job passed. Right?

No. json.loads tells you the bytes parse. It says nothing about truncation. A cut-off response can still close its braces if the model got lucky with structure.

So check two things, not one: the finish signal, and the schema.

import json, sys

def strict(content: str, finish: str):
    if finish != "stop":
        sys.exit("incomplete: finish_reason=" + str(finish))
    data = json.loads(content)
    if not isinstance(data.get("items"), list) or not data["items"]:
        sys.exit("schema: items missing or empty")
    return data

Fail closed here. Do not paper over it with a silent retry. A retry spends budget and hides the real problem. Decide your retry policy separately, and write it down.

Myth 4: "One green run is a passing run"

A single success proves one thing. The endpoint worked once. It tells you nothing about tomorrow, or about the next ten calls.

So sample. Five runs is cheap. Five runs already separates "works" from "worked".

for i in 1 2 3 4 5; do
  python3 check.py >> runs.log 2>&1
done
grep -c '^ok' runs.log

Four out of five is not a pass. It is a rate. Write the rate down, then decide if it is acceptable for your job.

The decision table I actually use

Question	Free endpoint + free server	Managed / paid tier
Batch job, forgiving latency?	Good fit	Overkill
Need a signed SLA?	No	Yes
Regulated or sensitive data?	Do not	Maybe, with review
Need a specific model version forever?	Verify by echo, every run	Contract it
Need always-on scheduling?	Test cold starts first	Design for it
Budget is zero and output is low-stakes?	Start here	Later

That last row is the honest one. Free access is great for learning, prototypes, weekend tools, and low-stakes batch work. It is a bad place for anything with a contract attached.

Limitations you should not skip

• I measured my workload, not yours. Your numbers will differ.
• I am not claiming any specific quota, model list, hardware, or uptime figure. Check current docs for those.
• The model echo may be absent or renamed in your response shape. Adapt the parser.
• Fingerprints catch environment drift, not logic bugs. They are one net, not all nets.
• None of this replaces a real test suite. It only catches the assumptions tests miss.

Who should not use this approach

• Teams that need a written uptime commitment. Buy one instead.
• Anyone handling data they cannot send to a third party.
• Workloads needing stable sub-second tail latency without measurement.
• People who will not read the log they just created.

The corrected mental model

A free endpoint is not a dependency. It is a measured resource. You observe it, you record it, and you decide what to do when it drifts.

Ask yourself one question before you ship. If the model string changed tomorrow, would I know?

If the answer is no, you have a probe to write. Run these four checks before you trust the endpoint again. If you try them, tell me which check broke first.