Is the Endpoint Free, or Just Unbilled?

Is the endpoint free, or just unbilled?

Those are different questions. Mix them and agents get brittle.

I keep hearing the same five claims. They sound calm in a design doc. They fail in boring, repeatable ways.

This is a FAQ. Not a glossary. Not a bake-off.

You get a probe and a decision table. Treat both as methods. I am not reporting your numbers.

Why this FAQ exists

Free inference changes incentives. Fast.

When the invoice is zero, people skip probes. Why wouldn't they? One chat reply looked fine.

That is the trap. Shared boxes have neighbors. Free models still cut connections. Your loop still needs a budget.

Chasing a list of agent nouns? Skip this piece. You already have enough vocabulary.

You need a mental model for a noisy dependency.

Where a free stack actually helps

I use free model access while prompts are still wet. I use a free server option when a laptop tab is not enough.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. That is the only product claim I will make. No model names. No quota theater. No forever promises.

Strip that paragraph. The FAQ still stands.

The five claims I keep hearing

1. "If it is free, I have the box to myself."

Who sold you that mutex?

A price of zero is a billing flag. It is not exclusive hardware.

What to collect, not feel:

• Overlap two in-flight requests on purpose.
• Record start and end for each call.
• Split 429, 503, timeout, and reset. Never blend them.
• Repeat once quiet, once busy.

Corrected model: free is not a lock. You share CPU. You share caches. You share niceness.

Exclusive silicon is a purchase. This is not that purchase.

2. "One green reply means the runtime is pinned."

Pinned to what, exactly?

Did you hash the image? Did you record the model identifier? Did you freeze decoding settings? A polite greeting is not a lockfile.

Client-side snapshot (example, not a supply-chain proof):

# example / unexecuted: capture what the client can see
curl -sS -D - --max-time 10 "$ENDPOINT/health" -o /tmp/health.body
sha256sum /tmp/health.body
echo "utc=$(date -u +%FT%TZ)"

Log the headers you got. Log the body hash. Do not pretend that equals a pinned image.

Corrected model: pin what you control. Log what you cannot pin. Never confuse a hello with a version.

3. "Timeouts are a paid-API problem."

Why would a free endpoint wait forever?

It will not. Proxies shed load. Idle cuts fire. Your agent tick still advances.

People skip timeouts because the call feels cheap. Cheap is not infinite.

What to wire before you trust a loop:

• Set a client timeout you can defend.
• Fail closed when it fires.
• Do not retry in a tight spin.
• Treat timeout as its own class, not "error."

Corrected model: timeout is part of the contract. Price does not write that contract.

4. "Retries are free, so spray them."

Retries cost wall time. They cost neighbor goodwill. They turn a blip into a stampede.

Is your backoff real code? Or is it hope?

Minimum retry hygiene:

• Cap attempts. Hard.
• Add jitter. Always.
• Budget max wait per user action.
• Count retries as a metric, not a footnote.

Corrected model: unbilled tokens are not unbilled tail latency. Storms are still storms.

5. "I can just re-run it. The call is free."

Re-runs do not reconstruct the past. Seeds drift. Neighbors change. Tool order changes.

If you skipped a request id, you cannot debug. Free to send is not free to remember.

Log the cheap things that actually help:

• request id, start, end, status class
• byte size, not the secret prompt
• model identifier the server returned, if any
• retry count and timeout flag

Corrected model: observability is the only time machine you get. An invoice of zero does not replace it.

Artifact: a contention probe

This is a method. Not a benchmark. I am not claiming results on your box.

Example below is unexecuted. Point it at an HTTP JSON path you already control. Do not fire it at strangers.

# example / unexecuted: overlapping HTTP probe
# usage: python probe.py https://example.invalid/v1/health
from __future__ import annotations

import json
import sys
import time
import urllib.error
import urllib.request
from concurrent.futures import ThreadPoolExecutor, as_completed

TIMEOUT_S = 8.0
OVERLAP = 4


def classify(exc: BaseException | None, code: int | None) -> str:
    if exc is None and code is not None and 200 <= code < 300:
        return "ok"
    if isinstance(exc, TimeoutError):
        return "timeout"
    if isinstance(exc, urllib.error.HTTPError):
        if exc.code == 429:
            return "429"
        if exc.code >= 500:
            return "5xx"
        return f"http_{exc.code}"
    if isinstance(exc, urllib.error.URLError):
        return "reset_or_dns"
    return "other"


def one_call(url: str) -> dict:
    started = time.perf_counter()
    req = urllib.request.Request(url, method="GET")
    code = None
    err = None
    try:
        with urllib.request.urlopen(req, timeout=TIMEOUT_S) as resp:
            code = resp.status
            resp.read(256)
    except Exception as exc:  # labeled example; tighten in real code
        err = exc
    ended = time.perf_counter()
    return {
        "class": classify(err, code),
        "ms": round((ended - started) * 1000, 1),
        "http": code,
    }


def main() -> None:
    if len(sys.argv) != 2:
        raise SystemExit("usage: python probe.py URL")
    url = sys.argv[1]
    t0 = time.perf_counter()
    rows = []
    with ThreadPoolExecutor(max_workers=OVERLAP) as pool:
        futs = [pool.submit(one_call, url) for _ in range(OVERLAP)]
        for fut in as_completed(futs):
            rows.append(fut.result())
    wall_ms = round((time.perf_counter() - t0) * 1000, 1)
    summary = {
        "overlap": OVERLAP,
        "timeout_s": TIMEOUT_S,
        "wall_ms": wall_ms,
        "classes": {},
        "rows": rows,
    }
    for row in rows:
        summary["classes"][row["class"]] = summary["classes"].get(row["class"], 0) + 1
    print(json.dumps(summary, indent=2))


if __name__ == "__main__":
    main()

How to read the JSON, without inventing a score:

• wall_ms near one timeout, not four sequential timeouts, means overlap happened.
• Mixed classes means you must not average status into one "error rate."
• Any timeout beside an ok means you need a budget, not a mean.
• A quiet-hour run and a busy-hour run are two samples. They are not a capacity plan.

Pair it with a single curl so you can see headers without Python:

# example / unexecuted
curl -sS -o /dev/null -D - --max-time 8 -w 'http=%{http_code} time=%{time_total}\n' "$ENDPOINT/health"

Decision table

Question you can answer this week	Prefer the free shared path	Do not use it as the only path
Can you name a timeout and a retry cap in code?	Yes, then probe it	No, fix the client first
Do you log request id and status class?	Yes, then compare quiet vs busy	No, you cannot debug a re-run
Is a wrong answer recoverable by a human?	Drafting, eval shaping, internal tools	User-facing money, safety, or legal
Do you have a local fixture for shape?	Canary against the free box	The free box is your only gate
Must the model identifier stay pinned for audit?	Log the identifier; keep a fixture	Shared free routes are the wrong control

Read the table left to right. It is a filter. It is not a scorecard.

Limitations

This FAQ does not size hardware. It does not name models. It does not promise uptime.

The probe sees your client, the network, and a status class. It does not see other tenants. It does not prove isolation.

A health path is not your real prompt. A real prompt can be slower. A tool-calling turn can be slower still.

Four overlapping GETs are not a load test. They are a smell check. Do not publish the JSON as a benchmark.

If the server requires auth, you still need a secret store. Free does not mean "put the key in the repo."

Who should not use this approach

Skip the "free box as the runtime" plan if any of these are true:

• You owe a user-facing latency SLO.
• You handle regulated data on that process.
• You need dedicated accelerators, not a neighbor.
• An auditor will ask for a pinned model artifact.
• You cannot fail closed when the probe is messy.

Those teams still can use a free path for drafting. They should not ship on it.

The mental model I actually want

Price is a label. Isolation is a property. Pinning is a file. Timeout is a contract. Logs are a time machine.

Do not let "unbilled" collapse those five words into one vibe.

Run the probe on the box you already have. Keep the fixture. Count the classes. That is the whole method.