Localhost Is Not a Server: A 3-Stage Probe for Free Model Endpoints

Your free model endpoint passes on localhost. It fails on a server. The model did not change. The network did.

This article builds a 3-stage probe. It measures reachability from your machine. Then it measures the same endpoint from a free server. The output is a decision table. You can run every step in under an hour.

Why localhost differs from a server

Free model endpoints sit behind shared infrastructure. Their behavior depends on where requests come from. Four things change when you deploy to a server.

1. Egress IP. Rate limits often key by IP. Your server shares its IP with other tenants.
2. DNS and TLS. Server resolvers differ. Some endpoints block datacenter IP ranges.
3. Latency budget. A 15-second timeout on your laptop may be a 3-second timeout in your server client.
4. Firewall rules. Free servers sometimes throttle outbound traffic. HTTP/2 support varies.

None of these appear in local tests. All of them appear in production.

Stage 1: Probe from localhost

Save this script as probe.py. It uses only the Python standard library.

#!/usr/bin/env python3
"""probe.py — measure reachability of a free model endpoint."""
import json
import sys
import time
import urllib.request
import urllib.error

URL = sys.argv[1] if len(sys.argv) > 1 else "https://api.example.com/v1/chat"
PROMPT = {"prompt": "Reply with the single word: pong", "max_tokens": 8}

def call_once():
    start = time.perf_counter()
    req = urllib.request.Request(
        URL,
        data=json.dumps(PROMPT).encode(),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urllib.request.urlopen(req, timeout=15) as resp:
            body = resp.read()
            return {
                "ok": True,
                "status": resp.status,
                "ms": round((time.perf_counter() - start) * 1000),
                "bytes": len(body),
            }
    except urllib.error.HTTPError as e:
        return {"ok": False, "status": e.code, "ms": round((time.perf_counter() - start) * 1000), "bytes": 0}
    except Exception as e:
        return {"ok": False, "status": "ERR", "ms": round((time.perf_counter() - start) * 1000), "error": str(e)}

def main():
    n = int(sys.argv[2]) if len(sys.argv) > 2 else 10
    results = [call_once() for _ in range(n)]
    ok = sum(1 for r in results if r["ok"])
    latencies = [r["ms"] for r in results if r["ok"]]
    print(json.dumps({
        "total": n,
        "ok": ok,
        "fail": n - ok,
        "p50_ms": sorted(latencies)[len(latencies) // 2] if latencies else None,
        "max_ms": max(latencies) if latencies else None,
        "results": results,
    }, indent=2))

if __name__ == "__main__":
    main()

Run it ten times from your machine:

python3 probe.py https://your-free-endpoint.example 10

A clean local baseline looks like this:

{
  "total": 10,
  "ok": 10,
  "fail": 0,
  "p50_ms": 812,
  "max_ms": 1430
}

Verify Stage 1. Check the ok and fail counts. Record p50_ms. Save the output. This is your local baseline. Do not trust a single call. Ten calls give you a median, not a mood.

Stage 2: Probe from a free server

Now deploy the same script to a server. Any free server works. A free-tier VM, a container host, or a free server option from a provider you already use. MonkeyCode offers free model access and a free server option. Both fit this workflow.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Copy the script. Run the identical command:

scp probe.py user@your-server:~
ssh user@your-server "python3 probe.py https://your-free-endpoint.example 10"

A failing server run looks like this:

{
  "total": 10,
  "ok": 2,
  "fail": 8,
  "p50_ms": null,
  "max_ms": null
}

Verify Stage 2. Compare the two JSON outputs. Look for three signals.

• Success rate dropped.
• p50_ms jumped.
• Error type changed, like timeout instead of HTTP 429.

Any one of these means the endpoint behaves differently from your server. The endpoint did not break. The network path did.

Reading the three signals

Use this table to name the failure mode.

Signal	Local	Server	Likely cause
Success rate	10/10	2/10	Egress blocked or IP rate limit
p50_ms	812	4100	Slow network path
Error type	none	timeout	Firewall or TLS issue

Each cause has a different fix. IP rate limits need a different endpoint or a retry. Slow paths need a closer server. Firewall issues need a different provider.

Stage 3: Schedule and decide

One run is a snapshot. A cron job is a trend. Add a fifteen-minute schedule.

crontab -e
*/15 * * * * cd ~ && python3 probe.py https://your-free-endpoint.example 10 >> probe.log 2>&1

Wait one day. Then apply this decision table.

Server failure rate	Latency vs local	Action
0%	under 2x	Route traffic directly
under 20%	under 3x	Add one retry with backoff
over 20%	over 3x	Use a different endpoint or a paid API

The table converts observations into actions. It is a rule, not a vibe.

A common failure signature

Here is a pattern you will see often. Localhost: 10/10 success, 900 ms median. Server: 2/10 success, connection resets. The endpoint returns 200 locally. The server cannot complete the request.

The cause is usually egress filtering or IP reputation. The fix is not a better prompt. The fix is a different network path or a different endpoint.

Limitations

This probe measures reachability. It does not measure model quality. A reachable endpoint can still return garbage.

Free endpoints change without notice. Your results expire. Re-run the probe after any provider announcement.

Sample sizes are small. Ten calls catch hard failures. They do not prove 99.9% availability.

The probe uses one tiny prompt. Some endpoints route differently by payload size. Test with your real payload too.

Who should not use this approach

• Teams with contractual uptime requirements. Use a paid API with an SLA.
• Anyone who cannot run Python on the target server. The probe needs a runtime.
• Workloads restricted to your local network. The server will never pass.

Next step

Run Stage 1 today. It takes five minutes. If your endpoint fails Stage 2, you just avoided a production incident.