Free Model Endpoints Fail Quietly: A Self-Hosted Health Log in 40 Lines

Free model endpoints fail quietly. No alert. No page. Just a slow trickle of errors. You notice after users complain. This article builds a 40-line health log. It runs on a cron. It stores every check in SQLite. It outputs a status page. Zero third-party dependencies.

Provider status pages are too coarse. They report region-level incidents. Your API key may fail while their page shows green. Your request path may hit a different backend. Your latency may degrade without a full outage. A local probe catches what the status page misses.

Why your own health log beats a provider status page

A provider status page answers one question: "Is the service down?" Your real question is: "Is the service down for me?" The two diverge often. Free endpoints are shared. Neighbors can exhaust quotas. Rate limits can hit your key while others work. A health log gives you evidence. It records what you actually observed.

The probe is a tiny request. It costs a fraction of a token. It runs every five minutes. That is 288 requests per day. Even a strict free tier can absorb that. The value is the history. After two weeks, you have a baseline. You can answer "when did this start?" without guessing.

Stage 1: Send a minimal probe

The probe is a POST with a one-word prompt. Keep max_tokens at 1. You only need a status code and a latency measurement. Do not use HEAD. Most model endpoints do not implement it.

import json
import time
import urllib.request
import urllib.error

ENDPOINT = "https://your-endpoint.example/v1/chat/completions"
API_KEY = "your-key"  # read from env in production
PROMPT = "ping"

def probe():
    payload = json.dumps({
        "messages": [{"role": "user", "content": PROMPT}],
        "max_tokens": 1,
    }).encode()
    req = urllib.request.Request(ENDPOINT, data=payload, headers={
        "Content-Type": "application/json",
        "Authorization": f"Bearer {API_KEY}",
    })
    start = time.monotonic()
    try:
        with urllib.request.urlopen(req, timeout=10) as resp:
            body = resp.read()
            latency_ms = (time.monotonic() - start) * 1000
            return resp.status, latency_ms, len(body), None
    except urllib.error.HTTPError as err:
        latency_ms = (time.monotonic() - start) * 1000
        return err.code, latency_ms, 0, None
    except Exception as exc:
        latency_ms = (time.monotonic() - start) * 1000
        return 0, latency_ms, 0, str(exc)

time.monotonic() measures wall time correctly. time.time() can jump. The timeout prevents a hung request from blocking the cron job.

Verify the probe:

python - <<'PY'
from healthcheck import probe
status, latency, size, error = probe()
print(f"status={status} latency={latency:.0f}ms size={size} error={error}")
PY

You should see a status code and a latency. If you see 0, the endpoint is unreachable. That is a valid data point.

Stage 2: Store every check in SQLite

SQLite is the right tool here. It is in the standard library. It handles concurrent reads and writes. It survives crashes. A CSV file would corrupt under cron races. A JSON file would need manual locking.

import sqlite3
from pathlib import Path

DB_PATH = Path(__file__).parent / "health.db"

def init_db():
    with sqlite3.connect(DB_PATH) as conn:
        conn.execute("""
            CREATE TABLE IF NOT EXISTS checks (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                ts TEXT NOT NULL,
                status_code INTEGER NOT NULL,
                latency_ms REAL NOT NULL,
                response_bytes INTEGER NOT NULL,
                error TEXT
            )
        """)

def record(status, latency, size, error):
    with sqlite3.connect(DB_PATH) as conn:
        conn.execute(
            "INSERT INTO checks (ts, status_code, latency_ms, response_bytes, error) VALUES (?, ?, ?, ?, ?)",
            (time.strftime("%Y-%m-%dT%H:%M:%S%z"), status, latency, size, error)
        )

The ts column uses ISO 8601. That sorts lexicographically. You can filter with string comparisons. No date parsing needed.

Verify the storage:

python - <<'PY'
from healthcheck import init_db, record
init_db()
record(200, 123.4, 15, None)
record(429, 5.2, 0, None)
print("records inserted")
PY

Then inspect:

sqlite3 health.db "select * from checks;"

You should see two rows.

Stage 3: Render a 24-hour report

The report turns raw rows into decisions. You need three numbers: success rate, average latency, p95 latency. Success rate tells you stability. Average latency tells you the trend. P95 latency tells you the worst normal case.

import statistics

def generate_report():
    with sqlite3.connect(DB_PATH) as conn:
        rows = conn.execute("""
            SELECT ts, status_code, latency_ms, response_bytes, error
            FROM checks
            WHERE ts >= datetime('now', '-24 hours')
            ORDER BY ts
        """).fetchall()
    if not rows:
        return "<p>No data in the last 24 hours.</p>"
    total = len(rows)
    ok = sum(1 for r in rows if r[1] == 200)
    success_rate = ok / total * 100
    latencies = [r[2] for r in rows if r[2] > 0]
    avg_latency = statistics.mean(latencies) if latencies else 0
    sorted_lat = sorted(latencies)
    p95 = sorted_lat[int(len(sorted_lat) * 0.95) - 1] if sorted_lat else 0
    rows_html = "".join(
        f"<tr><td>{r[0]}</td><td>{r[1]}</td><td>{r[2]:.0f}</td><td>{r[3]}</td><td>{r[4] or ''}</td></tr>"
        for r in rows[-20:]
    )
    return f"""
    <html><body>
    <h1>Model Endpoint Health</h1>
    <p>Success rate: {success_rate:.1f}%</p>
    <p>Avg latency: {avg_latency:.0f}ms</p>
    <p>P95 latency: {p95:.0f}ms</p>
    <table border="1">
    <tr><th>Time</th><th>Status</th><th>Latency</th><th>Bytes</th><th>Error</th></tr>
    {rows_html}
    </table></body></html>
    """

The report shows the last 20 checks. Enough for a quick glance. The summary covers the full 24 hours.

Verify the report:

python - <<'PY'
from healthcheck import generate_report
html = generate_report()
print(html[:200])
PY

You should see the success rate line.

Stage 4: Deploy on a free server with cron

A health log only helps if it runs continuously. Your laptop is not enough. You need an always-on machine. MonkeyCode's free server option can host this script. It has no OS-specific dependencies. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Create a wrapper script:

#!/bin/sh
cd /path/to/healthcheck
python3 healthcheck.py

Make it executable:

chmod +x run_healthcheck.sh

Add a cron entry:

*/5 * * * * /path/to/run_healthcheck.sh

The cron job runs every five minutes. Each run inserts one row. After a day, you have 288 rows. After a week, 2016 rows. SQLite handles that easily.

Serve the report with a one-liner:

python3 -m http.server 8080 --directory /path/to/report

Generate the report before serving:

python3 - <<'PY'
from healthcheck import generate_report
open("/path/to/report/index.html", "w").write(generate_report())
PY

Add that to the cron job too. Or use a second cron entry.

Verify the whole pipeline

Run the probe twice manually. Insert a fake failure. Generate the report. Confirm the failure appears.

python healthcheck.py
python healthcheck.py
python - <<'PY'
from healthcheck import record
record(503, 2000, 0, "simulated outage")
PY
python - <<'PY'
from healthcheck import generate_report
print(generate_report())
PY

The report should show two 200s and one 503. The success rate should be 66.7%. That proves the pipeline works end to end.

Limitations

The probe is a synthetic signal. It does not exercise real prompts. A long prompt may hit different timeouts. A streaming request may fail where a single response succeeds. Treat this as a tripwire, not a full load test.

The probe can be rate-limited itself. If the endpoint throttles your key, the health log records 429s. That is useful data. But it can also consume quota. Keep max_tokens at 1. Keep the prompt short.

The report is static. It does not send alerts. You need a second step for paging. A cron job can check the latest row and call a webhook. That is left as an exercise.

Who should skip this

Skip this if you have a paid SLA. Your provider already offers dashboards and alerts. A local probe adds noise.

Skip this if you need request-level tracing. This log stores one row per probe. It does not correlate with your application logs.

Skip this if you cannot tolerate any extra quota usage. The probe is cheap, but it is not free.

The takeaway

Free model endpoints are not static. They degrade. They throttle. They fail for your key while the status page stays green. A 40-line health log gives you a record. You can stop guessing and start measuring. Run it this weekend. You will have a baseline before the next outage.