Build a Tiny Endpoint Canary Before Trusting a Free Model Tier

Build a Tiny Endpoint Canary Before Trusting a Free Model Tier

A free model tier only matters if your code can reach it, measure it, and fail cleanly without burning your afternoon.

I keep seeing the same student mistake: pick a free model endpoint, spend an hour improving prompts, then realize the real problem was a missing route, an unread 401 body, or a quota that reset at an inconvenient time. A lot of current AI conversation treats cheaper model access as the headline, but the more boring operational question is whether a free tier can serve a deterministic two-token reply when your app actually asks.

MonkeyCode fits here as a lab, not as a benchmark.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode describes itself as an open-source project with free model access and a free server option. I am not quoting a token amount because free-tier quotas change, and that is exactly the kind of number that goes stale. Check the current docs before coding. If you have a free server slot, run this probe from that server so you measure the same network path your real app will use.

What you will build

• A Python 3 script using only the standard library.
• One OpenAI-compatible chat completion request.
• Output for status, wall-clock latency, token usage, and the reply.
• A provider-neutral check: any endpoint is untrusted until one round trip returns the expected fields.

Prerequisites

• Python 3.10 or newer.
• A terminal.
• An endpoint that exposes /chat/completions in the common OpenAI-compatible shape.
• The provider's base URL, API key, and model name.

If your provider uses a different request shape, keep the same structure and replace only the request builder.

The canary

import json
import os
import time
import urllib.error
import urllib.request

BASE_URL = os.environ.get("OPENAI_BASE_URL", "http://127.0.0.1:8000/v1")
API_KEY = os.environ.get("OPENAI_API_KEY", "local-key")
MODEL = os.environ.get("MODEL_NAME", "test-model")

payload = {
    "model": MODEL,
    "messages": [{"role": "user", "content": "Reply with exactly: pong"}],
    "max_tokens": 5,
    "temperature": 0,
}

req = urllib.request.Request(
    f"{BASE_URL}/chat/completions",
    data=json.dumps(payload).encode(),
    headers={
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    },
)

start = time.perf_counter()
try:
    with urllib.request.urlopen(req, timeout=8) as resp:
        body = json.loads(resp.read().decode())
        elapsed_ms = (time.perf_counter() - start) * 1000
        usage = body.get("usage", {})
        print(f"status={resp.status}")
        print(f"elapsed_ms={elapsed_ms:.1f}")
        print(f"prompt_tokens={usage.get('prompt_tokens')}")
        print(f"completion_tokens={usage.get('completion_tokens')}")
        print(f"total_tokens={usage.get('total_tokens')}")
        print(f"reply={body['choices'][0]['message']['content']!r}")
except urllib.error.HTTPError as e:
    print(f"status={e.code}")
    print(f"body={e.read().decode()[:200]}")
except Exception as e:
    print(f"error={type(e).__name__}: {e}")

Run it

export OPENAI_BASE_URL="https://your-provider.example/v1"
export OPENAI_API_KEY="your-key"
export MODEL_NAME="model-name-from-docs"
python endpoint_canary.py

Expected output:

status=200
elapsed_ms=742.3
prompt_tokens=18
completion_tokens=2
total_tokens=20
reply='pong'

Read the output

• status=200 is necessary but not sufficient. A provider can return 200 and still send an HTML login page or an empty JSON body.
• elapsed_ms tells you about the path from your current host, which matters more than a dashboard benchmark.
• prompt_tokens and completion_tokens let you compare actual accounting with the free allowance. Do not hardcode the allowance; verify it each week.
• reply='pong' confirms the endpoint accepted the message format, model name, and max_tokens setting.

One deliberate failure: the wrong key

export OPENAI_API_KEY="wrong-key"
python endpoint_canary.py

status=401
body={"error":{"message":"Invalid API key"}}

This failure is useful because it proves the script prints the response body. Many free-tier problems are hidden in that body, not in the HTTP status alone.

What this teaches

• A one-request canary validates more than a login: route, auth, schema, model behavior, and token accounting.
• Free model access is only useful if you can observe it from the host that will run the app.
• A tiny deterministic prompt gives you a stable baseline; do not use a vague prompt because the reply itself becomes hard to verify.

Where this breaks

• This is not a load test. It does not predict behavior under parallel requests.
• It does not test streaming, tool calls, multi-turn memory, or model routing changes.
• Free tiers can return 429 responses at different limits than their documentation suggests.
• One green run does not mean the endpoint stays stable during a study session or demo.

Common mistakes

• Printing only status and hiding the response body.
• Assuming a 200 means the correct model answered.
• Ignoring usage and then being surprised when the free allowance disappears.
• Running the probe only from a laptop when the app will run on a free server.

Extension

• Store each sample in SQLite and run the probe every five minutes.
• On 429, read and respect Retry-After instead of looping immediately.
• Add validation for choices, message, and usage so a malformed 200 fails loudly.
• Run the same probe from both your laptop and your free server, then compare latency and failures.

Who should skip this

• You need streaming, tool calls, multi-modal input, or real rate-limit simulation.
• You already have server-side observability for model calls.
• You are choosing a production provider with an SLA and support agreement.

If you already have access to MonkeyCode's free tier or any OpenAI-compatible lab endpoint, run this thirty-second canary before writing your RAG loop. The failure it surfaces is usually cheaper than a late-night debugging session.