Learn to Validate a Free LLM Endpoint by Building a Tiny Request Probe

A free model endpoint is only useful if you can measure what free actually gives you.

Most free-tier pages show a token allowance and a server URL. They do not show how the endpoint behaves when a request is malformed, when the queue is busy, or when your retry logic is wrong. Before I let any free endpoint into a class project, I run a small probe.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The problem

A free LLM server can fail in at least three places that a quota page does not show:

• it can accept a request and return an error after a long delay
• it can return HTTP 200 with usage numbers that do not match your actual prompt
• it can rate-limit after one request, so a batch job dies halfway

Token allowance is the easiest number to check. It is not the most important number.

What you will build

A standard-library Python probe, roughly 70 lines, that sends a chat-style request to any OpenAI-compatible endpoint, records latency and token usage, treats 429 and 5xx as retryable once, and prints a compact report.

Prerequisites:

• Python 3.11 or newer
• no external packages
• an endpoint URL and API key from your provider dashboard
• a model id that the endpoint accepts

The probe

Save this as free_probe.py.

import json
import os
import time
import urllib.error
import urllib.request

ENDPOINT = os.environ.get('LLM_ENDPOINT', 'https://api.example.com/v1/chat/completions')
API_KEY = os.environ.get('LLM_API_KEY', 'replace-me')
MODEL = os.environ.get('LLM_MODEL', 'replace-me')

def call_once(payload):
    data = json.dumps(payload).encode('utf-8')
    req = urllib.request.Request(
        ENDPOINT,
        data=data,
        headers={
            'Content-Type': 'application/json',
            'Authorization': f'Bearer {API_KEY}',
        },
        method='POST',
    )
    start = time.monotonic()
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            body = resp.read().decode('utf-8')
            return resp.status, json.loads(body), time.monotonic() - start
    except urllib.error.HTTPError as exc:
        body = exc.read().decode('utf-8', errors='replace')
        return exc.code, {'error': body}, time.monotonic() - start
    except urllib.error.URLError as exc:
        return None, {'error': str(exc.reason)}, time.monotonic() - start

def main():
    payload = {
        'model': MODEL,
        'messages': [
            {'role': 'system', 'content': 'You are a minimal test harness.'},
            {'role': 'user', 'content': 'Reply with exactly the word pong.'}
        ],
        'temperature': 0,
        'max_tokens': 8,
    }

    attempts = []
    for attempt in range(2):
        status, body, seconds = call_once(payload)
        attempts.append((status, seconds, body))
        if status == 429 or (status is not None and status >= 500):
            print(f'attempt {attempt + 1}: retryable status {status} after {seconds:.2f}s')
            time.sleep(2 ** attempt)
            continue
        break

    status, seconds, body = attempts[-1]
    print('status:', status)
    print(f'latency: {seconds:.2f}s')

    if status != 200:
        print('error body:', json.dumps(body)[:300])
        return

    usage = body.get('usage', {})
    print('usage:', usage)
    content = body.get('choices', [{}])[0].get('message', {}).get('content', '')
    print('content:', repr(content))

    prompt_tokens = usage.get('prompt_tokens')
    completion_tokens = usage.get('completion_tokens')
    if prompt_tokens is None or completion_tokens is None:
        print('warning: usage fields missing; the endpoint may not report token accounting')

if __name__ == '__main__':
    main()

Run it with your real values:

export LLM_ENDPOINT='your provider /v1/chat/completions URL'
export LLM_API_KEY='your key'
export LLM_MODEL='the model id from the dashboard'
python free_probe.py

Expected output, with sample numbers:

attempt 1: retryable status 429 after 1.12s
status: 200
latency: 0.44s
usage: {'prompt_tokens': 17, 'completion_tokens': 1, 'total_tokens': 18}
content: 'pong'

These numbers are a sample, not a benchmark. Your endpoint, region, and current queue will change them.

How to read the result

• A first-attempt 429 means capacity is shared. Do not start a batch at the top of an hour.
• A 200 response with missing usage fields means you cannot track the advertised token allowance.
• A prompt_tokens value far from your actual prompt means the tokenizer or accounting path is doing something unexpected.
• Long latency on one small request may be cold start, not permanent speed.

One error input that matters

Change messages to an empty list and run the probe again:

payload = {
    'model': MODEL,
    'messages': [],
    'temperature': 0,
    'max_tokens': 8,
}

A well-built endpoint should reject this with HTTP 400. A sloppy wrapper may hang until the timeout. That difference matters when your client sends an empty context after a filtering step.

Where MonkeyCode fits

MonkeyCode advertises free model access, a free server option, and a 30 million token allowance. I treat those as operator-supplied claims to verify, not as permanent guarantees. As of 2026-08-14, the useful next step is to run this probe against the current endpoint and model id from the dashboard, then save the output as your baseline. If a single small request gets a 429 during a quiet hour, that baseline tells you more than the allowance number.

I did not include an endpoint or key here because those rotate and belong in your account settings, not in a tutorial.

Limitations

• One request is not a load test, and latency is not p50 or p95.
• A free server may be single-region and carry no uptime promise.
• A 30 million token allowance can disappear quickly with long prompts or careless retries.
• The probe does not test privacy, so do not send private code.

Who should not use this approach

• production applications that need a latency SLO
• large batch evaluations that need deterministic retry budgets
• teams that cannot keep API keys out of source control

Extension exercise

Run the probe ten times with temperature: 0 and compare content. If the response changes, you may be hitting different backends or a nondeterministic router. Add median and maximum latency across twenty requests, then record a local token sum that stops before it reaches your own budget.

If you run this against a free endpoint, keep the printed report in your project notes. It is a more useful artifact than a screenshot of the quota page.