Contract-Route Free Model Endpoints Before You Let Them Touch Real Requests

Free model endpoints change under you. A response can look right and still violate the integration contract. Route by contract, not by vibe. This tutorial builds a small router that compares a baseline and a candidate endpoint before you promote traffic.

You do not need paid compute for the smoke test. A local mock and Python standard library are enough.

1. Define the contract first

Pick one tool call that your app already depends on. In this example, the model must return a review comment. The contract has five rules.

• HTTP status is 2xx.
• message.content is non-empty.
• tool_calls exists.
• arguments parses as JSON.
• The arguments object contains file, line, and comment.
• line is an integer greater than zero.

A model that says looks fine but returns no tool call fails the contract. A model that returns valid JSON but misses line also fails. You check the shape, not the tone.

2. Build the contract router

Save this file as canary_router.py.

#!/usr/bin/env python3
import json
import os
import sys
import time
import urllib.error
import urllib.request

REQUIRED_KEYS = ('file', 'line', 'comment')

def post_chat(url, key, payload):
    req = urllib.request.Request(
        url,
        data=json.dumps(payload).encode(),
        headers={
            'Content-Type': 'application/json',
            'Authorization': 'Bearer ' + key,
        },
        method='POST',
    )
    with urllib.request.urlopen(req, timeout=20) as resp:
        return resp.status, json.loads(resp.read().decode())

def extract_text(data):
    try:
        return data['choices'][0]['message']['content']
    except (KeyError, IndexError, TypeError):
        return ''

def extract_tool_args(data):
    try:
        calls = data['choices'][0]['message'].get('tool_calls') or []
    except (KeyError, IndexError, TypeError):
        return []
    out = []
    for call in calls:
        fn = call.get('function') or {}
        raw = fn.get('arguments', '{}')
        try:
            out.append(json.loads(raw))
        except (json.JSONDecodeError, TypeError):
            out.append({'__invalid_json__': True})
    return out

def validate(content, tool_args):
    if not content.strip():
        return False, 'empty content'
    if not tool_args:
        return False, 'no tool call'
    for args in tool_args:
        if args.get('__invalid_json__'):
            return False, 'invalid tool JSON'
        for key in REQUIRED_KEYS:
            if key not in args:
                return False, 'missing ' + key
        line = args.get('line')
        if not isinstance(line, int) or line < 1:
            return False, 'bad line'
    return True, 'ok'

def check(url, key, payload):
    started = time.time()
    try:
        status, data = post_chat(url, key, payload)
        content = extract_text(data)
        tool_args = extract_tool_args(data)
        ok, reason = validate(content, tool_args)
        return ok, reason, round(time.time() - started, 3), status
    except Exception as exc:
        return False, repr(exc), round(time.time() - started, 3), None

def main():
    if len(sys.argv) != 2:
        print('usage: python3 canary_router.py request.json')
        sys.exit(2)
    baseline = os.environ['BASELINE_URL']
    candidate = os.environ['CANDIDATE_URL']
    key = os.environ.get('API_KEY', '')
    payload = json.load(open(sys.argv[1]))
    b = check(baseline, key, payload)
    c = check(candidate, key, payload)
    print('endpoint  ok  reason  status  latency_s')
    print('baseline ', b[0], b[1], b[3], b[2])
    print('candidate', c[0], c[1], c[3], c[2])
    if b[0] and c[0]:
        print('VERDICT: PROMOTE')
    elif b[0] and not c[0]:
        print('VERDICT: FALLBACK')
    elif not b[0] and c[0]:
        print('VERDICT: QUARANTINE')
    else:
        print('VERDICT: STOP')

if __name__ == '__main__':
    main()

The router uses only the Python standard library. It does not call any vendor SDK. That keeps the gate easy to audit.

3. Create a request fixture

Write request.json with a small Python snippet.

python3 - <<'PY'
import json
json.dump({
    'model': 'candidate',
    'messages': [
        {'role': 'user', 'content': 'Post one review comment for the diff.'}
    ],
    'tools': [
        {
            'type': 'function',
            'function': {
                'name': 'post_review',
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'file': {'type': 'string'},
                        'line': {'type': 'integer'},
                        'comment': {'type': 'string'}
                    },
                    'required': ['file', 'line', 'comment']
                }
            }
        }
    ]
}, open('request.json', 'w'), indent=2)
PY

Then run the router with environment variables.

export BASELINE_URL='https://baseline.example/v1/chat/completions'
export CANDIDATE_URL='https://candidate.example/v1/chat/completions'
export API_KEY='your-key'
python3 canary_router.py request.json

The output shows each endpoint, the pass/fail reason, HTTP status, and latency. The verdict maps to one action.

Baseline	Candidate	Verdict	Action
pass	pass	PROMOTE	Allow candidate for low-risk requests
pass	fail	FALLBACK	Keep baseline and file a bug
fail	pass	QUARANTINE	Inspect baseline regression or contract drift
fail	fail	STOP	Block both and debug the request

4. Smoke test against a local mock

Do not spend real tokens before you verify the router. Save this as mock_server.py.

import json
from http.server import BaseHTTPRequestHandler, HTTPServer

def make_response(ok):
    if ok:
        return {
            'choices': [{
                'message': {
                    'content': 'ok',
                    'tool_calls': [{
                        'function': {
                            'name': 'post_review',
                            'arguments': json.dumps({
                                'file': 'app.py',
                                'line': 10,
                                'comment': 'check this path'
                            })
                        }
                    }]
                }
            }]
        }
    return {'choices': [{'message': {'content': '', 'tool_calls': []}}]}

class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = int(self.headers.get('Content-Length', '0'))
        self.rfile.read(length)
        body = json.dumps(make_response(self.path == '/pass')).encode()
        self.send_response(200)
        self.send_header('Content-Type', 'application/json')
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, format, *args):
        pass

HTTPServer(('127.0.0.1', 8000), Handler).serve_forever()

Run the mock, then point both URLs at it.

python3 mock_server.py &
export API_KEY=''
export BASELINE_URL='http://127.0.0.1:8000/pass'
export CANDIDATE_URL='http://127.0.0.1:8000/fail'
python3 canary_router.py request.json

Expected verdict is FALLBACK. Stop the mock and swap the candidate to /pass to get PROMOTE. This proves the router fails closed before you involve real endpoints.

5. Use a free model endpoint as the candidate

Disclosure: This article was prepared as part of MonkeyCode's product outreach. For this tutorial, I treat MonkeyCode's free model access and free server option as operator-supplied availability claims. The router does not depend on a specific model name, quota, or hardware.

Point the candidate at a free OpenAI-compatible chat completions URL. Point the baseline at your current endpoint. Keep the mock as a regression fixture in CI.

for fixture in fixtures/*.json; do
  BASELINE_URL=$BASELINE_URL CANDIDATE_URL=$CANDIDATE_URL API_KEY=$API_KEY python3 canary_router.py $fixture
done

Run each fixture three to five times. Free model endpoints are non-deterministic. Record the pass rate, not a single pass.

Limitations

• Contract checks are structural. They do not prove the comment is useful.
• One run is not enough for a non-deterministic model.
• Rate limits and transient outages look like candidate failures. Check the status and latency columns.
• The router does not execute tool calls. Keep them dry-run.
• A malicious endpoint can return valid JSON with wrong data. This is not a security boundary.

Who should not use this approach

Skip this if you need strict latency SLOs from free infrastructure. Do not send private code or personal data. Do not use a single contract run as proof of semantic quality. Teams that need exact output should run a golden set with human review instead.

Core result

Contract routing turns an unstable free endpoint into a controlled experiment. Build the contract first. Smoke test with a local mock. Then compare a real candidate. Use the verdict, not excitement.

If you want to use MonkeyCode's free server option, start with the /pass mock and one fixture before sending any real request.