My CI Doesn't Know Which Free Model It's Calling Anymore

At 2:13 a.m. last Tuesday, my GitLab job went red. The stack trace ended with 404: /v1/chat/completions not found.

I checked my code. Nothing had changed.

The free model route had moved. My YAML still pointed at the old URL.

The problem was never the model

The model was fine. The weak point was the wiring.

I had copied the same endpoint into too many places. Every time a provider rotated a route, I was editing YAML, pushing, and waiting.

The same URL appeared in:

• the base_url in three CI jobs
• a local test script
• a teammate's one-off notebook

That is not model engineering. That is endpoint babysitting.

One small sidecar on a free server

MonkeyCode gives me access to free model routes plus a free server option. I use the server for one tiny sidecar. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The sidecar owns exactly one decision: which model should my jobs call.

Here is the service in plain Python:

# sidecar.py
import json
import os
from http.server import BaseHTTPRequestHandler, HTTPServer

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == '/config':
            body = json.dumps({
                'base_url': os.environ['MODEL_BASE_URL'],
                'model': os.environ['MODEL_NAME'],
                'timeout_s': int(os.environ.get('MODEL_TIMEOUT_S', '20')),
            }).encode()
            self.send_response(200)
            self.send_header('Content-Type', 'application/json')
            self.end_headers()
            self.wfile.write(body)
        elif self.path == '/healthz':
            self.send_response(200)
            self.end_headers()
            self.wfile.write(b'ok')
        else:
            self.send_response(404)
            self.end_headers()

if __name__ == '__main__':
    port = int(os.environ.get('PORT', '8000'))
    HTTPServer(('0.0.0.0', port), Handler).serve_forever()

Two things matter here:

• /config returns the current model endpoint.
• The sidecar never holds tokens or request bodies.

It is a config shim, not a proxy.

CI gets dumber

The job no longer knows the model URL. It asks the sidecar first.

contract-check:
  image: python:3.12-slim
  script:
    - pip install -q requests pytest
    - python -m pytest -q tests/test_sidecar_contract.py

The test verifies the shape before any real model call happens:

# tests/test_sidecar_contract.py
import os
import requests

SIDECAR_URL = os.environ['MODEL_SIDECAR_URL']

def test_config_shape():
    r = requests.get(f'{SIDECAR_URL}/config', timeout=5)
    r.raise_for_status()
    data = r.json()
    assert set(data) == {'base_url', 'model', 'timeout_s'}
    assert data['base_url'].startswith('https://')
    assert isinstance(data['model'], str) and data['model']
    assert 1 <= data['timeout_s'] <= 60

The actual model-calling script only has to do:

import json, os, urllib.request

url = os.environ['MODEL_SIDECAR_URL']
cfg = json.load(urllib.request.urlopen(f'{url}/config', timeout=5))

That is the whole client-side change.

Why I prefer this over a YAML hot-fix

• One edit, not five. When the route moves, I update the sidecar environment.
• The contract fails without a model call. I catch bad config before spending tokens.
• The CI stays model-agnostic. I can swap a route without touching jobs.
• The sidecar is too small to hide much logic.

When the sidecar is worth it

Situation	Direct call	One-file sidecar
One experiment last night	Fine	Overkill
One stable private deployment	Fine	Optional
Several jobs calling same model	Painful	Better
Free routes that rotate sometimes	Risky	Better
You need one place to rotate config	No	Yes

What still breaks

The sidecar is not magic.

• If the model behavior changes, the contract can still pass while output quality drops.
• If the free server is down, every job loses the config endpoint at once.
• If the route moves mid-job, a cached config can still go stale.
• If the sidecar is exposed without access controls, it becomes a config leak.

So I keep the sidecar private. I do not put tokens in it.

Who should skip this

This is not for everyone.

Skip it if:

• you have only one CI job
• your model endpoint is stable and owned by you
• you cannot run a small always-on service
• you want a full retry/streaming proxy

For those cases, a direct call is simpler.

For me, the sidecar removed the 2 a.m. YAML archaeology. That alone was worth the small service.

If you run a similar shim, what belongs inside it? I would like to hear where you draw the line between config and model behavior.