Jordan Huang

Posted on Aug 17

Free Model Endpoints Are Not Magic: Five Checks Before CI Trust

#gitlab #ai #cicd #testing

A free model endpoint looks great in a demo.

CI is not a demo.

A model call is a dependency. It can throttle, time out, or return broken JSON. I treat it as untrusted until it passes a small test battery.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What I am testing

The target is MonkeyCode's free model endpoint and free server option.

I am not assuming a model name, quota, or hardware. Those details change. The harness only needs an endpoint URL and a token.

A free server is useful here. Run the harness from that server, not from a laptop. You want to separate endpoint slowness from your Wi-Fi.

A free endpoint is not automatically bad. It just needs evidence.

The five checks

Contract validity. Does the response parse as JSON? Does it contain the fields you asked for?
Latency under retry. How long does one call take? What happens when the first call fails?
Idempotency. If you send the same prompt twice, do you get the same structure? Does a retry change the answer?
Failure modes. What does a 429, 500, empty body, or truncated JSON look like?
Drift. Run the same prompt ten times. How much does the output vary?

These checks do not measure quality. They measure whether the endpoint is safe to put in a pipeline.

The harness

Save this as free_endpoint_probe.py.

import json
import os
import time
import urllib.error
import urllib.request

ENDPOINT = os.environ['MONKEYCODE_ENDPOINT']
TOKEN = os.environ['MONKEYCODE_TOKEN']

CASES = [
    {'id': 'json_extract', 'prompt': 'Return JSON only with title and severity keys.'},
    {'id': 'empty_prompt', 'prompt': ''},
    {'id': 'long_prompt', 'prompt': 'Analyze this issue: ' + ('context ' * 500)},
]

def parse_json(raw: str):
    try:
        return json.loads(raw)
    except json.JSONDecodeError:
        start = raw.find('{')
        end = raw.rfind('}')
        if start != -1 and end != -1 and end > start:
            try:
                return json.loads(raw[start:end + 1])
            except json.JSONDecodeError:
                pass
        return None

def call_once(prompt: str, timeout: int = 20):
    payload = json.dumps({'prompt': prompt}).encode()
    req = urllib.request.Request(
        ENDPOINT,
        data=payload,
        headers={
            'Authorization': f'Bearer {TOKEN}',
            'Content-Type': 'application/json',
        },
    )
    start = time.monotonic()
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            raw = resp.read().decode()
            elapsed_ms = (time.monotonic() - start) * 1000
            return {
                'ok': True,
                'status': resp.status,
                'ms': round(elapsed_ms, 1),
                'raw': raw,
                'json': parse_json(raw),
            }
    except urllib.error.HTTPError as exc:
        elapsed_ms = (time.monotonic() - start) * 1000
        return {
            'ok': False,
            'status': exc.code,
            'ms': round(elapsed_ms, 1),
            'body': exc.read().decode()[:200],
        }
    except Exception as exc:
        elapsed_ms = (time.monotonic() - start) * 1000
        return {'ok': False, 'status': 'exception', 'ms': round(elapsed_ms, 1), 'error': str(exc)}

for case in CASES:
    result = call_once(case['prompt'])
    print(json.dumps({'case': case['id'], **result}, indent=2))

Run it like this:

MONKEYCODE_ENDPOINT=https://your-endpoint MONKEYCODE_TOKEN=your-token python free_endpoint_probe.py

Do not commit the token.

Turn it into a scorecard

Make pass/fail rules before you run.

Check	Pass when
Contract	Every non-empty response has valid JSON with the expected keys.
Latency	p50 is under your CI timeout, usually 15 seconds.
Retry	A failed call can be retried once without doubling total time.
Idempotency	Two identical calls return the same top-level keys.
Failure modes	429 includes a Retry-After header or recoverable error body.
Drift	The output schema stays stable across ten calls.

If any row fails, do not wire the endpoint into CI yet.

What I expect to see, but verify

These are common failure patterns in free endpoints. Do not treat them as facts about MonkeyCode. Test them against your own account.

A 429 on a burst of calls.
JSON wrapped in a sentence or markdown fence.
A retry that changes the output because sampling is non-deterministic.
A long prompt that times out.
An empty body on an empty prompt.

Some of these are fine. A 429 is fine if the client backs off. Broken JSON is not fine if the next job assumes an integer.

What this harness does not test

It does not test accuracy.

It does not test whether the model gives the right severity. It only tests whether the response is usable.

It also cannot prove uptime over a month. Run it as a scheduled job if you need that signal.

I have not run this exact battery against current MonkeyCode quotas at publish time. The point is the protocol, not the number.

Who should not use a free endpoint in CI

Teams that need a strict SLA in production.
Tasks where a retry changes the meaning of the result.
Workloads that send private or regulated data.
Teams that will not pin and review the prompt contract.

If you cannot afford to inspect broken JSON, a free endpoint is not your bottleneck.

Final note

A free model endpoint can be a useful first pass.

But trust is a test result, not a marketing claim.

Copy the harness. Set thresholds. Run it from a free server. Let the scorecard decide.

Test MonkeyCode's free endpoint the same way you would test any new dependency.

DEV Community