Most glue code around a free AI endpoint fails for a very boring reason: the request returned a 200, but the body was not what the downstream code expected. A quota hit can truncate JSON. A proxy restart can return an HTML error page with the same status. A model can send valid JSON that is missing the one field your code reads.
If you are using MonkeyCode's free model access, this is still true. Treat a free endpoint as a third-party API, not as a trusted library call.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The fix is not a better model or a longer retry loop. It is a contract probe: a small deterministic request that checks the response shape, size, and error behavior at the boundary before the rest of the system touches it.
Why try/except is not enough
Most integrations stop at json.loads inside a try block. That catches two failures:
- The response is not valid JSON.
- The request timed out.
It does not catch the worse failure: the response is valid JSON and structurally wrong.
For example, a prompt designed to return {"result": "..."} may come back as {"text": "..."} after a model update. A Python call like data.get('result').strip() then raises AttributeError three functions later, or worse, data.get('result') returns None and the code writes None into a production record. The HTTP request succeeded, but the system still failed.
A contract probe moves the validation from the middle of the workflow to the first contact point.
What belongs in a probe
A contract probe should check at least these properties:
- Transport contract: status code, content type, body size, latency budget.
- Shape contract: required fields and their types.
- Cost contract: token usage is present and within a budget.
- Error contract: does the endpoint return JSON errors, HTML errors, or both?
The probe is not an evaluation of model quality. It only asks: Can this endpoint satisfy the response contract for a known prompt right now?
Reproducible artifact: a small Python probe
The following probe uses only the Python standard library. It is a prototype, not a production SDK.
import json
import os
import time
import urllib.error
import urllib.request
class ContractError(Exception):
pass
def check_contract(data):
if not isinstance(data, dict):
raise ContractError('root must be an object')
if 'result' not in data:
raise ContractError('missing result')
if not isinstance(data['result'], str):
raise ContractError('result must be a string')
if len(data['result']) > 2_000:
raise ContractError('result is over the size budget')
if 'usage' in data:
usage = data['usage']
if not isinstance(usage, dict):
raise ContractError('usage must be an object')
total = usage.get('total_tokens')
if total is not None and (not isinstance(total, int) or total < 0):
raise ContractError('usage.total_tokens must be a non-negative integer')
def run_probe(url, prompt, timeout=5.0):
payload = json.dumps({'prompt': prompt, 'stream': False}).encode('utf-8')
req = urllib.request.Request(
url,
data=payload,
headers={
'Content-Type': 'application/json',
'Accept': 'application/json',
},
method='POST',
)
start = time.monotonic()
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read(8_192)
if resp.status != 200:
raise ContractError(f'unexpected status {resp.status}')
if len(raw) == 0:
raise ContractError('empty response body')
data = json.loads(raw)
except urllib.error.HTTPError as exc:
raise ContractError(f'HTTP error {exc.code}') from exc
except (urllib.error.URLError, TimeoutError) as exc:
raise ContractError(f'network failure: {exc}') from exc
check_contract(data)
elapsed_ms = (time.monotonic() - start) * 1000
data['_probe_latency_ms'] = elapsed_ms
return data
if __name__ == '__main__':
endpoint = os.environ.get('AI_ENDPOINT_URL')
if not endpoint:
raise SystemExit('Set AI_ENDPOINT_URL before running this probe.')
print(json.dumps(run_probe(endpoint, 'reply with exactly: ok'), indent=2))
Run the probe with a fixed prompt. The expected answer is not important; the contract is. If the probe fails on a known prompt, the endpoint should not receive production traffic until someone investigates.
Decide fail-open or fail-closed per signal
A single probe result can feed a small policy table. The right response depends on the cost of a wrong answer.
| Probe signal | Fail open | Fail closed |
|---|---|---|
Missing result field |
No | Yes |
result over 2,000 characters |
No | Yes |
HTTP 429 with no Retry-After
|
Only if safe stale cache exists | Yes |
| Timeout over 5 seconds | Only if stale cache is safe | Yes |
| HTML body with status 200 | No | Yes |
The key point: the probe fails with a reason, and the policy acts on that reason. A timeout and a schema change are not the same problem.
Use the free server option as a deterministic test double
If you run the probe against a public endpoint during every CI build, you are testing the provider's availability as much as your own code. Flakiness will teach people to ignore the probe.
A better setup is to point the same probe at a local server that implements the same response schema. If MonkeyCode's free server option is available to you, it can serve as that local test double: the server returns canned fixtures that satisfy the contract, so your test fails only when the contract changes, not when the network wobbles.
The public endpoint probe still runs separately, on a schedule or before a release, not on every commit.
Limitations
This workflow catches structural and transport failures. It does not measure whether the model's answer is correct.
If the provider changes the response schema, the probe catches it. If the provider returns valid schema with a semantically wrong answer, the probe may pass. You still need separate evaluation, human review, or domain-specific checks for high-stakes outputs.
Do not use a free endpoint as the only dependency when the output affects money, health, safety, or irreparable user data. If you have no runbook for what to do when the contract fails, adding a probe will only turn silent failures into louder ones.
Bottom line
Free AI endpoints are useful for prototyping, but reliability is an explicit engineering choice. Add a contract probe at the boundary, make it deterministic with a local test double, and decide fail-open or fail-closed before a quota hit surprises you. If you already have free model access and a free server option, you can build and run this probe without spending anything first.
Top comments (0)