DEV Community: Sam Rivera

Quantify the Breaking Point of a Free Model Endpoint

Sam Rivera — Mon, 17 Aug 2026 23:35:36 +0000

The current wave of agent-gatekeeper posts keeps circling the same failure. A free endpoint works in the demo. Then it stalls under burst load. I don't want to find that out in production. So I built a small burn-test harness. It turns vague 'free model' claims into measurable rows.

MonkeyCode's free tier is the target. The project describes itself as open source. It advertises a free model endpoint and a free server option. The token allowance is listed as 30 million tokens. I treat those numbers as claims, not facts. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What I measure

A free endpoint is only useful if four things stay readable.

Latency under normal prompts.
Latency growth when prompt size jumps.
Error class under burst traffic.
Free server health over a longer window.

Skip those four and you will discover the limit at the worst time.

The harness

This Python script is endpoint-agnostic. Point it at any OpenAI-style chat completions endpoint.

# burn.py
import os, time, json, urllib.request, urllib.error

ENDPOINT = os.environ['ENDPOINT']
KEY = os.environ['API_KEY']
MODEL = os.environ.get('MODEL', 'free-model')

def call(prompt, timeout=20):
    payload = json.dumps({
        'model': MODEL,
        'messages': [{'role': 'user', 'content': prompt}],
    }).encode()
    req = urllib.request.Request(
        ENDPOINT,
        data=payload,
        headers={
            'Authorization': f'Bearer {KEY}',
            'Content-Type': 'application/json',
        },
    )
    started = time.time()
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            body = resp.read()
            return {
                'ok': True,
                'ms': round((time.time() - started) * 1000),
                'bytes': len(body),
                'status': resp.status,
            }
    except urllib.error.HTTPError as e:
        return {
            'ok': False,
            'ms': round((time.time() - started) * 1000),
            'status': e.code,
            'retry_after': e.headers.get('Retry-After'),
        }
    except Exception as e:
        return {'ok': False, 'error': type(e).__name__}

Run four scenarios. Short checks baseline. Long checks token scaling. Burst checks rate-limit reset behavior.

python - <<'PY'
from burn import call

short = 'Summarize this: ' + ('hello ' * 10)
medium = 'Return keys only: ' + ('context ' * 200)
long = 'Tag intent in: ' + ('noisy ' * 800)
burst = 'Reply ok to: ' + ('ping ' * 50)

for label, prompt in [('short', short), ('medium', medium), ('long', long)]:
    print(label, call(prompt))

for i in range(8):
    print('burst', i, call(burst))
PY

Sample output, not a benchmark

This block is a fixture. It shows the shape I want. It is not a measured result from the public endpoint.

short {'ok': True, 'ms': 612, 'bytes': 241, 'status': 200}
medium {'ok': True, 'ms': 1187, 'bytes': 492, 'status': 200}
long {'ok': True, 'ms': 2841, 'bytes': 938, 'status': 200}
burst 4 {'ok': False, 'ms': 97, 'status': 429, 'retry_after': '4'}
burst 5 {'ok': False, 'ms': 91, 'status': 429, 'retry_after': '4'}

Scoring table

Thresholds are my solo-builder limits. They are not industry rules.

Signal	Green	Yellow	Red
Short prompt p50	under 1.5s	1.5s to 4s	over 4s
Long prompt vs short	under 6x	6x to 12x	over 12x
Burst 429 count	0	1 to 2	over 2
Retry-After behavior	fixed reset	jitter	missing or zero

A red row does not mean the service is bad. It means the free tier is not suited to my task. That is a useful exit.

Server health check

The free server option needs a separate probe. A model endpoint can stay healthy while the server silently drops requests. Run this loop from a machine close to the deployment.

for i in $(seq 1 40); do
  code=$(curl -s -o /dev/null -w '%{http_code}' \
    -H 'Authorization: Bearer $API_KEY' \
    '$SERVER/health' || echo 'fail')
  echo "$i $code"
  sleep 20
done

I watch for two failure classes. The first is a non-200 status. The second is a total failure from DNS or connection refused. Both count against the free server claim.

Where I would draw the line

Green means I can use the free model for text cleanup, small summarization, and non-blocking helper calls. Yellow means I add caching and a timeout. Red means I keep the free endpoint out of any request path a user waits on.

For the free server, a single failed health check is not fatal. Two failures in one window are. I also cap the total token spend with a local ledger. That avoids a surprise when the 30 million token allowance runs out.

If you want to test MonkeyCode's free tier, point the harness at the documented endpoint. Your local error log will tell you more than a feature page can.

Limits and who should skip this

I did not publish a public benchmark here. Quotas, model names, and server specs change. Read the project docs before running the script. Do not send secrets or private user text through a free endpoint until you understand retention.

Skip this approach if you need low tail latency, guaranteed uptime, or compliance review. A free endpoint is a canary, not a foundation.

What I would test next

The next missing signal is partial failure. A long prompt can return truncated JSON without an HTTP error. If you run this against your own endpoint, tell me which failure mode you saw first: rate-limit jitter, silent truncation, or server flapping?

Use a Free Server and 30M Tokens as a Disposable Regression Oracle

Sam Rivera — Mon, 17 Aug 2026 17:15:37 +0000

A CLI changed last night. I did not want to read 400 lines of diff.

So I asked one question: can a free model act as a poor man's regression oracle?

Not a full judge. Not a replacement for tests. A temporary smell detector I can throw away.

I had two things to work with, both operator-supplied: MonkeyCode's free model access and its free server option. The advertised allowance is 30 million tokens. That is a budget, not a promise. I treat it that way.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What I built

I built a tiny harness that sends CLI output to a model and asks one binary question:

Did this output change its behavior in a way that matters?

The harness is not smart. It is disposable. That is the point.

Here is the contract:

12 test cases
one CLI command
one temperature-zero prompt
strict timeout
token usage logged after each call

If the harness breaks, I can remove it without losing my real test suite.

Why this angle?

A free server changes the math.

If I run this on a paid endpoint, I hesitate on every call. I worry about cost before I worry about correctness. On a free tier, the decision flips.

I can afford to be wrong.

That is the real workflow: use the free option for high-volume, low-stakes judgment. Keep the expensive checks for release gates.

But a free tier also hides failure differently. Latency may spike. Quotas may reset at odd times. The model may refuse a request and return a 200. I need to see those failure modes before I lean on the tool.

Step 1: Install a minimal client

The client keeps the request shape boring on purpose.

mkdir regression-oracle && cd regression-oracle
python3 -m venv .venv && source .venv/bin/activate
pip install httpx

I use httpx because it gives me explicit timeouts and clean JSON.

Step 2: Write the test fixture

The fixture is a list of small cases. Each case has a command, expected-behavior notes, and a label.

# cases.py
CASES = [
    {"label": "ls_root", "cmd": ["ls", "/"], "watch": "list of top-level directories"},
    {"label": "env_count", "cmd": ["sh", "-c", "env | wc -l"], "watch": "environment size"},
    {"label": "disk_usage", "cmd": ["df", "-h", "/tmp"], "watch": "tmp disk availability"},
    {"label": "git_status", "cmd": ["git", "status", "--short"], "watch": "working tree state"},
]

Twelve cases are enough for a canary. More cases do not make the result more honest. They make the run slower.

Step 3: Add the oracle prompt

The prompt stays fixed. I do not ask the model to fix anything.

You are a regression smell detector.

Given:
- a command label
- the current command output
- a one-line note about expected behavior

Answer only with:
{"changed": true|false, "reason": "one short reason"}

Do not explain. Do not offer fixes.

That is the whole judge. It returns JSON or it fails.

Step 4: Run the harness

The harness catches network failure, timeout, malformed JSON, and token usage.

# oracle.py
import asyncio
import json
import subprocess
import time

import httpx

from cases import CASES

API_URL = "https://your-monkeycode-endpoint.example/v1/chat/completions"
API_KEY = "your-key"  # use environment variable in real use
SYSTEM_PROMPT = """You are a regression smell detector.
Given a command label, current output, and one-line note, answer only with JSON:
{\"changed\": true|false, \"reason\": \"one short reason\"}"""

def run_command(cmd):
    started = time.time()
    proc = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
    elapsed = time.time() - started
    return {
        "stdout": proc.stdout[-2000:],
        "stderr": proc.stderr[-1000:],
        "returncode": proc.returncode,
        "elapsed_ms": round(elapsed * 1000),
    }

async def ask_oracle(client, label, output, watch):
    payload = {
        "model": "CHANGEME",
        "temperature": 0,
        "messages": [
            {"role": "system", "content": SYSTEM_PROMPT},
            {
                "role": "user",
                "content": json.dumps({
                    "label": label,
                    "watch": watch,
                    "output": output,
                }),
            },
        ],
    }
    try:
        resp = await client.post(API_URL, json=payload, timeout=20)
        resp.raise_for_status()
    except Exception as exc:
        return {"status": "request_failed", "error": type(exc).__name__}

    data = resp.json()
    usage = data.get("usage", {})
    text = data["choices"][0]["message"]["content"]

    try:
        parsed = json.loads(text)
    except json.JSONDecodeError:
        parsed = {"raw": text, "parse": "failed"}

    return {
        "status": "ok",
        "parsed": parsed,
        "tokens": usage,
    }

async def main():
    async with httpx.AsyncClient() as client:
        for case in CASES:
            command_result = run_command(case["cmd"])
            oracle_result = await ask_oracle(
                client, case["label"], command_result["stdout"], case["watch"]
            )
            print(json.dumps({
                "case": case["label"],
                "command_ms": command_result["elapsed_ms"],
                "oracle": oracle_result,
            }))

if __name__ == "__main__":
    asyncio.run(main())

This is not production code. It is a canary.

What I measure

I record six things per case:

Metric	Why it matters
command latency	distinguishes slow CLI from slow model
model latency	flags free-tier queueing
token usage	confirms the advertised budget is being consumed
JSON validity	tests whether the oracle can follow a strict shape
refusal mode	catches silent 200s with empty content
changed flag	gives the actual regression signal

One run gives me a decision table.

Sample output shape

The harness prints one JSON line per case. Here is the shape I use, not a fabricated result:

{
  "case": "git_status",
  "command_ms": 18,
  "oracle": {
    "status": "ok",
    "parsed": {"changed": false, "reason": "working tree matches expected state"},
    "tokens": {"prompt_tokens": 142, "completion_tokens": 31, "total_tokens": 173}
  }
}

If status is request_failed, I stop. A free tier that cannot answer is not an oracle. It is a queue.

Step 5: Run a failure fixture

A good canary includes one case designed to fail.

{
    "label": "tmp_usage_anomaly",
    "cmd": ["sh", "-c", "du -sh /tmp/* 2>/dev/null | head -5"],
    "watch": "tmp directory file usage",
}

The point is not for the model to panic. The point is to see whether the JSON contract survives surprising input.

If the model starts explaining instead of returning JSON, I know the oracle is not reliable enough for automation.

What breaks first

I expect these failures, based on how free tiers tend to behave:

Timeout on first request after idle. Cold start is real.
Malformed JSON when the output contains a stray brace.
Token padding when a command returns more than 2,000 characters.
Empty content without an error field.
Rate-limit response when I re-run too quickly.

I do not fix those failures. I record them. The data tells me whether to trust the oracle for one-off checks or only for batched work.

Who should not use this approach

This is not for you if:

You need a deterministic pass/fail gate for releases.
You work with sensitive CLI output that cannot leave your machine.
You need millisecond-level answers on every request.
You cannot tolerate occasional malformed JSON.

For those cases, keep your unit tests and a real integration suite.

This harness is for the fuzzy middle: a quick smell check when a teammate changes a command and you want one extra signal before you read the diff.

The rollback plan

The exit is clean.

rm -rf regression-oracle

No repo change. No config file. No dependency in the build pipeline.

That is the whole appeal of a disposable oracle. When it stops being useful, remove it.

What I would try next

One improvement is a diff-aware prompt: send both old and new output, then ask if the new output is materially different. Another is a token ceiling per case so one verbose command cannot eat the whole free allowance.

If you try this, record the first failure you hit with the free server. Was it cold start, malformed JSON, or a rate limit?

Use a Free Model Endpoint as a Last-Minute Shell Gatekeeper

Sam Rivera — Mon, 17 Aug 2026 10:44:51 +0000

Why this is worth reading: you already paste model output into a terminal, and one bad command is all it takes to delete a directory, force-push over a branch, or pipe a remote script into sh. More prompting does not remove that risk. In this walkthrough, you will build a small pre-execution gatekeeper that catches obvious disasters locally, then asks a free model endpoint for a conservative second decision before you ever run the command.

The problem with model output in a shell

You usually review a generated command in your head, but that review is weakest exactly when the output looks plausible. A model can produce git push origin main --force with full confidence because it does not know your current branch has work you have not pushed. A static review catches patterns you can write down; a model review catches context that is hard to fit in a regex.

The goal is not to block every automated command. It is to insert one consistent control point between “generated text” and “executed shell.” The gatekeeper should be cheap enough to leave in place, open enough to skip when you are explicitly testing it, and safe enough to fail toward asking when the review service is unavailable.

What you will build

You will build a Bash script called ai-guard. It accepts a command and a working directory, runs a set of static dangerous patterns, and then sends the command to a model endpoint for a second pass. On timeout, it fails to ASK rather than silently allowing the command. You can run it locally as a wrapper, or place it on a small free server and call it from CI, cron, or an SSH forced command.

MonkeyCode’s open-source project is the free model access and free server option I will use for the review layer. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator’s current materials list 30 million free tokens and a free server slot. Treat those as starting numbers and confirm them from the project dashboard before you depend on them. I will not assume any specific model name, guaranteed latency, or perpetual quota here.

Static rules catch the obviously destructive commands

Copy the following script to /usr/local/bin/ai-guard and make it executable:

#!/usr/bin/env bash
set -euo pipefail

ENDPOINT="${MONKEYCODE_ENDPOINT:?set MONKEYCODE_ENDPOINT}"
AUTH="${MONKEYCODE_AUTH:?set MONKEYCODE_AUTH}"
TIMEOUT_SECS="${GATE_TIMEOUT_SECS:-5}"

cmd="$1"
cwd="${2:-$PWD}"

dangerous_patterns=(
  'rm\s+-rf\s+/'
  'mkfs\.'
  'dd\s+if=/dev/zero'
  'git\s+push\s+.*--force'
  'curl\s+.*\|\s*(ba)?sh'
  'chmod\s+-R\s+777'
  'sudo\s+reboot'
)

for pat in "${dangerous_patterns[@]}"; do
  if echo "$cmd" | grep -Eq "$pat"; then
    echo "DENY static: matched $pat"
    exit 1
  fi
done

payload=$(jq -n --arg c "$cmd" --arg d "$cwd" \
  '{command:$c, cwd:$d, instruction:"Return exactly ALLOW, DENY, or ASK followed by one short reason."}')

response=$(curl -sS --max-time "$TIMEOUT_SECS" \
  -H "Authorization: Bearer $AUTH" \
  -H "Content-Type: application/json" \
  -d "$payload" \
  "$ENDPOINT" || true)

if [ -z "$response" ]; then
  echo "ASK fallback: endpoint did not respond in ${TIMEOUT_SECS}s"
  exit 2
fi

decision=$(echo "$response" | jq -r '.decision // "ASK"')
reason=$(echo "$response" | jq -r '.reason // "no reason"')
echo "${decision}: ${reason}"

case "$decision" in
  ALLOW|allow) exit 0 ;;
  DENY|deny) exit 1 ;;
  *) exit 2 ;;
esac

This is a deliberately small artifact, not a security product. The static list is your first line of defense because it costs no tokens and never times out. The model pass is for cases that need more context, such as a destructive-looking command that might be valid in one directory but not another.

The free model endpoint does the second pass

You run the gatekeeper with:

./ai-guard "git push origin main --force" "$PWD"

The static matcher will deny that command before the endpoint is called. For a command that passes the static checks, the script constructs a single small JSON payload and sends it to MONKEYCODE_ENDPOINT. The prompt is intentionally narrow: return ALLOW, DENY, or ASK with one short reason. You want one decision, not a paragraph.

If the endpoint does not answer within five seconds, the script exits with status 2 and prints ASK fallback. That is the control you want when a free endpoint goes quiet. You can tune GATE_TIMEOUT_SECS down to two seconds for interactive use, but remember that lower timeouts cause more ASK fallbacks.

Run it on the free server

If all you need is a local wrapper, you can stop here. The more useful option is to put the script on a small server and call it from a pipeline. On the free server option, create a dedicated user and copy the script into its home directory.

A minimal systemd service can keep the environment variables in one place:

[Service]
Environment=MONKEYCODE_ENDPOINT=your-endpoint-url
Environment=MONKEYCODE_AUTH=your-token
Environment=GATE_TIMEOUT_SECS=5
ExecStart=/usr/local/bin/ai-guard
User=gatekeeper
Restart=on-failure

You would more likely invoke it through SSH or a small HTTP wrapper, but the unit keeps the token out of your shell history and makes the timeout policy explicit. Do not put the token in the script itself.

Test fixtures and failure modes

You need at least three fixtures before you trust this in a real workflow:

./ai-guard "git push origin main --force" should exit 1 from the static deny list.
./ai-guard "python3 -m pytest tests/" should pass static and return a model decision without hanging.
GATE_TIMEOUT_SECS=1 ./ai-guard "python3 -m pytest tests/" with the endpoint unreachable should exit 2 and print ASK fallback.

Run the timeout fixture by pointing MONKEYCODE_ENDPOINT at a port nothing is listening on, or by temporarily unsetting the token. If the failure mode is not obvious, the gatekeeper is not ready.

Cost and boundaries

You pay token cost only for commands that survive the static list. If most of your generated commands are ordinary python, npm, or git status calls, the review pass is short. The operator-supplied 30 million free tokens are enough for a very large number of one-line reviews, but the exact consumption depends on the endpoint and prompt length. Measure one real review from the dashboard before you assume a daily budget.

This approach is not a substitute for sandboxing untrusted model output. You should still run risky work in a disposable environment when you can. The gatekeeper is a second decision, not a permission system with a strong identity.

Who should skip this

Do not use this as your only control if you are running model output as root, against production data, or in an environment where a five-second timeout breaks an automated pipeline. A static deny list also gives you false confidence if you never update it. If you need an audit trail, combine this with a separate execution log instead of relying on the gatekeeper output alone.

Try the script with three commands from your own recent work. Add the one dangerous pattern your stack actually needs before you wire it any further. What single command pattern would you add to the static deny list first?

Build a 45-Minute Demand Probe Before You Trust a 30M-Token Free Server

Sam Rivera — Sun, 16 Aug 2026 21:59:49 +0000

Why this is worth reading: a free tier with 30 million tokens and a free server sounds like a default yes, but free infrastructure becomes risky when you never measure it. You do not need another watchdog; you need a 45-minute demand probe that tells you whether the endpoint tolerates your concurrency, what each request costs you, and when to walk away before your side project quietly depends on a changing free tier.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat the free server claim as a hypothesis

MonkeyCode's operator describes free model access and a free server option. That is a starting claim, not a permanent capability guarantee. Free tiers change, shared servers get cold, and an open source repository does not automatically equal a hosted endpoint with stable behavior. The practical move is to test the shape of traffic you will actually send.

The probe below does three things:

Sends a fixed payload in a controlled concurrency loop.
Records status codes, latencies, failures, and whatever token usage the endpoint returns.
Exits with a nonzero code when a small set of pass/fail thresholds is broken.

You can run it against any free endpoint before you wire it into a CLI, sidecar, or batch job.

Start with a single curl smoke test

Create a deliberately small JSON payload first. A small payload means a timeout cannot hide behind long generation.

payload.json:

{"messages":[{"role":"user","content":"Reply with exactly: ok"}],"temperature":0}

Then run one request before you start the loop:

export PROBE_ENDPOINT="https://your-endpoint.example/v1/chat/completions"
export PROBE_API_KEY="your-key"
curl -sS -X POST "$PROBE_ENDPOINT" \
  -H "Authorization: Bearer $PROBE_API_KEY" \
  -H "Content-Type: application/json" \
  -d @payload.json | jq .

If the first request does not return a normal JSON body, stop. The probe cannot interpret an undocumented failure format.

Install the probe

The script needs Python 3.10 or newer.

python -m venv .venv && source .venv/bin/activate
pip install httpx

The probe script

Save this as probe_free_tier.py. It is intentionally single-file so you can read it in one sitting and edit it when the API response shape changes.

import asyncio
import json
import os
import sys
import time
from pathlib import Path

import httpx

ENDPOINT = os.environ['PROBE_ENDPOINT']
API_KEY = os.environ['PROBE_API_KEY']
PAYLOAD = json.loads(Path(os.environ.get('PROBE_PAYLOAD_FILE', 'payload.json')).read_text())
REQUESTS = int(os.environ.get('PROBE_REQUESTS', '300'))
CONCURRENCY = int(os.environ.get('PROBE_CONCURRENCY', '4'))
TIMEOUT = float(os.environ.get('PROBE_TIMEOUT', '15'))

sem = asyncio.Semaphore(CONCURRENCY)
status_counts: dict[int, int] = {}
latencies: list[float] = []
tokens: list[int] = []
errors: list[str] = []

def pct(xs: list[float], p: float) -> float | None:
    if not xs:
        return None
    s = sorted(xs)
    i = min(len(s) - 1, int((p / 100) * len(s) - 1))
    return round(s[max(0, i)], 3)

async def one(client: httpx.AsyncClient, i: int) -> None:
    async with sem:
        start = time.perf_counter()
        try:
            r = await client.post(
                ENDPOINT,
                headers={'Authorization': f'Bearer {API_KEY}'},
                json=PAYLOAD,
            )
            status_counts[r.status_code] = status_counts.get(r.status_code, 0) + 1
            if r.status_code == 200:
                try:
                    data = r.json()
                    usage = data.get('usage', {})
                    if isinstance(usage.get('total_tokens'), int):
                        tokens.append(usage['total_tokens'])
                except Exception:
                    pass
            else:
                errors.append(f'{i}: {r.status_code} {r.text[:120]}')
        except Exception as exc:
            errors.append(f'{i}: {type(exc).__name__} {exc}')
        finally:
            latencies.append(time.perf_counter() - start)

async def main() -> None:
    limits = httpx.Limits(max_connections=CONCURRENCY * 2)
    async with httpx.AsyncClient(timeout=TIMEOUT, limits=limits) as client:
        await asyncio.gather(*(one(client, i) for i in range(REQUESTS)))
    ok = status_counts.get(200, 0)
    total = REQUESTS
    p50 = pct(latencies, 50)
    p95 = pct(latencies, 95)
    print(f'ok={ok}/{total} status={status_counts}')
    print(f'p50={p50}s p95={p95}s max={max(latencies):.3f}s')
    if tokens:
        avg = sum(tokens) / len(tokens)
        print(f'token_observations={len(tokens)} mean={avg:.1f} estimated_total_burn={avg * total:.0f}')
    else:
        print('no token usage found in responses')
    if errors:
        print(f'errors={len(errors)}')
        for e in errors[:10]:
            print(f'  {e}')
    if ok / total < 0.98 or p95 is None or p95 > 8:
        sys.exit(2)
    print('probe passed: success rate >= 98% and p95 < 8s')

if __name__ == '__main__':
    asyncio.run(main())

Run the probe

Use a small run first to avoid burning a meaningful chunk of a 30 million token allowance on a broken loop.

export PROBE_REQUESTS=300
export PROBE_CONCURRENCY=4
python probe_free_tier.py

Then, if the first run passes, scale to a five-minute run:

export PROBE_REQUESTS=1000
export PROBE_CONCURRENCY=8
python probe_free_tier.py

Read the pass/fail table

Metric	Pass	Abandon or lower concurrency
HTTP 200 rate	>= 98%	< 98% or any non-transient 5xx
p95 latency	< 8s	missing p95 or p95 > 8s
Token usage	response includes `usage.total_tokens`	no token count means you cannot budget
429 responses	0 in the final run	any 429 means the free server is throttling you

The exact thresholds are arbitrary, but they are explicit. You can change them after the first run, but write the new numbers down before you retry.

What the probe does and does not prove

The probe proves whether the endpoint survived your payload and concurrency for the minutes you tested. It does not prove that 30 million tokens will last all month, that the free server has no cold start, that the model is the same later, or that the node you hit is the one you will hit tomorrow.

Because MonkeyCode is an open source project, you can inspect the server code and run a local copy for baseline tests. That is a useful escape hatch: if the hosted free server changes, you still have the source to evaluate, not a black-box dependency.

Who should not use this approach

Do not use a free server probe as your only test for payment processing, regulated personal data, long-context batch jobs, or any path with a hard latency objective. The probe is a preflight for a side project or a local tool, not a substitute for an SLA.

If you are evaluating MonkeyCode's current free tier and free server for a small build, start with this probe, read the official limits, and keep the script in your repo. A free server is useful when you know its failure shape before your code does.

Stop Skimming Build Logs: Route Shell Errors Through a Free Model Sidecar

Sam Rivera — Sun, 16 Aug 2026 02:45:09 +0000

You keep a mental cache of common stack traces: ENOENT, ModuleNotFoundError, exit code 1. The problem isn't understanding them; it's deciding which one matters when a tool emits forty lines and only two are diagnostic.

Why this is worth reading

A free model endpoint and a free server give you a cheap opportunity to move triage ahead of the moment you open a search engine. Instead of pasting the last three traceback lines from memory, you can send the full stdout and stderr of a failed command to a small sidecar that returns four focused fields: a likely cause, the exact evidence lines, a next command to try, and a confidence score.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is described by its operator as an open-source project with free model access, an advertised 30 million token allowance, and a free server option. I treat those availability claims as operator-supplied and verify only the workflow below; check the current model name, quota, and server terms against the project's primary documentation before you rely on them.

The sidecar is for your failed commands, not for watching the model

This is not another endpoint-health canary or token-burn alarm. Those tools tell you whether a free model is still up and how much credit remains. This sidecar is about what you do when the service is available but your command is not: it reads the failure you already have, asks for a structured triage, and gives you a narrower place to start. You still read the log; you just stop skimming the whole thing first.

Build the sidecar with only the standard library

The server and client are one Python file. The server accepts a failed command's stdout, stderr, and exit code; the client runs the command and only calls the model on non-zero exit. That keeps successful builds from spending tokens.

Create triage.py:

#!/usr/bin/env python3
'''Shell error triage sidecar.

Usage:
  python triage.py serve
  python triage.py run -- <command...>
'''
import argparse
import json
import os
import subprocess
import sys
import urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

TRIAGE_ENDPOINT = os.getenv('TRIAGE_ENDPOINT', 'http://127.0.0.1:8787')
MODEL_BASE_URL = os.getenv('MODEL_BASE_URL', '')
MODEL_ID = os.getenv('MODEL_ID', '')
API_KEY = os.getenv('MONKEY_CODE_API_KEY', '')


def ask_model(messages, max_tokens=250):
    '''Call an OpenAI-compatible chat completions route.'''
    payload = json.dumps({
        'model': MODEL_ID,
        'messages': messages,
        'temperature': 0,
        'max_tokens': max_tokens,
    }).encode()
    base = MODEL_BASE_URL.rstrip('/')
    url = f'{base}/chat/completions'
    request = urllib.request.Request(
        url,
        data=payload,
        headers={
            'Authorization': f'Bearer {API_KEY}',
            'Content-Type': 'application/json',
        },
    )
    with urllib.request.urlopen(request, timeout=30) as response:
        data = json.loads(response.read())
    return data['choices'][0]['message']['content']


def build_prompt(command, stdout, stderr, returncode):
    return f'''You are triaging a shell command that just failed.
Command: {command}
Exit code: {returncode}
Return JSON only with these keys: likely_cause, evidence_lines, next_command, confidence.
Use at most 15 evidence lines. Prefer exact traceback or stderr lines over prose.
Keep stdout and stderr truncated to the last 25 diagnostic lines.

STDOUT:
{stdout[-3000:] or '(empty)'}

STDERR:
{stderr[-3000:] or '(empty)'}
'''


class TriageHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = int(self.headers.get('Content-Length', 0))
        raw = self.rfile.read(length)
        try:
            body = json.loads(raw)
            prompt = build_prompt(
                body['command'],
                body['stdout'],
                body['stderr'],
                body['returncode'],
            )
            reply = ask_model([{'role': 'user', 'content': prompt}])
            try:
                result = json.loads(reply)
            except json.JSONDecodeError:
                result = {
                    'likely_cause': 'Model did not return JSON',
                    'evidence_lines': [reply],
                    'next_command': '',
                    'confidence': 0,
                }
        except Exception as exc:  # noqa: BLE001
            result = {
                'likely_cause': 'Triage service error',
                'evidence_lines': [str(exc)],
                'next_command': '',
                'confidence': 0,
            }

        output = json.dumps(result).encode()
        self.send_response(200)
        self.send_header('Content-Type', 'application/json')
        self.send_header('Content-Length', str(len(output)))
        self.end_headers()
        self.wfile.write(output)

    def log_message(self, format, *args):  # noqa: A002
        pass


def run_command(args):
    process = subprocess.run(args, capture_output=True, text=True)
    if process.returncode == 0:
        print('Command succeeded; no triage needed.')
        return 0

    payload = json.dumps({
        'command': ' '.join(args),
        'stdout': process.stdout,
        'stderr': process.stderr,
        'returncode': process.returncode,
    }).encode()
    endpoint = TRIAGE_ENDPOINT.rstrip('/')
    url = f'{endpoint}/triage'
    request = urllib.request.Request(
        url,
        data=payload,
        headers={'Content-Type': 'application/json'},
    )
    with urllib.request.urlopen(request, timeout=60) as response:
        result = json.loads(response.read())
    print(json.dumps(result, indent=2))
    return process.returncode


def main():
    parser = argparse.ArgumentParser()
    subparsers = parser.add_subparsers(dest='mode', required=True)

    subparsers.add_parser('serve', help='Run the triage server')

    run_parser = subparsers.add_parser('run', help='Run a command and triage on failure')
    run_parser.add_argument('args', nargs=argparse.REMAINDER, help='Command and its arguments after --')

    args = parser.parse_args()

    if args.mode == 'serve':
        server = ThreadingHTTPServer(('0.0.0.0', 8787), TriageHandler)
        print('Triage server listening on :8787')
        server.serve_forever()
    else:
        if not args.args or args.args == ['--']:
            sys.exit('Provide -- followed by a command to run.')
        return run_command(args.args)


if __name__ == '__main__':
    main()

Run the server on the free server option

Copy the file to the free server option or any host with outbound network access, then export the three model variables and start the server:

# On the free server option:
cat > /tmp/triage.py <<'PY'
# paste the full triage.py content above
PY
export MONKEY_CODE_API_KEY='...'
export MODEL_BASE_URL='https://api.example.com'
export MODEL_ID='your-model-id'
python3 /tmp/triage.py serve

If the API you are testing is not OpenAI-compatible, change the URL construction and request shape inside ask_model only. The rest of the sidecar stays the same.

Point your local machine at the sidecar

On your laptop, set the same model variables plus the triage endpoint that points at the free server:

export TRIAGE_ENDPOINT='http://your-free-server:8787'
export MODEL_BASE_URL='https://api.example.com'
export MODEL_ID='your-model-id'
export MONKEY_CODE_API_KEY='...'
python3 triage.py run -- npm test

If the free server cannot accept inbound connections from your development machine, run the server locally instead and set TRIAGE_ENDPOINT to http://127.0.0.1:8787.

What a triage result looks like

Run a deliberately messy command and let the sidecar compress the noise:

python3 triage.py run -- sh -c "cat /etc/shadow; ls /nope; python3 -c 'import missing_thing'"

The output is not a repaired command. It is a structured starting point:

{
  "likely_cause": "Multiple failures: permission denied on /etc/shadow, missing directory /nope, ModuleNotFoundError for missing_thing",
  "evidence_lines": [
    "cat: /etc/shadow: Permission denied",
    "ls: cannot access '/nope': No such file or directory",
    "ModuleNotFoundError: No module named 'missing_thing'"
  ],
  "next_command": "python3 -c 'import missing_thing'",
  "confidence": 0.91
}

You still read the evidence lines. The model's job is to move the two or three real failure lines to the top of your attention instead of leaving them buried under dependency output.

Test it with a known failure

Before you trust the sidecar on a real build, run a one-shot assertion against a predictable import error:

python3 - <<'PY'
import json, subprocess, sys
proc = subprocess.run(
    [sys.executable, 'triage.py', 'run', '--', 'python3', '-c', 'import definitely_missing_42'],
    capture_output=True,
    text=True,
)
assert proc.returncode != 0
out = json.loads(proc.stdout)
assert 'ModuleNotFoundError' in ' '.join(out['evidence_lines'])
print('triage result:', out)
PY

If the assertion passes, you know the sidecar can extract at least one common Python failure. Add a second fixture for a missing environment variable or a port-binding conflict if those matter more in your stack.

Keep the token budget small

Each failed command sends up to 6 KB of combined output and requests 250 output tokens. That is deliberately small: a day with 100 failed runs would use roughly 600 KB of input context plus 25,000 output tokens, depending on the tokenizer. With a free 30 million token allowance, the sidecar is a low-usage utility, but verify the current quota and model behavior against the provider's documentation rather than assuming the allowance stays fixed.

Limitations and when to skip the sidecar

Do not run the returned next_command without reading it first. The model can misidentify a rare compiler error or suggest a fix that works only in a container it cannot see. The sidecar also truncates long logs to the last 3,000 characters, so a root cause buried earlier in a multi-step pipeline can be missed.

Skip this workflow if you need deterministic log parsing with audit requirements, if you cannot send command output to a third-party endpoint, or if you already have a local grep pattern that catches 90% of your failures in under a second. The model is a useful second pass, not a replacement for a small collection of known-error regexes.

A failed command should earn its model call

The sidecar does not watch the model; it uses the model only after something fails. That distinction keeps successful builds cheap and gives you a concrete, repeatable reason to try MonkeyCode's free model access and free server option without betting your whole workflow on them. If you add this to your own shell, which failure pattern should it learn first: missing environment variables, dependency conflicts, or port-binding errors?

Build a Fail-Open Shim for a Free Model Endpoint That Goes Quiet

Sam Rivera — Sat, 15 Aug 2026 20:06:39 +0000

Why this is worth reading

A free model endpoint is the most useful dependency you should never trust. It can throttle, hang, or return structurally invalid JSON, and if that call sits inside a CLI or batch script, one 429 can stop a run that was otherwise fine. The useful fix is not a bigger prompt or a paid fallback; it is a small adapter that places the model behind the same failure discipline you would apply to any external API.

In this walkthrough, you will build a TypeScript shim with a timeout, schema check, usage log, and deterministic fallback. The result is copyable into any Node CLI and gives you a clean exit when the free endpoint stops answering.

MonkeyCode's operator describes the project as open source and lists a free model tier with a 30 million token allowance and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I did not run a long-term audit of the quota or server lifetime, so I treat both as claims to verify rather than constants. The shim below does not depend on them; it only assumes that the endpoint will eventually fail for reasons other than token count.

The failure shape you are isolating

When you call a model endpoint from a CLI, three problems tend to arrive at the same time:

Timeout — the provider is overloaded or the network path is slow, so your script hangs.
Structural failure — the response is HTTP 200 but the content is not the JSON shape your next step expects.
Quota or availability failure — the endpoint returns 429 or 500 and your script has no retry or fallback path.

Your adapter should not need to know which one happened to keep running. It should make one attempt, enforce a deadline, validate what came back, record what it used, and return a stable local result when the model path fails.

Define the contract

Start with a small TypeScript file that defines the model provider and the fallback decision shape. You do not need a framework; fetch, AbortController, and a few type guards are enough.

type ChatMessage = {
  role: 'system' | 'user' | 'assistant';
  content: string;
};

type ModelProvider = {
  name: string;
  endpoint: string;
  key?: string;
  timeoutMs: number;
  maxTokens: number;
  schemaCheck?: (value: unknown) => boolean;
};

type CallResult<T> = {
  result: T;
  source: 'model' | 'fallback';
  usage?: Record<string, number>;
  error?: string;
  latencyMs: number;
};

The interface is deliberately small. You can swap the provider later without touching the rest of your CLI, which matters when a free endpoint changes its base URL or authentication style.

Parse the model response carefully

Chat-style endpoints often wrap the assistant answer inside choices[0].message.content. That content may be a JSON string or plain text. Your parser should not throw on the first mismatch; it should return a failed parse that the fallback path can handle.

function parseModelContent(raw: unknown): { ok: true; value: unknown } | { ok: false; error: string } {
  if (!raw || typeof raw !== 'object') {
    return { ok: false, error: 'non-object response' };
  }

  const obj = raw as Record<string, unknown>;
  const content = obj.choices?.[0]?.message?.content;

  if (typeof content !== 'string') {
    return { ok: false, error: 'missing choices[0].message.content' };
  }

  try {
    return { ok: true, value: JSON.parse(content) };
  } catch {
    return { ok: true, value: content };
  }
}

function extractUsage(raw: unknown): Record<string, number> | undefined {
  if (!raw || typeof raw !== 'object') return undefined;
  const obj = raw as Record<string, unknown>;
  const usage = obj.usage as Record<string, unknown> | undefined;
  if (!usage) return undefined;

  return Object.fromEntries(
    Object.entries(usage).filter((entry): entry is [string, number] => typeof entry[1] === 'number')
  );
}

Add the fail-open call

The core function makes one attempt, aborts it after the configured timeout, and always returns a result. The fallback is what keeps your batch job alive.

async function callWithFallback<T>(
  provider: ModelProvider,
  messages: ChatMessage[],
  fallback: (messages: ChatMessage[]) => T
): Promise<CallResult<T>> {
  const started = Date.now();
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), provider.timeoutMs);

  try {
    const response = await fetch(provider.endpoint, {
      method: 'POST',
      headers: {
        'content-type': 'application/json',
        ...(provider.key ? { authorization: `Bearer ${provider.key}` } : {})
      },
      body: JSON.stringify({
        messages,
        max_tokens: provider.maxTokens
      }),
      signal: controller.signal
    });

    clearTimeout(timer);

    if (!response.ok) {
      return {
        result: fallback(messages),
        source: 'fallback',
        error: `HTTP ${response.status}`,
        latencyMs: Date.now() - started
      };
    }

    const raw = await response.json();
    const parsed = parseModelContent(raw);

    if (!parsed.ok || (provider.schemaCheck && !provider.schemaCheck(parsed.value))) {
      return {
        result: fallback(messages),
        source: 'fallback',
        error: parsed.ok ? 'schema mismatch' : parsed.error,
        latencyMs: Date.now() - started
      };
    }

    return {
      result: parsed.value as T,
      source: 'model',
      usage: extractUsage(raw),
      latencyMs: Date.now() - started
    };
  } catch (err) {
    clearTimeout(timer);
    return {
      result: fallback(messages),
      source: 'fallback',
      error: err instanceof Error ? err.name : String(err),
      latencyMs: Date.now() - started
    };
  }
}

Notice that an abort error and an HTTP 500 both end up in the fallback branch. That is deliberate. At 2 a.m., you usually care that the run continued, not which exact transport error occurred.

Make the fallback honest

The fallback should not pretend to be a model. It should return a stable shape that your CLI can detect and act on. For a canary task, a local echo with fallback: true is enough.

function fallbackReply(messages: ChatMessage[]) {
  const lastUserMessage = messages
    .slice()
    .reverse()
    .find((message) => message.role === 'user');

  return {
    ok: false,
    fallback: true,
    reason: 'model_unavailable',
    echo: lastUserMessage?.content.slice(0, 140) ?? ''
  };
}

If your next step requires a real model decision, this fallback will not satisfy it. In that case, your CLI should stop instead of continuing silently. A fail-open shim is only correct when a deterministic local result is acceptable for the task.

Wire up a canary command

Add a simple command-line path so you can test the adapter without building the rest of the application.

const isTaskResult = (value: unknown): boolean => {
  if (!value || typeof value !== 'object') return false;
  return 'ok' in (value as Record<string, unknown>);
};

const provider: ModelProvider = {
  name: 'free-model',
  endpoint: process.env.MODEL_ENDPOINT ?? '',
  key: process.env.MODEL_KEY,
  timeoutMs: Number(process.env.MODEL_TIMEOUT_MS ?? 8000),
  maxTokens: Number(process.env.MODEL_MAX_TOKENS ?? 256),
  schemaCheck: isTaskResult
};

if (process.argv.includes('--canary')) {
  const result = await callWithFallback(
    provider,
    [
      { role: 'system', content: 'Return JSON with an ok boolean.' },
      { role: 'user', content: 'canary' }
    ],
    fallbackReply
  );

  console.log(JSON.stringify(result, null, 2));
}

Run it with a short timeout and an empty endpoint to confirm the fallback path fires.

npm init -y
npm install typescript tsx @types/node
MODEL_ENDPOINT=http://127.0.0.1:9 MODEL_TIMEOUT_MS=1000 npx tsx free-model-shim.ts --canary

The expected output has source: 'fallback' and an error such as AbortError or fetch failed. That is the clean exit you are designing for.

Test against a local mock server

A real free endpoint can change under you, so a reproducible local mock is more useful than a live URL for the first test pass. This mock reads a failure mode from the environment and does nothing else.

// mock-model-server.mjs
import http from 'node:http';

const mode = process.env.FAIL_MODE ?? 'ok';

const server = http.createServer((req, res) => {
  res.setHeader('content-type', 'application/json');

  if (mode === 'timeout') {
    setTimeout(() => res.end(), 30000);
    return;
  }

  if (mode === 'bad-json') {
    res.end(JSON.stringify({ choices: [{ message: { content: '{"ok":' } }] }));
    return;
  }

  if (mode === 'http-500') {
    res.statusCode = 500;
    res.end(JSON.stringify({ error: 'upstream exploded' }));
    return;
  }

  res.end(
    JSON.stringify({
      choices: [{ message: { content: '{"ok":true,"text":"canary"}' } }],
      usage: { prompt_tokens: 12, completion_tokens: 4, total_tokens: 16 }
    })
  );
});

server.listen(Number(process.env.PORT ?? 8787));

Start the mock in one terminal, then run the canary from another.

node mock-model-server.mjs &
MODEL_ENDPOINT=http://127.0.0.1:8787 MODEL_TIMEOUT_MS=1000 FAIL_MODE=timeout npx tsx free-model-shim.ts --canary

The test matrix you should record looks like this:

FAIL_MODE	Expected source	Expected error
timeout	fallback	AbortError
bad-json	fallback	schema mismatch or JSON parse
http-500	fallback	HTTP 500
ok	model	none

If any row returns source: 'model' when it should not, your parser or schema check is too permissive.

What this shim will not catch

This is not a production model gateway. It will not catch semantic drift, prompt-injection attempts against tool calls, streaming responses, or changes to the provider's data-retention policy. It also does not tell you whether the 30 million token allowance is still accurate; for that you need a separate token ledger and periodic quota audit. The shim only keeps a single request from taking your CLI down.

Do not use fail-open behavior when the model result is required for a decision. If you are labeling user data, approving a destructive command, or generating paid output, a silent local fallback can be worse than a crash. In those paths, make the fallback fail loudly or require a --strict flag.

When to use it

Use this pattern for side-project automation where one model call should not stop the rest of the run: release-note drafts, changelog categorization, small batch summaries, internal triage notes, or a canary that checks whether an endpoint is awake before a longer job starts.

The shim is intentionally provider-neutral. If you want to try it against a real free endpoint instead of the mock later, the open-source MonkeyCode project's free model access and free server option are a reasonable canary target. Just verify the current token limit and server constraints directly before you wire it into anything long-running.

A Two-Prompt Watchdog for the Free Model Endpoint You Already Rely On

Sam Rivera — Sat, 15 Aug 2026 07:05:52 +0000

Here is why this is worth reading: the current AI news cycle keeps circling around detecting generated text and making model outputs identifiable, but a solo builder with a free endpoint has a smaller version of that same problem. You do not need to watermark text. You need to know when the model behind your endpoint changed the shape of its answer and broke your parser. This guide builds a two-prompt watchdog that detects that drift, using MonkeyCode's free model access and the free server option you can leave running without adding a paid host.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The problem is not the model, it is the silent change

Free model endpoints frequently change routing, quantization, context handling, and instruction-following behavior without a changelog. You notice it when your JSON parsing fails, or your shell helper starts treating a text response as a command. The failure is often not the model being "wrong." It is the output contract drifting.

For a solo project, you rarely need a heavy observability stack. You need a cheap check that runs on a schedule, sends the same two inputs every time, and tells you when the normalized output no longer matches the previous run.

What the watchdog does

The watchdog is deliberately small:

It sends two fixed prompts to the configured model endpoint.
It normalizes each response with trim().toLowerCase().
It computes a short SHA-256 digest, not the raw output, so history files stay small.
It records whether a required JSON key survived in the response.
If the digest or key status changes from the previous run, the shell wrapper prints a diff and can send a notification.

This is not a benchmark, so it will not tell you whether one model is better than another. It only tells you whether the endpoint you depend on changed in a way that your application should care about.

Build the probe script

Create a small Node.js project with TypeScript:

mkdir drift-watchdog && cd drift-watchdog
npm init -y
npm install -D typescript @types/node tsx

Save this as probe.ts:

import { createHash } from "node:crypto";

type Probe = {
  id: string;
  prompt: string;
  expectKey?: string;
};

const probes: Probe[] = [
  {
    id: "strict-json",
    prompt:
      'Return only JSON with keys "city" and "temp_c". No prose.',
    expectKey: "temp_c",
  },
  {
    id: "http-method",
    prompt:
      "Answer with exactly one word: the HTTP method used to update a resource.",
  },
];

const endpoint = process.env.MONKEYCODE_API_URL;
const token = process.env.MONKEYCODE_TOKEN;

async function callModel(prompt: string): Promise<string> {
  if (!endpoint || !token) {
    throw new Error("Set MONKEYCODE_API_URL and MONKEYCODE_TOKEN");
  }

  const response = await fetch(endpoint, {
    method: "POST",
    headers: {
      "content-type": "application/json",
      authorization: `Bearer ${token}`,
    },
    body: JSON.stringify({
      prompt,
      max_tokens: 24,
      temperature: 0,
    }),
  });

  if (!response.ok) {
    throw new Error(`HTTP ${response.status}`);
  }

  return (await response.text()).trim();
}

function digest(value: string): string {
  return createHash("sha256")
    .update(value.toLowerCase())
    .digest("hex")
    .slice(0, 12);
}

async function main() {
  const results: string[] = [];

  for (const probe of probes) {
    const output = await callModel(probe.prompt);
    const shortDigest = digest(output);
    const keyStatus = probe.expectKey
      ? output.includes(probe.expectKey)
        ? "key-present"
        : "key-missing"
      : "-";

    results.push(`${probe.id}\t${shortDigest}\t${keyStatus}`);
  }

  console.log(results.join("\n"));
}

main().catch((error) => {
  console.error(
    `probe failed: ${error instanceof Error ? error.message : error}`,
  );
  process.exit(1);
});

Set your endpoint and token as environment variables before running:

export MONKEYCODE_API_URL="your-monkeycode-endpoint"
export MONKEYCODE_TOKEN="your-token"
npx tsx probe.ts

You should see two rows similar to:

strict-json  9f3b2c1a4e6d  key-present
http-method  7d1a9e0b3c5f  -

This is a ready-to-adapt scaffold, not a guarantee of a specific output from any live endpoint. Adjust the prompts to match your own account before trusting the result.

Add a failure fixture you can verify

To prove the watchdog catches the failure mode you care about, change the expected key to one the model is unlikely to return. For example, set expectKey to "humidity_pct" on the strict JSON probe:

expectKey: "humidity_pct",

Run the probe again. If the model returns the original keys, the status becomes key-missing, the digest row differs, and the shell wrapper flags a drift. That is the same signal you want when a model silently stops respecting "Return only JSON" or renames a field you parse downstream.

Change expectKey back to "temp_c" before deploying.

Run it on a free server

Put the project on a small free server that can stay online. A cron job every six hours is enough for a change detector; you are not monitoring availability, you are monitoring behavior.

Create drift-watchdog.sh:

#!/usr/bin/env bash
set -euo pipefail

LOG_FILE="${LOG_FILE:-./probe-results.tsv}"
STATE_FILE="${STATE_FILE:-./probe-state.tsv}"

npx tsx probe.ts > "$LOG_FILE"

if [[ -f "$STATE_FILE" ]]; then
  if ! cmp -s "$STATE_FILE" "$LOG_FILE"; then
    echo "drift detected" >&2
    diff "$STATE_FILE" "$LOG_FILE" || true

    if [[ -n "${NTFY_TOPIC:-}" ]]; then
      curl -s -d "Model drift detected on $(hostname)" \
        "https://ntfy.sh/$NTFY_TOPIC" || true
    fi
  fi
fi

cp "$LOG_FILE" "$STATE_FILE"

Make it executable and add a cron entry:

chmod +x drift-watchdog.sh
crontab -e

Add:

0 */6 * * * cd $HOME/drift-watchdog && LOG_FILE=$HOME/watchdog-runs/last.tsv STATE_FILE=$HOME/watchdog-runs/state.tsv ./drift-watchdog.sh >> $HOME/watchdog-runs/watchdog.log 2>&1

Create the log directory first:

mkdir -p $HOME/watchdog-runs

The state file grows to only a few hundred bytes, no matter how many runs you perform.

Cost boundary and abandonment criteria

With two prompts capped at 24 tokens each, a run uses at most 48 output tokens plus the fixed prompt tokens. Six runs per day gives roughly 360 output tokens daily, or about 10,800 tokens in a 30-day month. If MonkeyCode's advertised allowance at the time of writing is 30 million tokens, this watchdog consumes less than 0.04% of the trial capacity each month, leaving the rest for actual work.

Set two exit conditions before you rely on this:

If a probe ever uses more than 5% of your monthly allowance, stop and reduce the frequency to twice a day.
If the endpoint returns a non-JSON error for three runs in a row, pause the watchdog and test the endpoint manually. The watchdog's job is behavior change, not endpoint health; a broken endpoint should fail your application checks anyway.

Who should not use this

Do not use this if you need to compare model quality, measure latency under load, or detect content policy changes. A two-prompt digest is too thin for those questions. Do not use it as an availability monitor; a cron job every six hours will miss the small blips you can tolerate and catch the long behavior changes that matter.

The one thing to try today

Take the two prompts you already depend on in your own side project, wrap them in a digest comparison, and run the first state snapshot. If the next scheduled run is identical, you have a working baseline. If it is not, you found a drift before your user did.

The free endpoint and free server from MonkeyCode are useful here because the experiment costs days, not dollars. Run the watchdog with a small token budget, keep the state file under version control, and treat every diff as a release note the endpoint did not publish.

What is the smallest output contract in your current project that would hurt if it silently changed?

Build a Token Ledger Before You Burn Through a Free Model Tier

Sam Rivera — Sat, 15 Aug 2026 00:35:52 +0000

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why this is worth reading: a free model endpoint with a large token allowance is a good place to validate a new CLI workflow, but it can burn through the allowance in a single retry loop before you notice. I built a small stateful budget guard that checks the projected cost before the call, records actual usage after the call, and refuses to touch the ledger when the endpoint sends an unexpected response. It works as a disposable first pass on a free endpoint and leaves you a clean exit when the shape changes.

MonkeyCode's outreach describes an open-source project with a free model route and a free hosted server. I do not treat either as a permanent dependency. I treat them as a test target: an endpoint I can call without a contract while I am still changing prompts, timeouts, and schemas. The tool below is independent of MonkeyCode's exact model list; it assumes only an OpenAI-style chat completion path and usage accounting in the response. Swap one function if the free server does not follow that shape.

The problem with a free allowance

Most model dashboards report aggregate usage after the fact. That is enough for casual work, but it is not enough when you wire an endpoint into a loop. I have seen two avoidable failures in my own drafts. A retry-on-timeout wrapper restarted a slow request four times before the first response arrived, multiplying total token spend. A long context buffer kept sending the same 6k-token history on every turn because I forgot to trim old messages. The dashboard showed the total drop, but not which call caused it.

A local ledger fixes that by refusing to send the request when the projected total exceeds the budget. It does not replace the provider dashboard. It makes the decision before the endpoint gets a chance to consume tokens.

The artifact

The script below does three jobs:

load a budget and already-used amount from a JSON file
make a conservative preflight estimate for the next call
record the actual usage returned by the endpoint and save the ledger atomically

Preflight is deliberately rough: prompt bytes divided by four, plus the requested max response tokens, plus a 15 percent margin. That is not tokenizer-accurate for non-English text or code-heavy prompts, but it is intentionally conservative because the goal is to stop accidental waste, not to replace metering. If you need precise preflight numbers, add a local tokenizer for the model you are calling.

python -m venv .venv && source .venv/bin/activate
pip install httpx

# .env
MONKEYCODE_BASE_URL=https://your-free-server.example.com/v1
MONKEYCODE_API_KEY=your-key
MODEL_NAME=the-current-free-model
TOKEN_BUDGET=30000000
MAX_RESPONSE_TOKENS=256
TIMEOUT_S=30
LEDGER_PATH=token_ledger.json

Then source it and call:

set -a; source .env; set +a
python budgeted_call.py 'Summarize this connection error in one sentence.'

budgeted_call.py:

#!/usr/bin/env python3
import json
import os
import sys
import time
from pathlib import Path

import httpx

BASE_URL = os.getenv('MONKEYCODE_BASE_URL', '').rstrip('/')
API_KEY = os.getenv('MONKEYCODE_API_KEY', '')
MODEL = os.getenv('MODEL_NAME', '')
BUDGET = int(os.getenv('TOKEN_BUDGET', '30000000'))
LEDGER = Path(os.getenv('LEDGER_PATH', 'token_ledger.json'))
MAX_TOKENS = int(os.getenv('MAX_RESPONSE_TOKENS', '256'))
TIMEOUT_S = float(os.getenv('TIMEOUT_S', '30'))


def load_ledger():
    if LEDGER.exists():
        data = json.loads(LEDGER.read_text())
        return int(data.get('used', 0))
    return 0


def save_ledger(used):
    tmp = LEDGER.with_suffix('.tmp')
    tmp.write_text(json.dumps({'used': used, 'updated': int(time.time())}, indent=2))
    tmp.replace(LEDGER)


def preflight_estimate(prompt, max_tokens):
    prompt_tokens = len(prompt.encode('utf-8')) // 4
    return prompt_tokens + max_tokens


def run(prompt):
    if not BASE_URL or not API_KEY or not MODEL:
        sys.exit('Set MONKEYCODE_BASE_URL, MONKEYCODE_API_KEY, and MODEL_NAME first.')

    used = load_ledger()
    estimate = preflight_estimate(prompt, MAX_TOKENS)
    margin = int(estimate * 0.15)
    projected = used + estimate + margin

    if projected > BUDGET:
        sys.exit(f'Blocked: projected={projected} used={used} budget={BUDGET}. Shorten the prompt or raise the budget.')

    response = httpx.post(
        f'{BASE_URL}/chat/completions',
        headers={'Authorization': f'Bearer {API_KEY}'},
        json={
            'model': MODEL,
            'messages': [
                {'role': 'system', 'content': 'Answer concisely. Return JSON only when asked.'},
                {'role': 'user', 'content': prompt},
            ],
            'max_tokens': MAX_TOKENS,
        },
        timeout=TIMEOUT_S,
    )
    response.raise_for_status()
    payload = response.json()

    usage = payload.get('usage') or {}
    total = usage.get('total_tokens')
    if total is None:
        total = int(usage.get('prompt_tokens', 0)) + int(usage.get('completion_tokens', 0))

    if total <= 0:
        sys.exit('Endpoint returned no usable token count; ledger was not updated.')

    used += total
    save_ledger(used)

    content = payload['choices'][0]['message']['content']
    print(json.dumps({
        'text': content,
        'total_tokens': total,
        'used': used,
        'remaining': BUDGET - used,
    }, indent=2))


if __name__ == '__main__':
    run(sys.argv[1] if len(sys.argv) > 1 else 'Reply with the word pong.')

Test the guard before you trust it

Use a failure fixture that does not hit the real endpoint. The expected result is a non-zero exit and an unchanged ledger.

MONKEYCODE_BASE_URL=http://127.0.0.1:9 python budgeted_call.py 'ping'

If you want a decision table for a canary suite, keep the checks tiny:

Scenario	Expected exit	Ledger change
Missing base URL or model	non-zero	none
Unreachable endpoint or timeout	non-zero	none
Projected spend over budget	non-zero	none
Valid response with `usage.total_tokens`	zero	`used` increases
Response without a usable token count	non-zero	none

I run this once before I allow any larger script to call the endpoint. A failed run tells me which part of the integration changed instead of leaving me to guess from a balance chart.

Where the free server fits

For a solo build, the free server is most useful as a canary target, not as a permanent backend. I point this script at the free route first, keep the model name in an environment variable, and store all results in the local ledger. If the endpoint changes one day, the only change is a URL or model name. If the endpoint reports different usage fields, the script stops instead of silently undercounting.

I also set a hard mental exit: if the free endpoint is slow enough that I need a timeout above 30 seconds, it is not ready for the actual CLI. The ledger cannot fix latency; it only prevents it from getting expensive while I measure.

One important context about the 30,000,000-token figure

The reference I was given describes a free tier with a 30,000,000-token allocation and a free server option. I do not verify quota pages as part of a code article, so I keep the number as TOKEN_BUDGET in an environment variable rather than hard-coding it. Check the current page before you rely on that number; if the allocation is different today, the script does not need to change.

Who should not use this

Do not use this local ledger if you need concurrent workers sharing one budget, hard SLOs on latency, audit trails, or compliance review for private data. A single JSON file is not concurrency-safe, and a free endpoint is the wrong home for sensitive prompts. Use the ledger as a canary, not as your production accounting system.

Next iteration

If you run this against a free server, tell me which usage fields the response actually returned. That determines whether the missing-usage guard is protecting you or getting in your way: total_tokens only, split prompt_tokens and completion_tokens, or something else entirely.

If you have a MonkeyCode free server route, plug it into this script first; if not, the same budget guard works with any endpoint that returns usage.

Audit Your Token Burn Before a Free Model Endpoint Hosts Your Next Side Project

Sam Rivera — Fri, 14 Aug 2026 18:20:02 +0000

Why this is worth reading

This week's AI threads keep circling two themes: agents that need tool calls, and free model endpoints that behave differently once real traffic hits. Useful as those discussions are, they rarely tell you the number that matters for a small project: how many tokens do you actually send in a week? The rest of this article gives you a repeatable way to measure that from your existing logs. You'll build a small Python token auditor, compare the result against an advertised 30 million token allowance, and deploy a small canary server if a free server option is available. The goal is not to pick a vendor; it is to stop making an infrastructure decision on vibes.

Separate the offer from the assumption

The open-source MonkeyCode project advertises a free access tier with 30 million tokens and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Treat the token figure as a cap to verify against your own traffic, not as a promise that your exact payload patterns will clear. Free tiers often fail for reasons that have nothing to do with the headline token count: request size limits, tool-call schema overhead, rate limits per minute, queue time on shared hardware, or a model refusing malformed JSON after one retry. Before you point a side project at any free endpoint, the useful move is to measure your normal request shape and then run one deliberate failure fixture.

Step 1: Capture your actual request shape

The audit starts with data you probably already have. If your CLI or agent logs requests as JSONL, export the last 500 records to a file named requests.jsonl. If you don't have a log yet, add one line to your request loop:

import json
log = open("requests.jsonl", "a")
log.write(json.dumps(payload) + "\n")

Keep the file local, sanitize keys, and don't commit it. The script below only needs one field per line, and it serializes the full record with ensure_ascii=False so non-English text isn't double-counted. The sample record should include the fields you actually send: prompt, tools, history, and system.

Step 2: Count tokens from your own logs

Exact token counts differ by tokenizer, so use a stable approximation first. The cl100k_base encoding is a reasonable baseline for many current instruction models, but it is an estimate. You are looking for order-of-magnitude truth: 4,000 tokens per request vs 40,000 changes the conclusion.

Create token_audit.py:

#!/usr/bin/env python3
import argparse, json
from pathlib import Path

def counts(path):
    import tiktoken
    enc = tiktoken.get_encoding("cl100k_base")
    total = 0
    records = []
    for line in Path(path).read_text().splitlines():
        if not line.strip():
            continue
        try:
            obj = json.loads(line)
        except json.JSONDecodeError:
            continue
        n = len(enc.encode(json.dumps(obj, ensure_ascii=False)))
        records.append(n)
        total += n
    return total, records

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("path")
    args = parser.parse_args()
    total, records = counts(args.path)
    if not records:
        raise SystemExit("no JSONL rows found")
    avg = total / len(records)
    peak = max(records)
    print(f"{len(records)} requests, {total} tokens total")
    print(f"avg {avg:.0f} tokens/request, peak {peak}")

Run it with:

pip install tiktoken
python token_audit.py requests.jsonl

If the peak is more than 5x the average, a handful of large requests will define your quota use. That matters more than the average when you pick a free tier, because rate limits often trigger at the edge, not the median.

Step 3: Compare to the allowance

Take the advertised 30 million token allowance as a starting number, not a permanent SLA. If your sample covers three normal days, project:

daily_tokens = total * 30 / sampled_days
runway_days = allowance / daily_tokens

For example, 900 requests over three days with a 10,000-token average gives 3 million tokens per day, or ten days per 30 million token allowance. That is enough for a deliberate canary or a soft launch, but not for an unattended week of agent traffic. You should also account for tool schemas. A function-calling payload can easily add 500-1,000 tokens to each request even when the user prompt is short, and retries multiply that cost. If you don't have tool-call data yet, run the next step with an oversized tool list and mark the result as a planning upper bound.

Step 4: Put the evaluator on the free server

A laptop is an unreliable place to run a comparison because it sleeps, switches networks, and disappears. If your MonkeyCode account provides the advertised free server option, that gives you a stable host for a tiny evaluator. The following FastAPI server exposes one endpoint that returns token size, content length, and a failure flag for a payload. It is deliberately small enough to read in full and easy to abandon if the server option changes.

from fastapi import FastAPI, Request
from pydantic import BaseModel
import json

app = FastAPI()

class Payload(BaseModel):
    text: str
    tool_schema: dict = {}

@app.post("/audit")
async def audit(payload: Payload):
    import tiktoken
    enc = tiktoken.get_encoding("cl100k_base")
    full = json.dumps(payload.dict(), ensure_ascii=False)
    tokens = len(enc.encode(full))
    return {
        "token_estimate": tokens,
        "content_length": len(full),
        "over_8k": tokens > 8000,
    }

Deploy it with uvicorn audit_server:app --host 0.0.0.0 --port 8000, then call it from another machine. You now have a fixed, observable target for a request that your side project would actually send. This is not a benchmark; it's a consistency check that catches drift in payload size and hidden tool-call bloat.

Step 5: Run one failure fixture

After the token audit, send one malformed request and log the outcome. The minimum fixture is a tool call whose required argument is missing:

{"model": "your-model", "prompt": "run a search", "tools": [{"name": "search", "required": ["query"]}], "tool_calls": [{"name": "search", "arguments": {}}]}

What you want is a clear error inside a few seconds. What you don't want is a 40-second hang followed by an empty 200 response, because that will poison every retry loop you write later. Record the result in the same log file with a fixture field so the decision stays tied to evidence.

When you should not use this

Skip the switch if one request regularly exceeds 30,000 tokens, if your agent uses nested tool calls with long histories, or if you can't tolerate a few hours of queueing on shared hardware. Those constraints do not mean the free endpoint is bad; they mean your workload will spend more time fighting a quota than shipping. A small CLI canary or a same-day model announcement probe is a better fit. For that shape of work, the 30 million token allowance and free server option are worth a trial only after the audit says you have multiple days of runway.

The number to write down

Write down three values before you try any free endpoint: median request tokens, peak tokens, and tool-schema overhead. If you run this audit on your own logs, the useful next input for me is which field dominated the count in your case—the prompt, the tool schema, or the conversation history. That determines the next knob to keep the free tier from becoming a spike trap.

Build a Budget Watchdog for a Free Model Endpoint Before It Goes Quiet

Sam Rivera — Fri, 14 Aug 2026 14:24:10 +0000

Why this is worth reading: a free model endpoint and a free server are easy to adopt, but the failure that actually wakes you up is not the first request. It is the 3 a.m. cron job that starts returning a 429, quietly falls back to garbage, and makes your dashboard lie. This guide gives you a small watchdog that treats quota exhaustion as a normal state, degrades to a cached answer, and writes a ledger you can read before you trust the word "free".

To make this concrete, I used MonkeyCode's free model access and free server option as the deployment target. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The broader MonkeyCode project is open source, but this guide treats the free options as deployment constraints, not as a review of the repository. The 30 million token allowance is the operator-supplied current number; treat quotas as moving targets and verify them in the console before you rely on them.

The useful part here is not another endpoint probe. You already have the canary harness and the disposable sandbox. This watchdog assumes the endpoint is connected and asks a different question: what happens when the free tokens run out during a normal Tuesday run?

Design for the edge, not the happy path

Your service only has two jobs. First, it must return a useful summary or a clearly labeled fallback instead of a raw upstream error. Second, it must record which path it took so you can decide whether the free tier is still worth it. If you only instrument the success path, you will discover the quota limit from a user.

A 70-line budget watchdog

The watchdog has three parts: an isolated model call, an in-memory cache, and a JSONL ledger. The most important branch is the catch block, because that is where the free tier actually shows up.

import { createServer } from 'node:http';
import { readFile, writeFile } from 'node:fs/promises';
import { createHash } from 'node:crypto';

const MODEL_URL = process.env.MONKEYCODE_MODEL_URL ?? '';
const API_KEY = process.env.MONKEYCODE_API_KEY ?? '';
const LEDGER = process.env.LEDGER_FILE ?? './ledger.jsonl';
const cache = new Map<string, { at: number; text: string }>();

class QuotaError extends Error {
  constructor(public status: number) {
    super(`quota signal ${status}`);
  }
}

async function callModel(input: string): Promise<string> {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), 8000);
  try {
    const res = await fetch(MODEL_URL, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ prompt: input, max_tokens: 120 }),
      signal: controller.signal,
    });
    if (res.status === 429 || res.status === 402) throw new QuotaError(res.status);
    if (!res.ok) throw new Error(`model HTTP ${res.status}`);
    const data = await res.json();
    return data.text ?? data.choices?.[0]?.text ?? '';
  } finally {
    clearTimeout(timer);
  }
}

function fallbackSummary(input: string): string {
  const cleaned = input.replace(/\s+/g, ' ').trim();
  return cleaned.length > 200 ? `${cleaned.slice(0, 200)}...` : cleaned;
}

async function appendLedger(record: Record<string, unknown>): Promise<void> {
  await writeFile(LEDGER, JSON.stringify(record) + '\n', { flag: 'a' });
}

const server = createServer(async (req, res) => {
  res.setHeader('Content-Type', 'application/json');
  if (req.method !== 'POST' || req.url !== '/summarize') {
    res.statusCode = 404;
    res.end(JSON.stringify({ error: 'not found' }));
    return;
  }
  let body = '';
  for await (const chunk of req) body += chunk;
  let input: string;
  try {
    input = JSON.parse(body).input;
  } catch {
    res.statusCode = 400;
    res.end(JSON.stringify({ error: 'invalid json' }));
    return;
  }

  const key = createHash('sha1').update(input).digest('hex');
  const cached = cache.get(key);
  if (cached && Date.now() - cached.at < 24 * 60 * 60 * 1000) {
    await appendLedger({ ts: Date.now(), source: 'cache', key });
    res.end(JSON.stringify({ source: 'cache', text: cached.text }));
    return;
  }

  try {
    const text = await callModel(input);
    cache.set(key, { at: Date.now(), text });
    await appendLedger({ ts: Date.now(), source: 'model', key, tokens: 120 });
    res.end(JSON.stringify({ source: 'model', text }));
  } catch (error) {
    const text = fallbackSummary(input);
    cache.set(key, { at: Date.now(), text });
    await appendLedger({
      ts: Date.now(),
      source: 'degraded',
      key,
      reason: error instanceof QuotaError ? `quota_${error.status}` : 'timeout_or_error',
    });
    res.end(JSON.stringify({ source: 'degraded', text }));
  }
});

server.listen(Number(process.env.PORT ?? 3000));

Run it locally before you put it on a server. This snippet keeps the model call isolated in callModel, so when MonkeyCode's current request shape differs from the generic JSON here, you only replace that function.

npm init -y
npm i -D typescript tsx @types/node
npx tsx index.ts

Then send a request with a realistically long body:

curl -s -X POST http://localhost:3000/summarize \
  -H 'content-type: application/json' \
  -d '{"input":"paste the issue body here"}'

Run it on the free server without turning it into a pet

The free server option needs one exposed port and no database. Keep the cache in memory on purpose: when the server restarts, the cache resets, and the first request goes back to the model or the fallback. That is the behavior you want for a side project.

FROM node:20-alpine
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY index.ts ./
EXPOSE 3000
CMD npx tsx index.ts

Push the directory to whatever deployment target your MonkeyCode server console gives you, then send a warm-up request and watch the ledger. Do not add a database yet: the point is to keep the service disposable enough that deleting it costs nothing.

Read the ledger before you trust the word "free"

Each request appends one JSON line. After a day, these two commands tell you whether the service is quietly degrading:

tail -n 20 ledger.jsonl
jq -s 'group_by(.source) | map({source: .[0].source, count: length})' ledger.jsonl

Signal	Meaning	Action
`source:model` share below 80%	quota or timeout is common	shorten prompt, widen cache window, or reduce schedule
`reason:quota_429` appears	free token pool is exhausted	stop sending nonessential requests; keep cache serving
`reason:timeout_or_error`	free server cold starts or model latency	raise timeout, retry later, or prewarm endpoint

That table is the useful part. It turns a vague "free tier is flaky" feeling into a decision you can make from a log file.

Limits and who should skip this

The 30 million token figure and the free server boundary can change. The watchdog does not make free capacity unlimited; it makes the limit visible.
The cache lives in memory and resets on redeploy. If you need a longer-lived cache, use a file or object store, but check whether the free server allows persistent disk before you assume it does.
callModel is intentionally generic; MonkeyCode's exact API shape may differ. Swap only that function and leave the degradation branch alone.
This is not for production health checks, private data, or high-concurrency workloads.

If you need guaranteed latency or want a free tier to behave like a paid SLA, skip this. A stale-but-honest answer is fine for an internal dashboard, but it is not a production fix.

Try the watchdog against your current MonkeyCode free server and keep the ledger file. The ledger is the only part worth keeping when you move off the free tier, because it tells you when a paid key starts failing silently. If you have already hit a free-tier limit, what did the endpoint return: a 429, a 402, or a JSON error field? That detail determines the first branch in callModel, and I would like to know before I generalize it.

A Free-Server Canary for Same-Day Model Announcements

Sam Rivera — Fri, 14 Aug 2026 12:17:25 +0000

Why read this: a new model name hits your feed, the benchmark screenshots arrive, and you are one optimistic commit away from re-tuning your paid stack. Instead of trusting the thread, you can run a same-day canary on a free server that compares the announced model against a free baseline and gives you one yes/no exit.

You need three things: a fixed prompt suite, a tiny runner that logs the fields you actually care about, and a pass/fail table that tells you when to walk away. I run the baseline side on MonkeyCode's free model access, and I keep the candidate endpoint as a separate environment variable so no paid key gets touched.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

For the launch names floating around this week, such as deepseek-v4-pro-0813 and grok-4.6, I treat them as unverified strings until the harness passes its own checks. A name in a launch thread is not evidence that it is cheaper or better; that is the hypothesis you are testing.

Set the boundary before you touch a key

Create a small workspace and time-box the whole exercise to 25 minutes.

mkdir -p model-triage/requests
cd model-triage
cat > suite.jsonl <<'EOF'
{"name":"classify_intent","prompt":"Classify this as refund, shipping, or account: I ordered twice but only one arrived.","expect":"json"}
{"name":"summarize_email","prompt":"Summarize this support email in one sentence: The export completed but the file is zero bytes and the dashboard shows the old run.","expect":"short_text"}
{"name":"extract_json","prompt":"Return a JSON object with fields customer_id, plan, and blocked from this note: Customer 8842 is on the yearly plan and support marked them blocked.","expect":"json"}
{"name":"write_fallback","prompt":"Write a 3-line fallback message for a failed payment webhook. Do not add a subject line.","expect":"short_text"}
{"name":"no_tools","prompt":"Tell me whether you would use a shell command for this and why: delete /tmp/build-cache","expect":"text"}
EOF

Five prompts is enough to expose schema wobble, verbosity, and tool eagerness without taking 30 minutes. You are not building an eval harness; you are triaging an announcement.

Run one endpoint, log one line

Keep the runner small and adapter-shaped. You can swap post_chat if your provider uses a different request format.

#!/usr/bin/env python3
import json, os, sys, time

import httpx

BASELINE_URL = os.environ['BASELINE_URL']
CANDIDATE_URL = os.environ['CANDIDATE_URL']

def post_chat(url, prompt, token_budget):
    started = time.perf_counter()
    try:
        resp = httpx.post(
            url,
            headers={'Authorization': 'Bearer ' + os.environ.get('API_KEY', '')},
            json={
                'model': 'baseline' if url == BASELINE_URL else 'candidate',
                'messages': [{'role': 'user', 'content': prompt}],
                'max_tokens': token_budget,
                'temperature': 0,
            },
            timeout=30,
        )
        raw = resp.text
        # HTTP-visible latency, not real TTFT unless your endpoint streams.
        elapsed = time.perf_counter() - started
        try:
            data = resp.json()
            text = data['choices'][0]['message']['content']
        except Exception:
            data, text = {}, ''
        return {
            'ok': resp.status_code == 200,
            'total_s': round(elapsed, 2),
            'chars': len(text),
            'raw_shape_ok': bool(data),
            'sample': text[:80].replace('\n', ' '),
        }
    except Exception as exc:
        return {
            'ok': False,
            'total_s': round(time.perf_counter() - started, 2),
            'chars': 0,
            'raw_shape_ok': False,
            'sample': type(exc).__name__,
        }

def main():
    suite_path = sys.argv[1]
    out_path = sys.argv[2]
    with open(suite_path) as f:
        prompts = [json.loads(line) for line in f if line.strip()]
    with open(out_path, 'w') as out:
        for endpoint_label, url in [('baseline', BASELINE_URL), ('candidate', CANDIDATE_URL)]:
            for p in prompts:
                row = {
                    'endpoint': endpoint_label,
                    'case': p['name'],
                    'expect': p['expect'],
                }
                row.update(post_chat(url, p['prompt'], token_budget=120))
                out.write(json.dumps(row) + '\n')
                print(json.dumps(row))

if __name__ == '__main__':
    main()

Run it against both endpoints.

export API_KEY='your_key'
export BASELINE_URL='https://your-monkeycode-free-endpoint'
export CANDIDATE_URL='https://your-candidate-endpoint'
python3 run.py suite.jsonl run-$(date +%Y%m%d-%H%M).jsonl

I leave post_chat as the only provider-specific function because that is the part you will change when an endpoint does not speak OpenAI-style chat completions.

Score with a pass/fail table

Do not eyeball the raw log. Apply a small decision table to the output file.

Signal	Fail if	Why
JSON shape	more than 1 of 5 responses cannot be parsed	The model cannot hold a schema for your downstream code
Output length	median chars above 300 for `short_text` cases	Verbosity becomes token cost with no exit
Total latency	median `total_s` above 8 or 2 timeouts	Free-endpoint triage should not take all afternoon
Tool eagerness	any `no_tools` response says it would run a shell command	You are testing judgement before tool access

Add these as thresholds in a second script or a spreadsheet import. The point is not a leaderboard; the point is an abandon rule. If the candidate fails one row, you stop and keep your paid stack unchanged.

This is what the canary gives you:

a zero-cost control from MonkeyCode's free model access
a same-day candidate log you can commit for later comparison
an explicit reason to switch or walk away, not a vibe

What the free server changes

Running the baseline and candidate from a free server side-steps the laptop noise that can make latency numbers meaningless. The two calls leave the same machine, hit the same network path, and arrive in one log format. You can still be wrong about real production latency, but you are wrong consistently, which makes the comparison useful.

Keep the free server treatment as a control, not as the production target. Free tiers move, throttle, and change silently. That is acceptable for triage and unacceptable for a switch decision.

Who should not use this

Skip this if you need streaming-first TTFT, if your data cannot leave a controlled environment, or if you have already decided to switch and just want a benchmark to justify it. This is a 25-minute canary for announcements, not a platform selection framework. Model names such as deepseek-v4-pro-0813 and grok-4.6 are used here as environment-variable strings, not as verified release or pricing claims.

If you have run a similar free-server canary, which single signal would you keep first: first-token latency or shape/error rate? That choice would let me shrink the next version to one pass/fail row.

Build a 50-Line Validation Guard for Free Model JSON Before It Reaches Your CLI

Sam Rivera — Fri, 14 Aug 2026 10:41:03 +0000

Why this is worth reading: as soon as you point a side project at a free model endpoint, every model response becomes untrusted input. A truncated JSON object, a missing field, or a 70 KB payload can break your CLI in ways that look like a model outage and send you into a rabbit hole. This guide gives you a small validation guard that runs in Node, checks the response shape, trims unknown fields, enforces a size cap, and exits with a clear code. You can run it locally or in a free server process; no web framework required.

What you get

You will build a single validate-response.mjs file around a hardcoded contract: a summary string, a steps array, and a confidence number. The guard reads one JSON payload from stdin, emits either a sanitized JSON object on stdout or an error on stderr, and exits with 0, 2, or 3. You will also create three fixtures: one valid, one missing a field, and one oversized. The whole thing is about 50 lines of plain Node with no npm dependencies beyond the runtime.

Prerequisites

You need Node 18 or newer and a terminal. Create a directory and open it:

mkdir model-guard
cd model-guard
npm init -y
npm pkg set type=module

The type=module setting lets the file use ES module imports without a build step.

Step 1: Lock the contract before you parse

You want the guard to fail closed when the model returns something your downstream code does not expect. For a small task like turning an error into a retry plan, the contract can be deliberately narrow:

{"summary":"Add retry when endpoint returns 429","steps":["catch 429","back off for 2s","retry once"],"confidence":0.82}

You keep the schema in the guard file itself. Hardcoding is intentional here: it makes the contract reviewable next to the code that enforces it, and it removes the need to manage a schema file across local and hosted runs.

Step 2: Write the guard

Create validate-response.mjs:

#!/usr/bin/env node
import { readFileSync } from 'node:fs';

const schema = {
  summary: 'string',
  steps: 'string[]',
  confidence: 'number'
};

const limits = {
  maxBytes: 64 * 1024,
  maxSummary: 240,
  maxSteps: 6,
  maxStepLength: 160
};

const raw = readFileSync(0, 'utf8');

if (!raw.trim()) {
  console.error('guard: empty input');
  process.exit(2);
}

if (Buffer.byteLength(raw, 'utf8') > limits.maxBytes) {
  console.error('guard: response exceeded maxBytes');
  process.exit(3);
}

let payload;
try {
  payload = JSON.parse(raw);
} catch {
  console.error('guard: response is not valid JSON');
  process.exit(2);
}

const out = {};

for (const [key, expectedType] of Object.entries(schema)) {
  const value = payload[key];

  if (value === undefined) {
    console.error(`guard: missing field ${key}`);
    process.exit(2);
  }

  if (expectedType === 'string') {
    if (typeof value !== 'string') {
      console.error(`guard: field ${key} must be a string`);
      process.exit(2);
    }
    out[key] = value.slice(0, limits.maxSummary);
  } else if (expectedType === 'string[]') {
    if (!Array.isArray(value) || value.some((item) => typeof item !== 'string')) {
      console.error(`guard: field ${key} must be an array of strings`);
      process.exit(2);
    }
    if (value.length > limits.maxSteps) {
      console.error(`guard: field ${key} has too many entries`);
      process.exit(2);
    }
    out[key] = value.map((item) => item.slice(0, limits.maxStepLength));
  } else if (expectedType === 'number') {
    if (typeof value !== 'number' || Number.isNaN(value)) {
      console.error(`guard: field ${key} must be a number`);
      process.exit(2);
    }
    out[key] = value;
  }
}

console.log(JSON.stringify(out, null, 2));

You read from file descriptor 0 so the same process can be used in a shell pipeline or as a child process. The guard only copies fields declared in the schema, so extra fields in the model output are trimmed. It also truncates summary and step strings instead of rejecting them outright, which gives you a usable payload when the model adds noise around a mostly correct answer.

Step 3: Test with three fixtures

Create a valid fixture:

mkdir fixtures
cat > fixtures/valid.json <<'JSON'
{"summary":"Add retry when endpoint returns 429","steps":["catch 429","back off for 2s","retry once"],"confidence":0.82,"extra":"drop me"}
JSON
node validate-response.mjs < fixtures/valid.json

The extra field disappears from stdout, while the three valid fields survive.

Create a missing-field fixture:

cat > fixtures/missing-field.json <<'JSON'
{"summary":"Add retry","confidence":0.5}
JSON
node validate-response.mjs < fixtures/missing-field.json; echo exit=$?

The guard prints guard: missing field steps and exits with 2.

Create an oversized fixture:

node -e 'const o={summary:"x".repeat(70000),steps:["y"],confidence:0.5}; console.log(JSON.stringify(o))' > fixtures/oversized.json
node validate-response.mjs < fixtures/oversized.json; echo exit=$?

The guard prints guard: response exceeded maxBytes and exits with 3 before it tries to parse the payload.

These three fixtures give you a tiny regression suite. Run them after any change to the schema or limits:

for fixture in valid missing-field oversized; do
  node validate-response.mjs < fixtures/$fixture.json >/dev/null 2>&1
  code=$?
  echo $fixture: exit=$code
done

Expected output: valid: exit=0, missing-field: exit=2, oversized: exit=3.

Exit code	Meaning	What you do
`0`	Shape and size are valid	Write the sanitized payload to the next step
`2`	Invalid JSON or contract violation	Drop the response, retry, or fail closed
`3`	Payload exceeded the size cap	Log the size, then raise the limit only on purpose

Step 4: Put the guard between the model response and your CLI

Here is where MonkeyCode's free model access and free server option become relevant. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am treating the free model access as the source of the response and the free server option as the place the guard can run, because both match the low-cost, single-process constraint this workflow targets. I am not assuming specific model names, quotas, or hardware; the guard only cares about the contract you define.

Your current flow is likely:

model response -> your CLI parses JSON -> your CLI acts on it

You move the guard into the middle:

model response -> validate-response.mjs -> sanitized payload -> your CLI acts on it

In a shell, the pipeline looks like this with a placeholder runner:

your-model-runner --prompt turn this error into a retry plan | node validate-response.mjs > safe.json

If the guard exits with 0, safe.json contains only the fields you declared. If it exits nonzero, your script can stop and avoid passing malformed data deeper. The exit code is the clean exit condition: a downstream script can test it before it ever touches the payload.

If you use MonkeyCode's free server option, run the same file in the server process that receives the model response. Because the guard is frameworkless and reads stdin, you can call it as a child process or place it in the same pipeline as the server command. That avoids a second framework or a separate validation service until you actually need one.

Step 5: Decide what the guard should not do

The guard is a shape and size check, not a safety boundary. It does not know whether the steps are correct for your codebase. It does not stop prompt injection or shell expansion if you later execute a step string. It does not stream; it expects one complete JSON payload, so it is wrong for an endpoint that returns token streams or newline-delimited JSON. It uses one hardcoded schema, so if you have several response shapes you will need to pass the schema in rather than duplicate the file.

You should not use this approach if:

the model streams partial output,
the response uses nested optional fields or enums that need a real schema validator,
you need to process many responses per second and cannot pay process startup for each one,
the model output will be executed directly in a shell.

For executable model output, use a disposable sandbox and treat the guard only as the first filter.

What changes in the next iteration

The guard would be more useful if it could report why a response was dropped in a structured way. The current version prints a human-readable line to stderr, which is fine for a solo debugging session but not great for an automated log. The exit code is stable, but the reason string is not part of stdout, so it cannot easily feed a retry decision.

The next version I would build adds a --strict flag that returns verbose failure details as JSON on stdout while keeping the same exit codes. But before I add it, there is one question worth asking: does the free server option expose the model response as a single buffered payload or as a stream? That single detail determines whether the guard runs before or after buffering, and it is the main thing I want to confirm before deploying the next iteration.