DEV Community: Charlie Hu

Weekend Scope Cut: An Assumption Ledger Before the AI Writes Code

Charlie Hu — Fri, 04 Sep 2026 18:16:26 +0000

AI coding assistants invent files, APIs, and environment variables when a repository is incomplete. A weekend-sized countermeasure is enough. This recipe keeps a JSON assumption ledger, turns that ledger into a constrained prompt, and checks the resulting patch against paths that actually exist. The loop is local, boring, and small enough to finish in one sitting.

The goal is not an agent platform. The goal is a working demo that refuses invented architecture before a human wastes a Saturday merging it.

The failure this project is built to catch

Agentic coding tools fill missing context. They propose modules that are not in the tree. They call helpers that were never merged. They assume a .env key because a README mentioned it once.

That pattern shows up in brownfield apps more than in greenfield toys. A model that cannot see package.json, a private SDK, or a migration folder will still emit a confident patch. The patch compiles in the model's head. It does not compile in the checkout.

A ledger does not make the model smarter. It makes the model's guesses visible, countable, and testable.

Weekend scope: what ships, what is skipped

This is a labeled recipe, not a production postmortem. No personal benchmark, customer, or production incident is claimed. The working demo is three files plus a sample repository snapshot.

In scope

A JSON schema for assumptions the assistant is allowed to make.
A prompt assembler that injects those assumptions and forbids the rest.
A checker that flags patch hunks touching missing paths or undeclared symbols.
A 90-minute command-line walkthrough a reader can run on a laptop.

Explicitly skipped

Multi-agent orchestration, MCP servers, and tool-calling loops.
A web UI, a database, and authentication.
Token accounting, secret redaction, and merge gates already covered in other write-ups.
Live evaluation against a named model or a claimed latency number.

Skipping those items is the point. A weekend build that tries to become a platform usually ships nothing.

Artifact 1: the assumption ledger

Store the ledger next to the repo, not in chat history. Chat history evaporates. A file can be grepped, reviewed, and reset.

{
  "task": "Add a dry-run flag to the invoice export CLI",
  "repo_root": ".",
  "allowed_paths": [
    "src/cli.py",
    "src/export/invoices.py",
    "tests/test_export_invoices.py"
  ],
  "forbidden_paths": [
    "src/legacy/**",
    "migrations/**"
  ],
  "known_symbols": [
    "export_invoices",
    "InvoiceRow",
    "parse_args"
  ],
  "assumptions": [
    {
      "id": "A1",
      "claim": "InvoiceRow already has a `status` field",
      "evidence": "src/export/invoices.py:41",
      "status": "verified"
    },
    {
      "id": "A2",
      "claim": "CLI uses argparse, not click",
      "evidence": "src/cli.py:1-20",
      "status": "verified"
    },
    {
      "id": "A3",
      "claim": "Dry-run should skip S3 upload",
      "evidence": "none-in-repo",
      "status": "unverified"
    }
  ],
  "hard_rules": [
    "Do not create files outside allowed_paths",
    "Do not invent symbols absent from known_symbols unless marked new",
    "Every unverified assumption must be listed before code"
  ]
}

Unverified rows are first-class. They are not shame. They are the only honest output when the tree does not support a product request.

A later pass can flip A3 to rejected if S3 upload lives in a worker the CLI does not own. The model then has to propose a smaller patch or stop.

Artifact 2: assemble a constrained prompt

The assembler is a short Python script. It reads the ledger and prints a prompt block. The assistant never sees the whole monorepo. It sees the task, the allowed paths, and the open assumptions.

# recipe: assemble_prompt.py — unexecuted example for a weekend checkout
from __future__ import annotations

import json
from pathlib import Path

LEDGER = Path("assumption_ledger.json")


def load_ledger() -> dict:
    return json.loads(LEDGER.read_text(encoding="utf-8"))


def render(ledger: dict) -> str:
    unverified = [a for a in ledger["assumptions"] if a["status"] != "verified"]
    lines = [
        "You are editing an existing repository.",
        f"Task: {ledger['task']}",
        "Allowed paths:",
        *[f"- {p}" for p in ledger["allowed_paths"]],
        "Forbidden paths:",
        *[f"- {p}" for p in ledger["forbidden_paths"]],
        "Known symbols:",
        *[f"- {s}" for s in ledger["known_symbols"]],
        "Hard rules:",
        *[f"- {r}" for r in ledger["hard_rules"]],
        "Unverified assumptions (do not treat as facts):",
    ]
    if not unverified:
        lines.append("- none")
    else:
        for item in unverified:
            lines.append(f"- {item['id']}: {item['claim']} ({item['evidence']})")
    lines.append("Reply with: (1) assumption updates (2) a unified diff limited to allowed_paths.")
    return "\n".join(lines)


if __name__ == "__main__":
    print(render(load_ledger()))

Run it from the repo root:

python3 assemble_prompt.py > /tmp/constrained_prompt.txt
wc -l /tmp/constrained_prompt.txt

The prompt is intentionally small. A weekend project that dumps the entire tree into context recreates the original problem: the model invents structure because the signal is noisy.

Artifact 3: check the patch against the real tree

After the assistant returns a unified diff, do not read it like prose. Parse it. Compare every +++ path to allowed_paths. Compare added identifiers to known_symbols plus an explicit new: list if the ledger grows that field later.

# recipe: check_patch.py — unexecuted example
from __future__ import annotations

import json
import re
from pathlib import Path

LEDGER = Path("assumption_ledger.json")
DIFF = Path("/tmp/proposed.patch")
PLUS_FILE = re.compile(r"^\+\+\+ b/(.+)$")
ADD_DEF = re.compile(r"^\+\s*def\s+([A-Za-z_][A-Za-z0-9_]*)")


def failures(ledger: dict, diff_text: str) -> list[str]:
    allowed = set(ledger["allowed_paths"])
    known = set(ledger["known_symbols"])
    problems: list[str] = []
    for line in diff_text.splitlines():
        m = PLUS_FILE.match(line)
        if m:
            path = m.group(1)
            if path != "/dev/null" and path not in allowed:
                problems.append(f"path-not-allowed: {path}")
        d = ADD_DEF.match(line)
        if d and d.group(1) not in known:
            problems.append(f"new-symbol-undeclared: {d.group(1)}")
    return problems


if __name__ == "__main__":
    ledger = json.loads(LEDGER.read_text(encoding="utf-8"))
    diff_text = DIFF.read_text(encoding="utf-8")
    found = failures(ledger, diff_text)
    if not found:
        print("CHECK_OK")
    else:
        print("CHECK_FAIL")
        for item in found:
            print(item)
        raise SystemExit(1)

The checker is crude on purpose. It does not understand Python scoping. It catches the expensive class of errors: brand-new files and brand-new functions that nobody agreed to.

A sample failing patch looks like this:

--- a/src/cli.py
+++ b/src/cli.py
@@ -10,0 +10,6 @@
+def upload_to_s3(path):
+    raise NotImplementedError
+
+++ b/src/export/s3_client.py
@@ -0,0 +1,8 @@
+def build_client():
+    return None

Expected checker output:

CHECK_FAIL
new-symbol-undeclared: upload_to_s3
path-not-allowed: src/export/s3_client.py
new-symbol-undeclared: build_client

That failure is the demo. The weekend is successful when the script says no.

A 90-minute working path

The following sequence is a recipe. Treat it as a checklist, not as a claim that a specific machine already ran it.

Copy a small CLI repo or a single package directory into a throwaway folder.
Hand-write assumption_ledger.json with at most eight allowed paths.
Mark every claim without a file:line evidence as unverified.
Generate /tmp/constrained_prompt.txt with assemble_prompt.py.
Paste that prompt into an AI coding session on a laptop or a free coding server.
Save the model output as /tmp/proposed.patch without manual cleanup.
Run python3 check_patch.py and keep the first failure list.
Either shrink the task or promote an assumption to verified with evidence.
Repeat once. Stop after two loops even if the patch is imperfect.

Stopping is part of the scope cut. A third loop usually means the task is too large for the ledger.

Where a free coding server fits this loop

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The ledger and checker do not require a vendor. They run with files and Python 3. A coding assistant still has to propose the diff. MonkeyCode's free model access and free server option are relevant when the weekend machine should not install a local GPU stack and should not send the whole company monorepo to an unbounded cloud session.

Use the free server as the place the constrained prompt is pasted. Keep the ledger and checker on disk the human controls. The server proposes. The laptop verifies. That split is the architecture.

Do not treat free access as infinite access. Do not treat a free server as a compliance boundary. Quotas, model catalogs, hardware, and uptime are out of scope here because they change and because this recipe does not depend on a named SKU.

A natural next step, if a hosted coding workspace is useful, is to try the same ledger loop there rather than expanding the script.

Decision table for the second hour

Symptom	Ledger change	Code change
Patch adds a new file	Keep `allowed_paths` closed	Reject the patch
Patch needs a real new file	Add one path with a reason	Allow a second loop
Model treats `A3` as fact	Leave status `unverified`	Require a comment, not code
Checker flags a helper that exists	Add the symbol with evidence	Re-run once
Task needs migrations	Out of weekend scope	Split into a later change

The table prevents the usual Saturday drift: one more file, then a client, then a config loader, then a rewrite.

Limitations

The checker does not parse imports, types, or test coverage. A model can still mutate a allowed file into nonsense. Path allowlists do not stop a destructive edit inside src/cli.py.

Unified diffs that rename files, use a/ prefixes inconsistently, or include binary blobs will confuse the regex. Human inspection remains mandatory.

The ledger can freeze a wrong architecture if verified rows are lazy. Evidence must be a path and a line range, not a vibe. Unverified rows that linger for weeks become folklore.

This workflow also assumes a reader can read a diff. It is a poor fit for purely conversational, no-repo coding.

Who should not use this approach

Skip it when the repository is generated from scratch and invention is the product. Skip it when legal review requires a vendor DPA, audit logs, or an approved model list this recipe does not provide. Skip it when the team already has a mature CI policy bot that checks the same invariants with real parsers.

Solo beginners can still use the ledger as a thinking aid. They should not treat CHECK_OK as proof the patch is correct.

What a later weekend can add, and why it waited

A real AST pass, a test runner hook, and a new_symbols field are useful. They are also how this project would have missed the demo. The valuable output of the weekend is a failing checker and a prompt that names its guesses.

Ship the refusal first. Expand parsers after a human has watched the script catch one invented file. That is the whole build log: a smaller surface, a visible assumption list, and a hard stop when the model starts building a parallel repository in the patch.

Your Code Leaves the Laptop: A Local Redaction Proxy for Free AI Tiers

Charlie Hu — Fri, 04 Sep 2026 14:05:52 +0000

Free AI coding servers are useful, but they have one structural consequence: the prompt leaves your machine. For teams that handle credentials, internal paths, or customer data, that single fact can block adoption. The solution is not to abandon free tiers. It is to filter what leaves the laptop before it reaches the server.

This article builds a small local proxy that sits between an IDE and any OpenAI-compatible endpoint. It scans outgoing requests, redacts sensitive patterns, and forwards the sanitized prompt. The proxy is about 150 lines of Python, uses only the standard library plus httpx, and can be extended with custom rules.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why a redaction proxy instead of a VPN or a sandbox

A VPN encrypts the connection but does not change the payload. A sandbox restricts what the model can do, but not what the prompt contains. The only way to prevent sensitive data from reaching a third-party server is to remove it before transmission.

MonkeyCode's free server and 10M token allowance (as of 2026-08-25; check the current docs before relying on the quota) make the free tier attractive. A redaction proxy makes it acceptable for teams that would otherwise refuse to send code to a managed endpoint.

The proxy design

The proxy listens on localhost:8080, accepts POST requests to /chat/completions, and rewrites the messages array. Each message passes through a chain of redaction rules. The rule chain is configurable and order-dependent: replace secrets first, then truncate long context, then apply custom patterns.

import json
import re
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer

import httpx


class RedactionProxy:
    def __init__(self, upstream_url, api_key, rules=None):
        self.upstream = upstream_url
        self.api_key = api_key
        self.rules = rules or default_rules()
        self.client = httpx.Client(
            base_url=upstream_url,
            headers={"Authorization": f"Bearer {api_key}"},
            timeout=120,
        )

    def redact(self, text):
        for rule in self.rules:
            text = rule(text)
        return text

    def forward(self, payload):
        for message in payload.get("messages", []):
            if message.get("role") == "user":
                message["content"] = self.redact(message["content"])
        return self.client.post("/chat/completions", json=payload)


def default_rules():
    return [
        # AWS access key
        lambda t: re.sub(r"AKIA[0-9A-Z]{16}", "AKIA_REDACTED", t),
        # GitHub token
        lambda t: re.sub(r"gh[pousr]_[0-9A-Za-z]{36,255}", "GH_REDACTED", t),
        # Private IP addresses
        lambda t: re.sub(r"\b(?:10|172\.(?:1[6-9]|2\d|3[01])|192\.168)\.\d{1,3}\.\d{1,3}\b", "IP_REDACTED", t),
        # Absolute home paths
        lambda t: re.sub(r"/home/[a-zA-Z0-9_]+/", "/home/USER/", t),
        # Long hex strings (possible API keys)
        lambda t: re.sub(r"\b[0-9a-fA-F]{32,}\b", "HEX_REDACTED", t),
    ]

The proxy handler is a thin HTTP layer that parses the incoming request, calls forward, and returns the upstream response unchanged.

class ProxyHandler(BaseHTTPRequestHandler):
    proxy = None

    def do_POST(self):
        length = int(self.headers.get("Content-Length", 0))
        body = self.rfile.read(length)
        payload = json.loads(body)

        try:
            response = self.proxy.forward(payload)
            self.send_response(response.status_code)
            self.send_header("Content-Type", "application/json")
            self.end_headers()
            self.wfile.write(response.content)
        except Exception as exc:
            self.send_response(502)
            self.send_header("Content-Type", "application/json")
            self.end_headers()
            self.wfile.write(json.dumps({"error": str(exc)}).encode())

    def log_message(self, format, *args):
        # Silence default logging; add your own if needed
        pass


def run(proxy, port=8080):
    ProxyHandler.proxy = proxy
    server = HTTPServer(("127.0.0.1", port), ProxyHandler)
    print(f"Redaction proxy listening on http://127.0.0.1:{port}")
    server.serve_forever()

Start the proxy with environment variables:

export UPSTREAM_URL="https://your-free-endpoint.example/v1"
export API_KEY="your-key"
python redaction_proxy.py

Then point any OpenAI-compatible client to http://127.0.0.1:8080 instead of the real endpoint. The IDE or script sees a normal API; the proxy does the filtering.

Testing the proxy with a real request

Create a small test script that sends a prompt containing a fake AWS key and an internal IP, then verify the upstream receives redacted content. The test can be run against a local mock server to avoid burning tokens.

import json
import unittest
from unittest.mock import patch

from redaction_proxy import RedactionProxy


class TestRedaction(unittest.TestCase):
    def setUp(self):
        self.proxy = RedactionProxy("http://mock", "test-key")

    def test_aws_key_redacted(self):
        text = "Key: AKIAIOSFODNN7EXAMPLE"
        result = self.proxy.redact(text)
        self.assertNotIn("AKIAIOSFODNN7EXAMPLE", result)
        self.assertIn("AKIA_REDACTED", result)

    def test_private_ip_redacted(self):
        text = "Server at 192.168.1.10 is down"
        result = self.proxy.redact(text)
        self.assertNotIn("192.168.1.10", result)
        self.assertIn("IP_REDACTED", result)

    def test_forward_calls_upstream(self):
        with patch.object(self.proxy.client, "post") as mock_post:
            mock_post.return_value.status_code = 200
            mock_post.return_value.content = b'{"ok": true}'
            payload = {"messages": [{"role": "user", "content": "Hello"}]}
            self.proxy.forward(payload)
            sent_payload = mock_post.call_args[1]["json"]
            self.assertEqual(sent_payload["messages"][0]["content"], "Hello")


if __name__ == "__main__":
    unittest.main()

The unit tests confirm two properties: redaction changes the content, and forwarding preserves the request structure. A more advanced integration test would spin up a local HTTP server and assert the exact body received.

Where the proxy falls short

Regular expressions are not semantic analysis. A hardcoded database password that does not match any pattern will pass through. A Base64-encoded secret will survive if it is shorter than 32 characters. The proxy also cannot redact information hidden in code structure, such as a variable named password with a literal value.

Teams with strict data-residency requirements should not rely on a redaction proxy. If the contract forbids sending any code to a third party, the proxy is not a compliance tool; it is a risk reduction. The only safe option is a fully self-hosted model.

Performance is another cost. Every request passes through Python regex evaluation. For short prompts the overhead is negligible, but for a 10,000-token diff it adds tens of milliseconds. That is usually acceptable, but a high-throughput CI pipeline should benchmark before adopting the proxy.

Who should use this pattern

Teams that want to try a free managed server but have internal IP addresses or placeholder secrets in their codebase.
Developers who use AI assistants for personal projects and want to avoid leaking personal tokens.
Teams evaluating MonkeyCode's free tier (or any free tier) and need a lightweight audit trail of what leaves the network.

The proxy is not a replacement for a data-loss-prevention policy. It is a practical layer that makes free tiers usable for a broader set of tasks.

The takeaway

Free AI coding servers are a legitimate infrastructure choice, but they require a boundary. A local redaction proxy is that boundary: it keeps the convenience of a managed endpoint while stripping the parts that should never leave the laptop. The code in this article is a starting point; the rule chain is where each team's specific sensitivity lives.

MonkeyCode's free server and 10M token allowance (as of 2026-08-25) work well behind such a proxy. When the allowance runs out, the proxy can point to a different upstream without changing the IDE configuration. The filter stays, the endpoint changes, and the code stays safer.

MonkeyCode provides free models that can run this workflow.

Measure the Burn Rate: A Token Meter for Quota-Limited AI Coding Servers

Charlie Hu — Fri, 04 Sep 2026 13:05:16 +0000

A free AI coding server is a budget, not a speed limit. The practical problem is not whether the quota is large enough — it is that nobody sees the consumption until a request fails. This article builds a small token meter that records every request, computes a daily burn rate, and predicts the exhaustion date. The tool works with any OpenAI-compatible endpoint, including the free server that the open-source MonkeyCode project currently advertises.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why invisible consumption is the real problem

Free tiers advertise a token allowance. The MonkeyCode project, for example, currently lists a free tier with 10 million tokens and a free hosted server. That number sounds abstract. In practice, consumption comes from many directions:

Editor autocomplete and inline chat requests
Scripts that batch-process files
CI jobs that run AI-assisted review
Manual experiments with prompts

Each request is small. The sum is not. Without a meter, the first sign of trouble is a 429 or a quota-exceeded error in the middle of a task.

A token meter in three parts

The meter has three components: a SQLite table for usage records, a wrapper around the OpenAI client, and a report function.

1. The storage layer

import sqlite3
import time
from pathlib import Path

DB_PATH = Path.home() / ".token_meter.db"

def init_db():
    conn = sqlite3.connect(DB_PATH)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS usage (
            ts INTEGER PRIMARY KEY,
            model TEXT,
            prompt_tokens INTEGER,
            completion_tokens INTEGER,
            total_tokens INTEGER
        )
    """)
    conn.commit()
    return conn

2. The metered client

from openai import OpenAI

class MeteredClient:
    def __init__(self, base_url, api_key, model):
        self.client = OpenAI(base_url=base_url, api_key=api_key)
        self.model = model
        self.conn = init_db()

    def chat(self, messages, **kwargs):
        resp = self.client.chat.completions.create(
            model=self.model,
            messages=messages,
            **kwargs
        )
        usage = resp.usage
        self.conn.execute(
            "INSERT INTO usage (ts, model, prompt_tokens, completion_tokens, total_tokens) VALUES (?, ?, ?, ?, ?)",
            (int(time.time()), self.model, usage.prompt_tokens, usage.completion_tokens, usage.total_tokens)
        )
        self.conn.commit()
        return resp

3. The report

def burn_rate(days=7):
    conn = sqlite3.connect(DB_PATH)
    cutoff = int(time.time()) - days * 86400
    row = conn.execute(
        "SELECT SUM(total_tokens), COUNT(*), MIN(ts), MAX(ts) FROM usage WHERE ts >= ?",
        (cutoff,)
    ).fetchone()
    total, count, first, last = row
    if not total or not first or first == last:
        return None
    elapsed_days = (last - first) / 86400
    return {
        "total_tokens": total,
        "requests": count,
        "elapsed_days": round(elapsed_days, 2),
        "tokens_per_day": int(total / elapsed_days),
    }

def report(allowance):
    rate = burn_rate()
    if not rate:
        print("No usage recorded yet.")
        return
    days_left = allowance / rate["tokens_per_day"]
    print(f"Usage over {rate['elapsed_days']} days:")
    print(f"  Total tokens: {rate['total_tokens']:,}")
    print(f"  Requests: {rate['requests']}")
    print(f"  Burn rate: {rate['tokens_per_day']:,} tokens/day")
    print(f"  At this rate, {allowance:,} tokens last ~{days_left:.0f} days")

Wiring it into a workflow

Replace the direct client construction with the metered version. The interface is identical, so existing code keeps working.

client = MeteredClient(
    base_url="https://your-server.example.com/v1",
    api_key="none",  # free tiers often ignore the key
    model="model-id-from-docs"
)

resp = client.chat([
    {"role": "user", "content": "Refactor this function to use async/await."}
])
print(resp.choices[0].message.content)

Check the report anytime:

python -c "from meter import report; report(10_000_000)"

The output shows the daily burn rate and a projected exhaustion date. Run it once a day, or add a cron job that emails the report when the projected days-left drops below a threshold.

Reading the numbers

Three patterns matter.

Steady burn below 25% of the allowance per month. The free tier is a reasonable fit. Keep monitoring, but no action needed.
Burn rate above 25% per month. The allowance will not last. Start evaluating paid APIs or self-hosting before the quota hits zero.
Burn rate accelerating. Compare this week to last week. If the rate is climbing, the workload changed — a new script, a new editor plugin, a teammate joining the project. Find the source before it becomes a surprise.

Limitations

The meter only sees requests that go through the wrapped client. Editor extensions, other scripts, or CI jobs that call the API directly bypass the recording. A complete view requires integrating the meter into every entry point, or reading server-side usage logs if the provider exposes them.

The prediction is a linear extrapolation. Bursty workloads — a large refactor, a batch review of a pull request — can push actual consumption far above the forecast. Treat the meter as an early warning system, not a billing audit.

Who should not use this approach

One-off experimenters. A single weekend of testing will not exhaust a 10-million-token allowance. The meter is overhead.
Paid API users. No quota, no need for a meter.
Teams that need exact billing. This tool is an approximation. It does not capture prompt caching, retries, or provider-side adjustments.

Conclusion

The quota problem on a free AI coding server is not the size of the allowance. It is the visibility. A thirty-line meter turns "how much is left" from a guess into a reading. Collect data for a few days, then decide whether the free tier, a paid API, or a self-hosted setup is the right host. The numbers decide; the meter just makes them visible.

MonkeyCode provides free models that can run this workflow.

Free AI Server vs. Self-Hosted: Score Before You Commit

Charlie Hu — Fri, 04 Sep 2026 12:05:00 +0000

Free AI servers and self-hosted stacks are not two points on a quality scale. They are different risk profiles. The real question is not which is better but which failure mode this team can afford — and a weighted scorecard answers that faster than a week of gut feeling.

AI coding tools now bundle free token allowances and managed servers as a default entry point. The price tag reads zero, but the cost shows up later: quota ceilings, data boundaries, latency spikes, and the migration effort when the workload outgrows the tier. Self-hosting inverts the tradeoff: infinite control, finite attention.

This article lays out a five-criterion scorecard, a runnable Python script, and a generic latency probe. Use them before committing a workflow to any tier.

The Five Criteria That Decide the Fit

Most tier debates collapse into price. Price is the easiest number to compare and the least useful one to decide with. These five criteria carry more signal:

Volume ceiling — How many tokens the workload consumes per week. A repo-wide refactor, a documentation sweep, and a test-suite generator have very different appetites.
Data sensitivity — What leaves the machine. Source code, test data, and internal comments are not equivalent. A managed server means someone else's infrastructure sees the payload.
Latency budget — Whether the workload is interactive or batch. Autocomplete needs a different time-to-first-token than an overnight migration script.
Integration depth — Whether the tier plugs into the editor, CI, or a custom pipeline. A free tier with a thin API surface can force more glue code than a self-hosted box.
Ops budget — How many hours per week the team can spend patching, monitoring, and restarting a server. For a solo developer, that number is often close to zero.

Score each criterion from 1 (poor fit) to 5 (excellent fit). Then weight the criteria by what the team actually fears most.

The Scorecard: A Runnable Script

The script below implements the weighted decision. Scores are inputs, not outputs — the team fills them in, and the script turns opinions into a ranked list.

# score_tiers.py — weighted fit score for AI coding tool tiers
# Scores are 1 (poor fit) to 5 (excellent fit). Weights must sum to 1.0.

TIERS = ["free_managed", "self_hosted", "paid_api"]

def recommend(scores, weights):
    results = {
        tier: round(sum(scores[tier][c] * weights[c] for c in weights), 2)
        for tier in TIERS
    }
    best = max(results, key=results.get)
    return results, best

if __name__ == "__main__":
    weights = {
        "volume_ceiling": 0.30,
        "data_sensitivity": 0.25,
        "latency_budget": 0.20,
        "integration_depth": 0.15,
        "ops_budget": 0.10,
    }
    scores = {
        "free_managed": {
            "volume_ceiling": 3, "data_sensitivity": 2,
            "latency_budget": 4, "integration_depth": 3,
            "ops_budget": 5,
        },
        "self_hosted": {
            "volume_ceiling": 5, "data_sensitivity": 5,
            "latency_budget": 4, "integration_depth": 4,
            "ops_budget": 2,
        },
        "paid_api": {
            "volume_ceiling": 4, "data_sensitivity": 3,
            "latency_budget": 5, "integration_depth": 4,
            "ops_budget": 4,
        },
    }
    results, best = recommend(scores, weights)
    for tier, score in sorted(results.items(), key=lambda x: -x[1]):
        print(f"{tier:14s} {score:.2f}")
    print(f"Recommended: {best}")

Run it with the sample scores to see the mechanics:

python3 score_tiers.py

The sample weights assume the team fears quota exhaustion more than data exposure. A regulated team should move data_sensitivity to 0.35 and watch the ranking flip. That flip is the point — the scorecard surfaces the assumption, not just the answer.

A Latency Probe Before You Trust Any Endpoint

Scorecards handle fit. Latency needs a measurement. The probe below is intentionally generic: fill in the endpoint and payload shape for the tier under test, then run it three times.

#!/usr/bin/env bash
# probe.sh — measure time-to-first-token for any AI endpoint
# Usage: ./probe.sh https://your-endpoint "prompt"
URL="$1"; PROMPT="$2"
for i in 1 2 3; do
  curl -s -o /dev/null -w "attempt $i: %{time_starttransfer}s\n" \
    -X POST "$URL" \
    -H "Content-Type: application/json" \
    -d "{\"prompt\": \"$PROMPT\"}"
done

Run it from the same network the editor will use, not from a datacenter. A managed server can look fast from a cloud shell and slow from a home office. Three attempts are a smoke test, not a benchmark — log a full day before making a latency decision.

Where the Free-Managed Column Scores High

MonkeyCode is one of the tools that makes the free-managed column worth scoring honestly. At the time of writing (August 2026), its free tier advertises a 10 million token allowance and a managed free server option, which lifts both the volume_ceiling and ops_budget scores for solo developers and small teams. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two caveats keep this honest. First, quotas and server terms change; verify the current numbers on the official documentation before building a workflow around them. Second, a free tier is a starting point, not a contract — measure weekly token consumption from day one so the ceiling never surprises the team.

Who Should Not Use This Approach

The scorecard has a bias: it assumes the team can change tiers later. Some situations break that assumption.

Data-residency rules. If the codebase cannot leave the machine, a managed free server is disqualified before scoring. Self-host or use an on-premises paid option.
Sustained high volume. A 10 million token allowance sounds large until a repo-wide refactor or a documentation sweep runs for a week. Measure first, then commit.
Production SLAs. Free tiers rarely carry uptime guarantees. A batch job that can wait is fine; a blocking code-review assistant is not.
No ops person. Self-hosting scores 5 on control and 1 on maintenance. If nobody owns patching and monitoring, the self-hosted column should lose on purpose.

The One-Week Pilot

The scorecard picks a tier. The pilot decides whether to stay. Run one week on a real task — not a demo — and log three numbers: token consumption, time-to-first-token, and prompt rewrites per task.

If the free-managed column wins the scorecard, a practical first step is to run that pilot on MonkeyCode's free server and keep the log. The scorecard tells you which tier to try. The pilot tells you whether the tier survives contact with an actual workload.

MonkeyCode provides free models that can run this workflow.

Sanitize Before You Send: A Local Proxy for AI Coding Assistants

Charlie Hu — Fri, 04 Sep 2026 11:02:46 +0000

A free AI server is a trade. You trade a little privacy for zero infrastructure cost. Most developers accept that trade without thinking about what exactly leaves their machine. Code snippets, API keys, internal hostnames, and even file paths all end up in a prompt. A lightweight local proxy can strip the sensitive parts before they travel anywhere. This article shows how to build one in under fifty lines of Python, and why it should sit between your editor and MonkeyCode's free server.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The Problem: Your Code Leaves the Building

MonkeyCode is an open source project that offers a free server and a 10-million-token grant. That is a generous offer for experimentation and daily coding tasks. But the free server is remote, which means every prompt you send crosses a network boundary. Even if the provider is trustworthy, minimizing exposure is a basic hygiene practice.

The risk is not abstract. A developer pastes a config file into a chat window and forgets that it contains a database password. Another sends a stack trace with internal IP addresses. These small leaks accumulate. A proxy gives you a single choke point where sensitive patterns can be caught before they leave.

The Solution: A Local Filtering Proxy

The idea is simple. Run a small HTTP server on localhost. Point your AI coding tool at that server instead of the remote endpoint. The proxy inspects every request body, replaces sensitive patterns with placeholders, and forwards the sanitized payload to the real server. Responses pass back unchanged.

This approach works with any tool that lets you configure a custom base URL. It adds a few milliseconds of latency, which is negligible compared to network round-trip time.

Building the Proxy

The following Python script uses only the standard library. It listens on 127.0.0.1:8080, sanitizes the JSON body, and forwards the request to an endpoint stored in the MONKEYCODE_ENDPOINT environment variable.

#!/usr/bin/env python3
"""A minimal privacy proxy for AI coding assistants."""

import json
import os
import re
import urllib.request
from http.server import BaseHTTPRequestHandler, HTTPServer

TARGET = os.environ.get("MONKEYCODE_ENDPOINT", "https://api.monkeycode.example/v1/chat/completions")

# Patterns that match common secrets and internal identifiers.
SECRET_PATTERNS = [
    (re.compile(r"(?i)(api[_-]?key|password|secret|token)\s*[:=]\s*[\"']?[A-Za-z0-9_\-]{8,}"), r"\1=[REDACTED]"),
    (re.compile(r"\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b"), "[IP_REDACTED]"),
    (re.compile(r"\b(?:AKIA|ASIA)[A-Z0-9]{16}\b"), "[AWS_KEY_REDACTED]"),
    (re.compile(r"(sk-[A-Za-z0-9]{20,})"), "[OPENAI_KEY_REDACTED]"),
]

def sanitize(text: str) -> str:
    for pattern, replacement in SECRET_PATTERNS:
        text = pattern.sub(replacement, text)
    return text

def sanitize_payload(payload: dict) -> dict:
    if "messages" not in payload:
        return payload
    for message in payload["messages"]:
        if isinstance(message.get("content"), str):
            message["content"] = sanitize(message["content"])
    return payload

class ProxyHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = int(self.headers.get("Content-Length", 0))
        body = self.rfile.read(length)
        try:
            payload = json.loads(body)
        except json.JSONDecodeError:
            self.send_response(400)
            self.end_headers()
            return

        clean_payload = sanitize_payload(payload)
        req = urllib.request.Request(
            TARGET,
            data=json.dumps(clean_payload).encode(),
            headers={"Content-Type": "application/json"},
            method="POST",
        )
        try:
            with urllib.request.urlopen(req) as resp:
                response_body = resp.read()
                self.send_response(resp.status)
                self.send_header("Content-Type", resp.headers.get("Content-Type", "application/json"))
                self.end_headers()
                self.wfile.write(response_body)
        except urllib.error.HTTPError as e:
            self.send_response(e.code)
            self.end_headers()
            self.wfile.write(e.read())

    def log_message(self, format, *args):
        # Silence default logging to keep the console clean.
        pass

if __name__ == "__main__":
    server = HTTPServer(("127.0.0.1", 8080), ProxyHandler)
    print(f"Proxy listening on 127.0.0.1:8080 -> {TARGET}")
    server.serve_forever()

Save the script as proxy.py, set the environment variable, and run it:

export MONKEYCODE_ENDPOINT="https://your-monkeycode-endpoint.example/v1/chat/completions"
python3 proxy.py

Then configure your AI coding tool to use http://127.0.0.1:8080 as the base URL. Every request now passes through the filter.

Testing the Proxy

Before pointing your editor at the proxy, verify it works with a simple curl command:

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "My password is hunter2 and my IP is 10.0.0.5"}]}'

If the proxy is working, the remote server receives a sanitized message: My password is [REDACTED] and my IP is [IP_REDACTED]. The response should still come back normally.

You can also add a logging line inside sanitize_payload to see what was replaced. That helps you tune the regular expressions for your own codebase.

Limitations and Who Should Skip This

A regex-based proxy is not a security boundary. It catches obvious patterns, but it will miss secrets in custom formats, encoded strings, or binary data. It also does not protect against prompt injection or malicious model output. Treat it as a hygiene layer, not a firewall.

Teams working with regulated data or strict confidentiality agreements should not use a remote AI server at all. For them, self-hosting a model is the only defensible option. The proxy is for developers who want to use a free server responsibly without over-engineering their setup.

Another limitation is latency. The proxy adds a local hop, but that overhead is usually under a millisecond. The bigger cost is the regular expression scan on large prompts. If you send 10,000-token prompts, the scan takes a few extra milliseconds. Acceptable for interactive use, but not for high-frequency batch processing.

The Takeaway

Free tokens are only worth something if you can use them without leaking what matters. A local filtering proxy is a small piece of code that gives you control over what leaves your machine. MonkeyCode's free server and 10-million-token grant become more useful when you can send code through a clean pipe. Run the proxy, test it with your own patterns, and keep the sensitive parts where they belong.

MonkeyCode provides free models that can run this workflow.

Weekend Build Log: A Slim Context Pack for Off-Laptop AI Patches

Charlie Hu — Thu, 03 Sep 2026 16:05:34 +0000

AI coding sessions pollute a working tree faster than they produce a mergeable patch. A weekend-sized fix is to compile a slim context pack, run the model work on a disposable server, and import only allowlisted paths. The local repo stays the source of truth. The remote box stays throwaway.

This log records a scoped side project: a pack compiler, a patch importer, and a short decision table for what never leaves the laptop. Fancy review scoring, commit-message generation, and debt heatmaps were left out on purpose. Those problems already have other notes. This one is about isolation and context waste.

Core constraints for the weekend

The build had three hard limits. No new SaaS account. No rewrite of the host application. No claim that a model “understands” architecture because a README was pasted into a prompt.

Pack size is measured in files and lines, not in vibes.
The remote session may write anything. The laptop only accepts paths on an allowlist.
Unexecuted examples are labeled. No fabricated latency, token burn, or pass-rate numbers appear below.

Those limits forced a boring shape: Python 3, git, and a few shell commands. The interesting part is the contract between pack, remote, and import.

The failure this tool targets

Free-tier models do not fail only by writing wrong code. They fail by reading the wrong tree. A whole monorepo dump mixes generated clients, vendored snapshots, and half-finished spikes. The patch that comes back then touches lockfiles, CI, and a public API in one sitting.

A second failure is local mixing. Generated files land next to real work. git diff becomes a negotiation. Reverts get personal.

The pack-and-import loop treats both as the same bug: unbounded context in, unbounded paths out.

What shipped in the demo

The working demo is three files plus a constraints document.

constraints/PACK.md — human rules the pack compiler also parses.
pack.py — walks a frozen file list and emits context_pack.json.
import_patch.py — applies a unified diff only if every path is allowlisted.
allowlist.txt — the only paths that may return from the remote session.

A reader can run the compiler and the importer without a model. That was the acceptance test for the weekend: the harness is useful even when the remote side is a stub.

PACK.md as a machine-readable freeze

PACK.md is not marketing copy. It is a freeze of what the model is allowed to know.

# PACK freeze
include:
  - src/billing/api.py
  - src/billing/types.py
  - tests/test_billing_api.py
exclude_globs:
  - "**/generated/**"
  - "**/*lock*"
  - ".env*"
max_file_lines: 400
public_symbols_only: true
never_send:
  - secrets
  - production hostnames
  - customer fixtures

The compiler refuses to pack a file that is not listed under include. Globs in exclude_globs are a second fence, not a substitute for the explicit list. Explicit beats clever when the failure mode is “the model saw a private key example.”

pack.py

The script below is a complete, runnable compiler for the freeze format above. It is a proposal for a weekend tool, not a report of production traffic.

#!/usr/bin/env python3
"""Compile a slim context pack from PACK.md. Unexecuted against private repos."""
from __future__ import annotations

import json
import re
from pathlib import Path

ROOT = Path(".")
PACK_MD = ROOT / "constraints" / "PACK.md"
OUT = ROOT / "context_pack.json"

INCLUDE_RE = re.compile(r"^\s+-\s+(\S+)")
GLOB_RE = re.compile(r'^\s+-\s+"([^"]+)"')


def parse_pack_md(text: str) -> dict:
    section = None
    spec = {"include": [], "exclude_globs": [], "max_file_lines": 400,
            "public_symbols_only": False, "never_send": []}
    for raw in text.splitlines():
        line = raw.rstrip()
        if line.startswith("include:"):
            section = "include"
            continue
        if line.startswith("exclude_globs:"):
            section = "exclude_globs"
            continue
        if line.startswith("never_send:"):
            section = "never_send"
            continue
        if line.startswith("max_file_lines:"):
            spec["max_file_lines"] = int(line.split(":", 1)[1].strip())
            section = None
            continue
        if line.startswith("public_symbols_only:"):
            spec["public_symbols_only"] = line.split(":", 1)[1].strip() == "true"
            section = None
            continue
        if section == "include":
            m = INCLUDE_RE.match(line)
            if m:
                spec["include"].append(m.group(1))
        elif section in {"exclude_globs", "never_send"}:
            m = GLOB_RE.match(line) or INCLUDE_RE.match(line)
            if m:
                spec[section].append(m.group(1))
    return spec


def public_slice(src: str) -> str:
    keep = []
    for line in src.splitlines():
        stripped = line.strip()
        if stripped.startswith(("def ", "class ", "async def ", "@", "from ", "import ")):
            keep.append(line)
        elif stripped.startswith(("#", '"""', "'''")) and keep:
            keep.append(line)
    return "\n".join(keep)


def main() -> None:
    spec = parse_pack_md(PACK_MD.read_text(encoding="utf-8"))
    files = []
    for rel in spec["include"]:
        path = ROOT / rel
        if not path.is_file():
            raise SystemExit(f"missing include: {rel}")
        text = path.read_text(encoding="utf-8", errors="replace")
        lines = text.splitlines()
        if len(lines) > spec["max_file_lines"]:
            text = "\n".join(lines[: spec["max_file_lines"]]) + "\n# [truncated]\n"
        if spec["public_symbols_only"] and path.suffix == ".py":
            text = public_slice(text)
        files.append({"path": rel, "bytes": len(text.encode()), "text": text})
    payload = {
        "spec": {k: spec[k] for k in spec if k != "never_send"},
        "file_count": len(files),
        "total_bytes": sum(f["bytes"] for f in files),
        "files": files,
        "instruction": (
            "Edit only files listed in allowlist.txt. "
            "Return a unified git diff. Do not invent paths."
        ),
    }
    OUT.write_text(json.dumps(payload, indent=2), encoding="utf-8")
    print(f"wrote {OUT} files={payload['file_count']} bytes={payload['total_bytes']}")


if __name__ == "__main__":
    main()

A typical compile looks like this:

python3 pack.py
python3 - <<'PY'
import json
from pathlib import Path
p = json.loads(Path("context_pack.json").read_text())
print(p["file_count"], p["total_bytes"])
for f in p["files"]:
    print(f["path"], f["bytes"])
PY

If total_bytes is still large, the include list is wrong. The fix is to cut files, not to “summarize the repo” with another model. Summaries hide the exact signatures the patch has to match.

Off-laptop session, on-laptop import

The remote side can be any disposable box the operator already trusts: a free server option, a spare VM, or a container that is deleted after the diff is copied out. The laptop never mounts the working tree as writable for the model.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is relevant here only as one place that currently offers free model access and a free server option for this kind of scratch session. No model names, quotas, hardware, or durability claims are attached to that mention. The scripts above do not depend on it.

A minimal remote loop, labeled as a proposed workflow:

# on the laptop
scp context_pack.json allowlist.txt user@scratch:~/session/
ssh user@scratch 'mkdir -p ~/session/work && cd ~/session/work && git init -q'

# on the scratch box (operator-supplied editor or model session)
# write files, then:
cd ~/session/work
git add -A
git diff --cached > ~/session/ai.patch

# back on the laptop
scp user@scratch:~/session/ai.patch ./incoming.patch
python3 import_patch.py incoming.patch allowlist.txt

The importer is the actual product of the weekend. Everything else is packaging.

import_patch.py

#!/usr/bin/env python3
"""Apply a unified diff only when every path is allowlisted."""
from __future__ import annotations

import subprocess
import sys
from pathlib import Path

FORBIDDEN_PREFIXES = (
    ".git/", ".env", "id_rsa", "id_ed25519", "secrets/", "prod/"
)


def paths_from_diff(diff: str) -> list[str]:
    found: list[str] = []
    for line in diff.splitlines():
        if line.startswith("+++ b/"):
            found.append(line[6:])
        elif line.startswith("diff --git "):
            parts = line.split()
            if len(parts) >= 4 and parts[3].startswith("b/"):
                found.append(parts[3][2:])
    return sorted(set(p for p in found if p != "/dev/null"))


def main() -> None:
    if len(sys.argv) != 3:
        raise SystemExit("usage: import_patch.py incoming.patch allowlist.txt")
    diff = Path(sys.argv[1]).read_text(encoding="utf-8", errors="replace")
    allow = {
        line.strip()
        for line in Path(sys.argv[2]).read_text(encoding="utf-8").splitlines()
        if line.strip() and not line.startswith("#")
    }
    paths = paths_from_diff(diff)
    if not paths:
        raise SystemExit("no file paths in patch")
    blocked = [p for p in paths if p not in allow]
    dangerous = [p for p in paths if p.startswith(FORBIDDEN_PREFIXES) or p in {"/etc/passwd"}]
    if dangerous:
        raise SystemExit(f"refusing dangerous paths: {dangerous}")
    if blocked:
        raise SystemExit(f"paths not on allowlist: {blocked}")
    subprocess.run(["git", "apply", "--check", sys.argv[1]], check=True)
    subprocess.run(["git", "apply", sys.argv[1]], check=True)
    print("imported", ", ".join(paths))


if __name__ == "__main__":
    main()

git apply --check is the cheap safety net. The allowlist is the policy. Policy first, git second.

A tight allowlist for the billing example:

# allowlist.txt — nothing else returns from the scratch box
src/billing/api.py
src/billing/types.py
tests/test_billing_api.py

If the remote patch also rewrites pyproject.toml, import fails. That failure is the feature. Dependency edits belong on a later weekend with a lockfile review, not in the same blast as a handler change.

Decision table used while cutting scope

Candidate feature	Shipped?	Reason
PACK.md freeze + compiler	Yes	Directly cuts context
Path allowlist importer	Yes	Directly cuts blast radius
`git apply --check`	Yes	Already on the machine
Token or cost dashboard	No	Needs vendor-specific counters
Multi-file architecture map	No	Weekend would become a static-analysis product
Auto-merge on green tests	No	Hides the import policy
Chat UI on the scratch box	No	Not required to prove the loop
Secret scanning beyond prefixes	Partial	Prefix denylist only

The cuts were not aesthetic. A dashboard would have turned the post into a vendor bake-off. An architecture map would have duplicated work already sitting in other write-ups. Auto-merge would have deleted the only human checkpoint the design still has.

A reproducible test plan (not executed here)

Label: proposed tests. No pass/fail metrics are claimed.

Happy path. Pack three listed files. Create a patch that edits src/billing/api.py only. Import succeeds. git status shows that one file.
Lockfile sneak. Same pack. Patch also edits poetry.lock. Import exits non-zero. Working tree unchanged.
Path invention. Patch adds src/billing/admin_backdoor.py. Import exits non-zero.
Truncation. A listed file exceeds max_file_lines. Pack contains a # [truncated] marker. No silent omission.
Missing include. PACK.md lists a file that does not exist. Compiler exits non-zero before any remote copy.
Round trip. After a good import, git diff --stat matches the allowlisted paths exactly.

Commands for tests 1–3:

python3 pack.py
# construct fixtures/good.patch and fixtures/lockfile.patch by hand
python3 import_patch.py fixtures/good.patch allowlist.txt
! python3 import_patch.py fixtures/lockfile.patch allowlist.txt

The ! is a shell assertion: a non-zero importer is success for the negative case.

What this does not solve

The pack compiler does not understand domain rules. It will happily ship a public function that must never change. Humans still own the include list.

The importer does not prove behavioral correctness. A patch can stay inside the allowlist and still break billing. Tests on the laptop remain mandatory after import.

Public-symbol slicing can drop comments that a human reviewer needed. That is a trade for smaller packs, not a quality upgrade.

A free remote box is not an isolation boundary against a determined leak. It is a convenience fence against accidental mixing. Secrets should not be on that box at all.

Who should skip this approach

Teams that already generate patches inside a reviewed agent sandbox with path policies.
Repos where the public surface is the whole tree and an include list would be dishonest.
Work that cannot leave the laptop for compliance reasons. A “free server” does not change data-handling rules.
Anyone hoping the pack will replace design. A freeze file is a fence, not an architecture.

Weekend residue

Kept: the freeze file, the compiler, the importer, and the habit of copying a diff instead of cloning a dirty tree back onto a laptop.

Skipped: model routing, reviewer scoring, session replay, and any dashboard that would have required invented quotas.

The next small step, if any, is a test that fails when PACK.md and allowlist.txt drift apart. That check is ten lines and belongs in CI. It does not need a product name to be worth running. Operators who already have a scratch box — MonkeyCode’s free server option is one such box — can drop context_pack.json there and keep the same import gate at home.

Weekend Build Log: A Cheap Technical Debt Radar for AI-Generated Code

Charlie Hu — Wed, 02 Sep 2026 14:04:04 +0000

Conclusion first: In one weekend, I built a lightweight scanner that flags AI-generated code with a high risk of accumulating tech debt. It runs on free model calls and a free server. The key was cutting scope aggressively, not building a big pipeline.

Recent community threads about technical debt in an AI-heavy workflow keep circling the same problem: code gets cheaper to create, but the cost shows up later. Most teams cannot reliably tell which files came from a model. Even when they can, there is no quick way to estimate how much that code will hurt the next person who touches it.

So I built a tiny radar. It scans a repo, finds files that look AI-generated, scores them on a rough debt risk scale, and publishes an HTML report. Nothing fancy. It just gives a starting point for a human review.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The one-sentence pitch

A CLI tool that takes a repository path and outputs a JSON + HTML report of low, medium, and high risk files.

What I actually built

debt_radar.py — the scanner and scorer.
A very basic HTML template to visualise the results.
A shell script to run it over a test repo.

That is the whole thing. No database. No background workers. No auth.

The scope cut

My first plan had three parts:

Scan the full git history and show debt trends over time.
Post inline comments on PRs when risky files are added.
Support JavaScript, Python, and Java.

All three got cut. I kept only one language (Python), only the current working tree, and no server-side automation. The report is static HTML.

That cut is why it shipped in a weekend.

How the scanner works

Step 1: Find candidate files

The scanner looks for markers commonly left by code generators:

generated by, auto-generated, or do not edit
Large single-function blocks with no explanation
Unusual indentation patterns from model output

A simple regex catches most of it.

import re

def looks_generated(text: str) -> bool:
    patterns = [
        r"auto[- ]generated",
        r"generated by",
        r"do not edit",
        r"spawned by"
    ]
    return any(re.search(p, text, re.I) for p in patterns)

Step 2: Score the risk

I used a small heuristic first, then sent the file to an LLM through MonkeyCode's free model access for a second opinion. The heuristic gives a baseline; the LLM adjusts it based on context like unexplained branches or missing error handling.

def heuristic_score(text: str) -> int:
    score = 0
    lines = text.splitlines()
    if len(lines) > 400:
        score += 10
    if len(re.findall(r"TODO|FIXME", text)) > 2:
        score += 15
    if not re.search(r"def |class ", text):
        score += 5
    return score

The final score is min(100, heuristic + llm_adjustment). The JSON report includes both numbers so you can see where the judgement came from.

Step 3: Host the report

MonkeyCode's free server option lets me host a static page without provisioning anything. I generate the HTML locally and push it with a simple scp command. That covers the demo.

A real run on a test repo

I ran the scanner on a small repository that already had a mix of hand-written and model-generated files.

python debt_radar.py --repo . --output report.json

The output looked like this:

{
  "files": [
    {
      "path": "src/parser.py",
      "heuristic": 35,
      "llm_adjustment": 20,
      "final": 55,
      "reason": "large parser with no exceptions handled"
    },
    {
      "path": "src/utils.py",
      "heuristic": 5,
      "llm_adjustment": 0,
      "final": 5,
      "reason": "looks hand-written and well documented"
    }
  ]
}

From there, the HTML report rendered red, yellow, and green rows. That was the demo.

What I skipped on purpose

Git history trend — would have required an index and a date-based query layer.
PR automation — needs authentication, event handling, and a merge gate.
Multi-language support — each language needs its own style rules and AST parsing.

All of those are valuable. None of them are needed to test the core idea: can a cheap tool tell you where AI code might cause pain?

Limitations

The heuristic is naive. It misses a lot of generated code that does not carry obvious markers.
The LLM adjustment is based on a prompt I wrote in thirty minutes. You can improve it, but it will never be precise.
The risk score is a hint, not a measurement. It cannot replace a human reading the file.
I only tested it on Python. Your mileage will vary on other languages and codebases.

Who should not use this

Teams that need strict governance and audit trails. This tool gives opinions, not compliance.
Teams with a tiny codebase where you already know every file. The noise outweighs the signal.
Anyone expecting a zero-config product. This is a weekend prototype, not a mature tool.

Final note

If your team is drowning in AI-generated PRs, a $0 fanout like this is a reasonable first step. Run it once, look at the red rows, and decide if the conversation is worth having. I learned more from cutting scope than from the features I originally wanted.

If you want to try the same approach, grab MonkeyCode's free model access and a free server slot — both work without a credit card.

Free-Tier AI Commit Messages: A MonkeyCode Build Log

Charlie Hu — Tue, 01 Sep 2026 13:14:28 +0000

Free-Tier AI Commit Messages: A MonkeyCode Build Log

Most code generation tools ask for a credit card before you can even test a realistic workflow. MonkeyCode's open-source platform takes a different approach: free model tokens and a free server tier for small apps. For a weekend project, that's enough to build something genuinely useful — a tool that reads your staged Git diff and suggests a commit message.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

This build log covers the full stack: a FastAPI service that calls the model API, a local CLI that sends diffs, and deployment onto MonkeyCode's free hosting. The result is a practical pattern you can reuse with any API provider.

Why a commit-message generator?

Commit messages are a low-stakes, high-frequency task. They are also easy to brute-force with an LLM because the input is a diff and the output is a short sentence. The model only sees the diff you give it, so there is no hidden context to trust — addressing the "AI remembers everything" concern from a recent DEV discussion.

The project also gives you a safe way to play with free-tier limits without risking anything important. If the API is down or the generated message is bad, you lose a few seconds, not a deployment.

Architecture overview

Local CLI: captures git diff --cached and POSTs it to your server.
Free server: runs a small FastAPI app that forwards the diff to MonkeyCode's model API.
Model API: returns a commit message, which the server relays back to the CLI.

The two-part design matters because it keeps your API key off your laptop and lets you reuse the same endpoint from multiple machines.

Step 1: Get API access

Create an account and grab an API key from MonkeyCode's dashboard. The free tier currently includes 10,000,000 tokens and a free server slot — enough for hundreds of small requests.

Verify current quotas and endpoint details in the official docs, since free tiers change over time.

Step 2: Write the FastAPI service

Here's a minimal server that accepts a diff and returns a suggested commit message. Replace MONKEYCODE_API_URL and MODEL_ID with the values from the current documentation (I'm using placeholders because these vary by region and billing plan):

# server.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import requests
import os

app = FastAPI()

class DiffRequest(BaseModel):
    diff: str

@app.post("/generate")
async def generate(req: DiffRequest):
    api_key = os.getenv("MONKEYCODE_API_KEY")
    if not api_key:
        raise HTTPException(status_code=500, detail="API key not configured")

    url = os.getenv("MONKEYCODE_API_URL")  # e.g., https://api.monkeycode.ai/v1/chat/completions
    headers = {"Authorization": f"Bearer {api_key}"}
    payload = {
        "model": os.getenv("MONKEYCODE_MODEL", "placeholder-model"),
        "messages": [
            {"role": "system", "content": "You are an expert at writing concise, conventional Git commit messages."},
            {"role": "user", "content": f"Write a commit message for this diff: {req.diff}"}
        ]
    }

    try:
        resp = requests.post(url, json=payload, headers=headers, timeout=30)
        resp.raise_for_status()
        data = resp.json()
        return {"message": data["choices"][0]["message"]["content"].strip()}
    except Exception as e:
        raise HTTPException(status_code=502, detail=str(e))

Run locally with uvicorn server:app --reload if you want to test before deploying.

Step 3: Deploy to MonkeyCode's free server

MonkeyCode's free server accepts standard containerized services. After you commit the server.py and a minimal requirements.txt, use their CLI or dashboard to deploy. For example, their CLI often looks like:

mc deploy --app my-commit-bot

I'm not going to copy every command because the precise workflow depends on your registry and region. The good news: the free server is intended for small personal projects, so you don't need to tune a Dockerfile or configure autoscaling.

Set the environment variables MONKEYCODE_API_KEY, MONKEYCODE_API_URL, and MONKEYCODE_MODEL in the dashboard after deployment.

Step 4: Build the local CLI

The CLI is a 20-line Python script that collects the staged diff and calls your server:

# commit-msg.py
import subprocess, sys, requests

def get_diff():
    result = subprocess.run(["git", "diff", "--cached"], capture_output=True, text=True)
    return result.stdout

def ask_server(diff):
    url = sys.argv[1] if len(sys.argv) > 1 else "https://your-app.onmck.dev/generate"
    response = requests.post(url, json={"diff": diff}, timeout=30)
    response.raise_for_status()
    return response.json()["message"]

if __name__ == "__main__":
    diff = get_diff()
    if not diff.strip():
        print("No staged changes. Run git add first.")
        sys.exit(1)

    message = ask_server(diff)
    print("Suggested commit message:")
    print(message)

Make it executable and run:

chmod +x commit-msg.py
./commit-msg.py https://your-app.onmck.dev/generate

What I learned about free tiers

Token limits go faster than you think. Each diff can be anywhere from 500 to 3,000 tokens when you include context. At 1,000 tokens per call, 10M tokens gives you ~10,000 requests — generous for personal use, but not for a team.
Latency varies. Free server instances may go to sleep. The first request after a pause can take 5–10 seconds extra. That's fine for interactive use, not for CI.
Error handling is essential. The API can return malformed JSON or rate-limit you. Always wrap the call in try/except and fall back to a manual message.

Who should not use this approach

Teams that generate dozens of messages per hour during code review.
Projects where commit messages must follow a strict internal format beyond convention.
Anyone who needs guaranteed uptime for a mission-critical pipeline.

For those cases, a paid plan with a service-level agreement makes more sense. But for a side project or a personal productivity boost, the free tier gets the job done.

Try it this weekend

The full pattern — local CLI, serverless API, free model — can be cloned and running in under an hour. If you want a zero-cost AI assistant that writes your commit messages, give MonkeyCode's free tier a shot. The code above is a solid starting point, and you can extend it to add emoji suggestions, multiple language styles, or even a pre-commit hook.

Weekend Build Log: A Scoring Harness for AI Code Reviewers on a Free Tier

Charlie Hu — Mon, 31 Aug 2026 12:19:40 +0000

Most AI review setups cannot tell you whether the reviewer is any good. This post shows a 100-line harness that scores AI code review comments against seeded bugs, built on MonkeyCode's free model access and free server option. The entire experiment costs nothing but a weekend.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why test the reviewer?

Recent developer discussions keep circling the same gap: AI promoted every developer to reviewer, but nobody tested the reviewer. A PR review comment looks plausible even when it misses the actual bug. The cheapest way to measure review quality is a fixture-based harness.

You plant a known bug in a diff, run the AI reviewer, and check whether its comments mention the bug. Repeat with ten diffs and you get a signal. That signal becomes even more useful when you tweak prompts and watch the score move.

MonkeyCode is an open-source AI coding toolkit. The project ships a server that you can run locally or use through their free server option. As of August 2026, the free tier includes 10 million tokens, which is plenty for a small review-scoring experiment.

What I built

A Python script called review_scorer.py and a folder named prs/. Each file in prs/ is a JSON fixture with three fields:

{
  "id": "pr-007",
  "diff": "--- a/auth.py\n+++ b/auth.py\n@@ -10,7 +10,7 @@ def login(user):\n-    if token is None:\n+    if token = None:\n        raise ValueError(\"missing token\")",
  "expected_flags": ["assignment in condition", "token handling"]
}

The script sends the diff to the MonkeyCode review server, collects the comments, and checks which seeded issues were mentioned.

The scoring core

Three functions do the heavy lifting:

import json
import os
import requests
from pathlib import Path

def run_review(diff: str, endpoint: str) -> list[str]:
    response = requests.post(
        endpoint,
        json={"prompt": build_prompt(diff)},
        timeout=int(os.getenv("REVIEW_TIMEOUT", "60"))
    )
    response.raise_for_status()
    return response.json()["comments"]

def score_comments(comments: list[str], expected: list[str]) -> tuple[int, int]:
    hits = sum(1 for e in expected if any(e.lower() in c.lower() for c in comments))
    return hits, len(expected)

def run_suite(prs: Path, endpoint: str) -> None:
    for fixture in prs.glob("*.json"):
        data = json.loads(fixture.read_text())
        comments = run_review(data["diff"], endpoint)
        hits, total = score_comments(comments, data["expected_flags"])
        print(f"{data['id']}: {hits}/{total} seeded bugs caught")

The code above is a simplified prototype. The prompt format and response schema will change depending on how you expose MonkeyCode in your environment. The point is the metric loop: change prompt, rerun, compare.

The rubric

I used a simple binary check: does the review comment contain any of the seeded flags? This is crude but effective.

Seed	Caught	Missed
assignment in condition	✅	❌
hardcoded secret	✅	❌
missing boundary check	❌	✅

You can extend the rubric with synonyms, severity parsing, or a second LLM call that judges whether the comment targets the same line. That turns the script from a toy into a real prompt-tuning instrument.

What got cut

No web UI. CLI output was enough.
No multi-turn conversation. This harness sends one diff and reads one response.
No model comparison. I only used the free model.
No human reviewer baseline. That would require a separate controlled study.

Every one of those cuts kept the project inside a single weekend.

What the harness teaches you

The first run of any fixture will likely surprise you. The free model will sometimes catch a bug you thought was obscure. Then it will miss a simpler bug right next to it. That variance is normal.

Run each fixture three times and take the median score. The harness makes that easy because the whole suite takes a few minutes on MonkeyCode's free server.

The biggest win is prompt iteration. For example, adding a line to the prompt that says "report line numbers and severity" usually changes recall more than changing the temperature. Without a scoring harness, you would judge that change by vibes. With it, you get a number.

Limitations

This is not a benchmark. The sample size is small, and the seeded issues are written by one person. The keyword matcher does not verify whether a comment is actually correct; it only checks whether the expected phrase appears.

AI reviewers are nondeterministic. Different runs produce different comments. Free tiers often have rate limits, and the exact token allowance may change. Always check the project documentation before running a large suite.

Who should not use this

Do not turn this harness into a formal review gate. It is a personal quality-checking tool, not an auditable process.

If your organization requires deterministic, compliant, or legally defensible code review, this approach is too loose. Payment code, medical software, and safety-critical systems need human review plus static analysis. A keyword-based score is not a substitute for real review.

Final notes

The harness lives in one folder with ten JSON fixtures and a single script. Adapting it to any AI review endpoint takes less than thirty minutes.

If you have been shipping AI review comments without measuring them, start with a fixture-based harness. With MonkeyCode's free tier, the cost is zero, and the insight is worth the weekend.

A Cheap, Repeatable Way to Catch AI Code That Ignores Your Repo Rules

Charlie Hu — Sun, 30 Aug 2026 11:25:59 +0000

Liquid syntax error: Unknown tag 'endraw'

Weekend Build Log: A Session Snapshot Trail for AI Coding — What I Built, Cut, and Kept

Charlie Hu — Sat, 29 Aug 2026 10:36:11 +0000

Reviewing AI-generated code fails most often at the boundary between turns, not between files. The merged diff hides the order in which edits happened, and the order is where intent breaks. This weekend build produced a small CLI that snapshots the working tree every few seconds, groups changes into turns, and writes a per-turn review note. The tool is deliberately small, the notes cost almost nothing in tokens, and the whole experiment stayed on free infrastructure.

The problem behind the build

AI coding tools turned every developer into a reviewer. Part of the current developer discussion asks what the human should do while the model does the editing. A previous post on this account covered a merge gate for AI-generated code. This build is the trail behind that gate: a record of what the AI touched, in what order, and where the human should look first.

The gap is simple. A combined diff shows the final state, not the sequence of edits. A one-line config change made early can be visually buried under a large formatting shift from a later turn. Reviewers either trust the model or rebuild the timeline by hand. Neither is a good option.

The shape of the tool

The tool is called snaplog. It polls git status instead of watching filesystem events. Polling is cross-platform, and human-paced AI edits do not need millisecond precision.

Every 5 seconds it reads the changed-file list and diff statistics for the current branch.
When no change appears for 90 seconds, the current turn is closed.
Each turn records start time, end time, changed files, and added/deleted line counts.
On Ctrl+C the session renders a Markdown report with one section per turn.

Turn detection is a heuristic. Long silent pauses split one turn into two, and two fast bursts can merge. For triage, that is acceptable.

The core script

snaplog uses only the Python standard library and git:

#!/usr/bin/env python3
"""snaplog — build a review trail for an AI-assisted coding session.

Start it, let the AI edit, press Ctrl+C to stop.
A Markdown report grouped into edit turns is written to session-report.md.
Only the Python standard library and git are required.
"""
import argparse
import json
import subprocess
import time
from datetime import datetime
from pathlib import Path

INTERVAL = 5   # seconds between git status polls
IDLE_GAP = 90  # seconds without edits -> the current turn ends


def git(*args: str) -> str:
    out = subprocess.run(["git", *args], capture_output=True, text=True)
    return out.stdout.strip()


def current_changes() -> tuple[list[str], dict[str, tuple[int, int]]]:
    status = git("status", "--porcelain")
    files = sorted(line[3:] for line in status.splitlines() if line.strip())
    stats: dict[str, tuple[int, int]] = {}
    for line in git("diff", "--numstat", "HEAD").splitlines():
        added, deleted, path = line.split("\t", 2)
        stats[path] = (int(added), int(deleted))
    return files, stats


def add_summary(turn: dict, command: str) -> None:
    result = subprocess.run(
        command, shell=True, input=json.dumps(turn),
        capture_output=True, text=True,
    )
    turn["summary"] = result.stdout.strip()


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--summarizer",
        help="command that reads a turn JSON on stdin and prints review notes",
    )
    args = parser.parse_args()

    turns = []
    current = None
    last_change = time.monotonic()

    try:
        while True:
            files, stats = current_changes()
            now = time.monotonic()
            if files:
                if current is None:
                    current = {
                        "start": datetime.now().isoformat(timespec="seconds"),
                        "files": [],
                        "stats": {},
                    }
                current["files"] = sorted(set(current["files"]) | set(files))
                current["stats"].update(stats)
                last_change = now
            elif current and now - last_change > IDLE_GAP:
                current["end"] = datetime.now().isoformat(timespec="seconds")
                if args.summarizer:
                    add_summary(current, args.summarizer)
                turns.append(current)
                current = None
            time.sleep(INTERVAL)
    except KeyboardInterrupt:
        if current:
            current["end"] = datetime.now().isoformat(timespec="seconds")
            if args.summarizer:
                add_summary(current, args.summarizer)
            turns.append(current)

    lines = ["# Session review trail\n"]
    for index, turn in enumerate(turns, 1):
        lines.append(f"## Turn {index} — {turn['start']} → {turn['end']}")
        for path, (added, deleted) in sorted(turn["stats"].items()):
            lines.append(f"- `{path}` (+{added}/-{deleted})")
        if turn.get("summary"):
            lines.append(f"> {turn['summary']}")
        lines.append("")
    Path("session-report.md").write_text("\n".join(lines))
    print(f"wrote session-report.md with {len(turns)} turns")


if __name__ == "__main__":
    main()

The summarizer is a separate seam. It reads a turn JSON object from stdin and prints a short review note:

#!/usr/bin/env python3
"""Adapter for snaplog --summarizer.

Reads a turn JSON object from stdin and prints a short review note.
Expects an OpenAI-compatible chat-completions endpoint at SNAPLOG_API_URL.
This weekend build pointed SNAPLOG_API_URL at MonkeyCode's free model access.
The response contract is the only assumption here.
"""
import json
import os
import sys
from urllib import request

turn = json.load(sys.stdin)
file_lines = [
    f"- {path} (+{added}/-{deleted})"
    for path, (added, deleted) in sorted(turn["stats"].items())[:20]
]
prompt = (
    "Summarize one AI-assisted coding turn for a human reviewer. "
    "Write two bullets: what likely changed, and what deserves a close look.\n"
    + "\n".join(file_lines)
)

payload = {
    "model": os.environ.get("SNAPLOG_MODEL", "default"),
    "messages": [{"role": "user", "content": prompt}],
}
headers = {"Content-Type": "application/json"}
if key := os.environ.get("SNAPLOG_API_KEY"):
    headers["Authorization"] = f"Bearer {key}"

req = request.Request(
    os.environ["SNAPLOG_API_URL"],
    data=json.dumps(payload).encode(),
    headers=headers,
)
with request.urlopen(req) as resp:
    data = json.load(resp)
print(data["choices"][0]["message"]["content"])

Where the free pieces fit

The summarizer step is a seam. In this build, each closed turn was sent to MonkeyCode's free model access, and the returned notes were inserted under the matching turn. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project. Its current offering includes free model access (10 million tokens per the project's documentation at the time of writing) and a free server option. Both terms can change, so the README is the source of truth before anyone relies on the allowance for more than a weekend.

The free server option ran the second half of the demo. The Markdown report renders to one static HTML page, and one long-running process serves the directory. That workload fits any free tier:

python -m http.server 8080 --directory ./report_out

A collaborator opens the port and sees the whole session trail in a browser. Nothing needs to be installed on the reviewer's machine.

The working demo

The command that produced the report:

SNAPLOG_API_URL=<chat-completions-endpoint> \
SNAPLOG_API_KEY=<key-if-required> \
python snaplog.py --summarizer "python summarize.py"

After the AI finishes, Ctrl+C stops the loop. The file session-report.md appears with one section per edit turn, in this format (illustrative output, not a real session):

# Session review trail

## Turn 1 — 14:02:11 → 14:05:33
- `src/parser.py` (+34/-10)
- `tests/test_parser.py` (+18/-0)

> Turn 1 replaced the hand-rolled tokenizer loop with a table-driven
> parser. Check the empty-input edge case: the new branch returns
> early before setting `self.pos`.

The adapter expects an OpenAI-compatible chat-completions response shape. A different gateway contract means swapping the last three lines of summarize.py.

What got cut

Scope control was the point of the weekend.

Filesystem watchers — dropped. They add platform-specific code and permissions. Polling every 5 seconds is enough.
Full-diff summarization — dropped. Sending whole diffs to a model burns tokens fast. The adapter sends only filenames and line counts, capped at the first 20 files per turn.
CI integration and dashboards — dropped. Version one is a local CLI plus one static HTML page.
Token budgeting — dropped. The hard input cap and the idle heuristic are the budget.

What the trail changed

In the test session, the review started from the report instead of from an empty diff. The per-turn notes pointed at the risky file first, and the turn order exposed an early config edit that the final diff made easy to miss. The effect is structural: the human reads a session map before reading code.

Limitations

Git-only. If the AI writes to files outside the working tree, the trail misses them. Untracked files also appear in the file list without line stats.
Heuristic turns. Silent thinking time splits a turn; overlapping bursts merge.
No security layer. Filenames and line counts leave the machine. Pointing the adapter at code that cannot be shared is a bad idea.
Triage, not audit. This is not a replacement for reading the diff, and it is not an immutable audit trail.

Who should skip this

Teams that need signed, compliant audit trails should use a real ALM tool. Developers whose review process already catches intent changes will see friction without payoff. Anyone who prefers to re-derive session history by hand can keep doing that.

Closing

The useful result is a receipt. A reviewer gets a map of the session before opening the diff, and the map costs a small prompt per turn instead of an hour of archaeology. If this pattern fits a weekend workflow, MonkeyCode's free model allowance and free server option are a low-risk place to test it — current terms are in the project README, and they can change, so verify before starting. The script and the habit are both free.

Zero-Cost CI Flake Triage: Build Notes from a Free-Tier Weekend

Charlie Hu — Fri, 28 Aug 2026 04:02:32 +0000

Every AI-coding discussion this week keeps landing on the same bottleneck: the reviewer. More generated code reaches merge queues, and CI failures make the review signal noisier. A small webhook bot was built over one weekend to classify those failures automatically — running on a free server and free model tokens. The scope got cut hard, the demo ran, and the mislabels were more instructive than the hits.

Why flake triage became the test case

The current AI-tooling debate splits into two camps: teams that trust model output and teams that benchmark every claim. Both camps still share one pain point — flaky tests. A timeout, a leaked database state, an order-dependent test: each one interrupts a review and burns a human decision. A classifier that labels the flake type before a human opens the logs is a small but honest use of a language model.

The previous weekend project digested repositories. This one watches CI events instead. Two different bots, one shared lesson: free-tier capacity is enough for narrow tasks if the task stays narrow.

The scope cut, decided before the first line of code

The original plan had five features. The shipped version had two.

Planned	Shipped	Reason for the cut
Multi-repo webhooks	Single repo	Webhook auth config is its own project
Persistent history	In-memory queue	One weekend, no Redis
Slack alerting	PR label only	Fewer moving parts
Prompt versioning UI	Hardcoded prompt	Over-engineering
Model fallback on parse errors	Single retry	Free-tier constraint

The rule was simple: if a feature needed another database, another credential, or another UI, it did not ship.

Architecture: one server, two routes, one prompt

The whole bot is a Node process with two endpoints. POST /ci-event receives a test name and the tail of the failure log. The classifier maps the text to one of five buckets: TIMING, ORDER_DEPENDENT, NETWORK, STATE_LEAK, or UNKNOWN. A comment or label then lands on the PR.

The classification function is deliberately small:

// Sketch, not production code. Match the client to your endpoint's SDK.
const FLAKE_BUCKETS = ["TIMING", "ORDER_DEPENDENT", "NETWORK", "STATE_LEAK", "UNKNOWN"];

async function classifyFlake(failureText) {
  const prompt = [
    "You are a CI triage assistant. Put the failure below into exactly one bucket.",
    "TIMING: timeout or slow wait. ORDER_DEPENDENT: passes alone, fails in the suite.",
    "NETWORK: DNS, TLS, or external call. STATE_LEAK: shared DB or files, missing reset.",
    "Reply with one word only.",
    "",
    failureText.slice(0, 2000)
  ].join("\n");

  const reply = await model.chat({ prompt });
  return FLAKE_BUCKETS.includes(reply.trim()) ? reply.trim() : "UNKNOWN";
}

The route adds no cleverness:

app.post("/ci-event", async (req, res) => {
  const { testName, logTail } = req.body;
  const bucket = await classifyFlake(logTail);
  await labelPr(testName, bucket);
  res.json({ ok: true, bucket });
});

Two details matter. First, the prompt constrains the model to a fixed vocabulary, so parsing is a one-line check instead of regex wrangling. Second, any out-of-vocabulary reply becomes UNKNOWN, which keeps the bot honest instead of confident.

The free tier: where the server and the tokens came from

Hosting came from the free server option in MonkeyCode, so the webhook never touched a paid VM. The model calls ran against the free model tier, which lists 10 million tokens at the time of writing. A weekend of triage traffic for a single repo stays well inside that envelope, but the quota should be re-checked before pointing the bot at a busier pipeline.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The point of the exercise was not that free hosting exists. It was that a narrow task with a constrained prompt can run without card-on-file anxiety. The trade-offs showed up quickly: the free endpoint had no latency guarantee, and one malformed JSON reply required a retry. That is the realistic cost of a zero-dollar stack.

The demo: one request, one label

A real-looking failure from the local test run:

POST /ci-event
{
  "testName": "checkout_test",
  "logTail": "connect ETIMEDOUT 52.0.1.1:443\n    at TCPConnectWrap.afterConnect"
}

Response:

{ "ok": true, "bucket": "NETWORK" }

The obvious cases classify instantly. The interesting cases are the UNKNOWNs. Every unknown is a prompt bug or a genuinely weird failure, and both are worth a human look.

The evaluation plan worth copying

Model confidence is not evidence. The reproducible check from this project is a 30-sample manual comparison:

Pull 30 real failures from the repo's CI history.
Label each one by hand before running the classifier.
Run the bot over the same 30 logs.
Build a confusion matrix by bucket.
Keep the matrix somewhere visible.

This takes about an hour and produces a number the team can trust — unlike a marketing chart. For a small repo, 25–30 samples is enough to find systematic mislabeling in the two buckets that matter most.

What got skipped, and why that was correct

No authentication on the webhook endpoint. Fine for a personal demo, wrong for anything shared.
No durable queue. A server restart drops in-flight events.
No metrics dashboard. The confusion matrix script replaced it.
No multi-model comparison. The budget was one free endpoint, not a benchmark lab.
No persistence beyond memory. Labels are re-derivable from CI history.

Who should not copy this setup

Teams with compliance constraints should not send failure logs to any third-party model endpoint without checking data classification rules. Repos with very few flakes do not need a classifier; the noise of a mislabel outweighs the time saved. Anyone who needs a response within a strict SLA should pay for a managed queue and a guaranteed tier instead. Free-tier tooling is a decision, not a default.

The narrowest version of this project is the useful one: a single repo, a constrained prompt, and an evaluation script. If the same 30-sample check is run on another CI, the flake distribution that comes back will likely surprise the team that owns it. That confusion matrix is the part of this weekend worth copying.