From Zero to Deployed: Build a CI Failure Explainer Bot With Free Model Access

AI agents write more code every week. The debate about whether that is good is all over DEV right now. The practical problem is quieter: when that code fails in CI, the logs are still confusing. This tutorial ships a working bot that explains CI failures in seconds. It runs on free model access and a free server. You can build it in one afternoon.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What you are building

A webhook that receives a failing test log. It sends the log tail to a model. It returns a root-cause hypothesis, supporting evidence, and a suggested next step. Then it runs on a public server.

Here is the flow:

1. CI fails and posts the log tail to your webhook.
2. The webhook sends it to a model with a strict prompt.
3. The model returns a hypothesis, evidence, and a next step.
4. You verify the hypothesis against the real code.

The stack is boring on purpose:

• Python 3.11+
• FastAPI and httpx
• An OpenAI-compatible chat endpoint
• MonkeyCode, an open-source project with a free tier

MonkeyCode's free tier gives you two things: 10 million tokens and a free server. That is enough to build and host this bot.

Why this stack? It is small. It is testable. It costs nothing to start.

Stage 1: Confirm your model access

Create a MonkeyCode account and grab the free tier. Then set two environment variables:

export MONKEYCODE_API_KEY="your_key_here"
export MONKEYCODE_BASE_URL="https://api.monkeycode.example/v1"

The exact base URL lives in the README. Check it before you run anything.

Verify it. Send a minimal chat request:

curl $MONKEYCODE_BASE_URL/chat/completions \
  -H "Authorization: Bearer $MONKEYCODE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"default","messages":[{"role":"user","content":"Say OK"}]}'

You should see a JSON reply. A 401 means your key is wrong. A 404 means your base URL is wrong. Fix those before moving on.

Stage 2: Write the core explainer

Create a file called explain.py. It takes a test name and a log tail. It returns a structured analysis.

# explain.py
import json
import os
import httpx

BASE_URL = os.getenv("MONKEYCODE_BASE_URL")
API_KEY = os.getenv("MONKEYCODE_API_KEY")

SYSTEM_PROMPT = """
You are a CI failure analyst. You get a failing test name and a log tail.
Return JSON with three fields:
- hypothesis: the most likely root cause
- evidence: the log lines that support it
- suggested_fix: one concrete step to verify it
Be concise. Never claim certainty.
"""

def build_prompt(test_name: str, log: str) -> str:
    tail = log[-4000:]  # keep only the end of the log
    return f"Test: {test_name}\n\nLog tail:\n{tail}"

def parse_json(content: str) -> dict:
    content = content.strip()
    if content.startswith("```

"):
        content = content.split("\n", 1)[1].rsplit("

```", 1)[0]
    return json.loads(content)

async def explain(test_name: str, log: str) -> dict:
    payload = {
        "model": os.getenv("MONKEYCODE_MODEL", "default"),
        "messages": [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": build_prompt(test_name, log)},
        ],
        "temperature": 0.2,
    }
    async with httpx.AsyncClient() as client:
        resp = await client.post(
            f"{BASE_URL}/chat/completions",
            headers={"Authorization": f"Bearer {API_KEY}"},
            json=payload,
        )
        resp.raise_for_status()
        content = resp.json()["choices"][0]["message"]["content"]
    return parse_json(content)

Note the log truncation. Four thousand characters is usually enough. Long logs hide the real error in the middle. Keep the tail.

Verify it. Run a one-off check:

python -c "
import asyncio
from explain import explain
log = open('sample.log').read()
print(asyncio.run(explain('test_auth_flow', log)))
"

You should see a JSON object with three fields. If parsing fails, the model returned non-JSON. Lower the temperature or ask for plain text.

Stage 3: Wrap it in a webhook

Now expose the explainer over HTTP. Create app.py:

# app.py
from fastapi import FastAPI, Request
from explain import explain

app = FastAPI()

@app.post("/explain")
async def handle(request: Request):
    body = await request.json()
    result = await explain(body["test_name"], body["log"])
    return {"test_name": body["test_name"], "analysis": result}

That is the whole server. One route, zero database, zero state. Error handling is intentionally minimal. Add retries before you trust it in CI.

Verify it. Run it locally:

pip install fastapi uvicorn httpx
uvicorn app:app --reload

Then post a sample failure:

curl -X POST http://localhost:8000/explain \
  -H "Content-Type: application/json" \
  -d '{"test_name":"test_auth_flow","log":"AssertionError: expected 200, got 500"}'

You should get a JSON analysis back. If you do, the core works. If not, debug the model call first.

Stage 4: Deploy to the free server

Now move it off your laptop. MonkeyCode's free server hosts small services like this one. No VPS to manage. The exact command depends on the current CLI, so check the README. The pattern looks like this:

monkeycode deploy --name ci-explainer

Verify it. The CLI prints a public URL. Hit it:

curl -X POST https://ci-explainer.your-server.example/explain \
  -H "Content-Type: application/json" \
  -d '{"test_name":"test_auth_flow","log":"500 on login"}'

Same JSON, now on a public endpoint. That is your free server doing real work.

Stage 5: Wire CI to call it (optional)

Add a step to your existing test job. It posts the log tail when tests fail.

First, create scripts/post_failure.py:

# scripts/post_failure.py
import os
import sys
import httpx

url = os.environ["EXPLAINER_URL"]
log = open(sys.argv[1]).read()[-4000:]
payload = {"test_name": "ci", "log": log}
httpx.post(url, json=payload).raise_for_status()

Then add this step to your workflow:

- name: Explain failure
  if: failure()
  env:
    EXPLAINER_URL: ${{ secrets.EXPLAINER_URL }}
  run: python scripts/post_failure.py ci.log

Store the endpoint in EXPLAINER_URL. Make sure your test job writes its log to ci.log.

Verify it. Break a test on purpose. Watch the bot reply. Then fix the test and confirm the analysis was right.

Limitations

Be honest about what this bot does:

• It produces hypotheses, not fixes. Verify everything.
• It truncates logs. The real error may sit in the middle.
• The free tier is for experimentation, not production scale.
• Do not send secrets or proprietary code to any hosted model.
• The model can be confidently wrong. That is why the evidence field exists.

Who should skip this

Skip this approach if you need strict data residency. Skip it if your logs contain customer data. Skip it if you expect the bot to fix bugs autonomously. It will not. It will only save you the first twenty minutes of staring at a log.

Try it

Build the bot. Break a test. See if the hypothesis matches reality. That test is the whole point. If you want a free place to start, MonkeyCode's 10 million tokens and free server are enough for plenty of experiments.