Free-Tier CI Triage: One Small Bot, End to End

The useful AI story this month is not a new benchmark. It is that small, scheduled agent tasks have become cheap enough to run for free. This case study follows one such project end to end: a bot that reads the last 24 hours of CI failures, summarizes them with a free model, and posts a single markdown report every morning. The monthly cost is zero.

The project was a small open-source repository with a handful of contributors. CI failed in boring ways: a flaky integration test, a missing secret, a dependency version that drifted. The failures were visible, but nobody read the alert channel.

The goal was not another notification. The goal was one short report per day that a maintainer could read in two minutes and act on. The constraint was a budget of zero dollars.

The tooling decision came down to two requirements: model calls without an API key, and a cron job without a VPS. MonkeyCode, an open-source coding agent, answered both. Its free model access includes a 10-million-token allowance at the time of writing, and its free server option runs scheduled tasks. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Both offers are subject to change, which is exactly why the script below does not depend on them.

The bot is one Python file. It does three things: fetch failed workflow runs from the GitHub API, send them to an OpenAI-compatible chat endpoint, and print a report. The environment variables point at the free model endpoint, and the same variables are what let the script switch to a paid provider later. No MonkeyCode-specific SDK is imported, which keeps the bot boring and portable.

#!/usr/bin/env python3
"""Fetch failed CI runs and summarize them with a chat model."""
import json
import os
import urllib.request
from datetime import datetime, timedelta, timezone

REPO = os.environ.get("TRIAGE_REPO", "octocat/hello-world")
GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")
MODEL_URL = os.environ.get("MODEL_URL")   # OpenAI-compatible endpoint
MODEL_KEY = os.environ.get("MODEL_KEY", "")
MODEL_NAME = os.environ.get("MODEL_NAME", "")

def fetch_failures():
    since = (datetime.now(timezone.utc) - timedelta(hours=24)).isoformat()
    url = (f"https://api.github.com/repos/{REPO}/actions/runs"
           f"?status=failure&created=%3E={since}")
    req = urllib.request.Request(url)
    if GITHUB_TOKEN:
        req.add_header("Authorization", f"Bearer {GITHUB_TOKEN}")
    with urllib.request.urlopen(req) as resp:
        data = json.load(resp)
    return [
        {
            "name": r["name"],
            "head_branch": r["head_branch"],
            "created": r["created_at"],
            "url": r["html_url"],
        }
        for r in data.get("workflow_runs", [])[:20]
    ]

def summarize(runs):
    if not runs:
        return "No failed CI runs in the last 24 hours."
    prompt = (
        "You are triaging CI failures. Group the runs below by likely root "
        "cause: flaky test, missing secret, version drift, or config error. "
        "For each group give the count, the probable cause, and one next "
        "step. Be brief.\n\n" + json.dumps(runs, indent=2)
    )
    payload = {
        "model": MODEL_NAME,
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0.2,
    }
    req = urllib.request.Request(
        MODEL_URL,
        data=json.dumps(payload).encode(),
        headers={
            "Content-Type": "application/json",
            "Authorization": f"Bearer {MODEL_KEY}",
        },
    )
    with urllib.request.urlopen(req) as resp:
        out = json.load(resp)
    return out["choices"][0]["message"]["content"]

def main():
    report = summarize(fetch_failures())
    stamp = datetime.now(timezone.utc).strftime("%Y-%m-%d")
    print(f"# CI Triage {stamp}\n\n{report}")

if __name__ == "__main__":
    main()

Three details matter in that script. The created=%3E= filter keeps the request small. The prompt is the real product: it asks for groups, counts, probable causes, and next steps, not a paragraph. The temperature is low because triage is classification, not prose.

The free server ran one cron line:

0 7 * * * cd ~/ci-triage && python3 triage.py >> reports/$(date +\%F).md 2>&1

The output file became the morning report. A second line in the same crontab could push it to a channel or an issue; for this project, a file in the repository was enough.

The first attempt failed. The bot ran at 7am and posted a report that was just the raw JSON of failed runs, because the model had been given no instruction and echoed the input. The fix was the prompt, not a bigger model. A GitHub Action could have run the same script on a schedule, but the point was to test the free server, and a cron file is easier to inspect when something goes wrong.

Results are best read as a decision table. When the same test appears in every failure group, the cause is flaky. When the same workflow fails with different tests, the cause is configuration. When failures are one-offs with no pattern, the cause is usually the runner. A sample run produces a report like this (synthetic data, not from a real repository):

# CI Triage 2026-08-24

Group 1: flaky integration test (3 runs)
- e2e.yml on main, 03:11, 05:40, 06:02 UTC
- Probable cause: Playwright timeout on slow runners
- Next step: raise the timeout or mark the test retryable

Group 2: missing secret (2 runs)
- deploy.yml on release, 01:20 and 01:47 UTC
- Probable cause: DATABASE_URL not set in the new environment
- Next step: add the secret to the environment config

Signal in the report	Likely cause	Action
Same test in every group	Flaky test	Add a retry or fix the test
Same workflow, different tests	Config or secret drift	Check env vars and versions
One-off failures, no pattern	Runner or network blip	Ignore, or check runner logs

The table is also the prompt in disguise, because the model was asked to produce exactly these categories. That is the loop that makes the bot useful: the output format is fixed, so the report is always readable, even when the underlying model changes.

Three lessons came out of the project. The first is that prompt shape did more work than model size: the vague prompt produced paragraphs, the structured prompt produced a usable report.

The second is that scope decides whether a free tier is the right tool. A missed report costs nothing, so a free tier with no SLA is acceptable; the same setup would be reckless for a payment pipeline.

The third is about lock-in. The script talks to any OpenAI-compatible endpoint, so when the free allowance changes, the fix is two environment variables, not a rewrite.

The limitations are real. The bot sends workflow names, branch names, and failure metadata to a third-party model endpoint. That is fine for a public repository and wrong for a private one with sensitive pipeline names. The free server is a small, non-critical environment; if it is down, the report is simply missing. GitHub's unauthenticated API allows 60 requests per hour, which is plenty for a daily job, but a token is safer for repositories with many workflows.

Teams with strict data residency rules, confidential pipeline names, or guaranteed delivery requirements should skip this setup. The bot is a convenience, not an alerting system. It is for the maintainer who wants to stop reading CI logs and start reading one short file.

The setup is reproducible in an afternoon. The script is short enough to read in one sitting, and the free tier it runs on is worth testing while the failure cost of a missed report is low. That is the real point of the exercise: free infrastructure is only useful when the work it does is small, and this work is small by design.