A Free-Tier Trace Loop for Agent Run Regression

Agent runs are stochastic. You fix one bug, rerun, and a completely different failure shows up. The only way to keep up is to compare runs at the trace level, not just final outputs. That comparison typically needs a server and a bit of model budget—two things you can get free from MonkeyCode right now.

As of August 2026, MonkeyCode is an open-source project that offers two things relevant to this workflow: free model access and a free server option. The model access includes a 10-million-token allowance, enough for daily trace diffs on a small agent. The free server gives you a place to collect and query those traces without spinning up your own VPS. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The loop I use is simple: record every tool call and input/output pair from an agent run into a structured store; serialize each run into a single document; then ask a model to diff two documents and highlight behavioral changes. It’s not a replacement for a full APM suite, but it catches the regressions that matter most: the agent started calling the wrong tool, the order of arguments changed, or a guardrail was silently removed.

Here’s a minimal implementation using FastAPI and SQLite. The code assumes MonkeyCode exposes an OpenAI-compatible API, which you can set via environment variables.

import sqlite3
import json
import os
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import openai

# Configure MonkeyCode endpoint via env vars
client = openai.OpenAI(
    api_key=os.environ['MONKEYCODE_API_KEY'],
    base_url=os.environ.get('MONKEYCODE_BASE_URL', 'https://api.monkeycode.ai/v1')
)

app = FastAPI()
DB_PATH = 'traces.db'

def init_db():
    conn = sqlite3.connect(DB_PATH)
    conn.execute('''
        CREATE TABLE IF NOT EXISTS runs (
            run_id TEXT PRIMARY KEY,
            created_at TEXT,
            payload TEXT
        )
    ''')
    conn.commit()
    conn.close()

init_db()

class TraceUpdate(BaseModel):
    run_id: str
    created_at: str
    events: list

@app.post('/traces')
def ingest(trace: TraceUpdate):
    conn = sqlite3.connect(DB_PATH)
    conn.execute(
        'INSERT OR REPLACE INTO runs (run_id, created_at, payload) VALUES (?, ?, ?)',
        (trace.run_id, trace.created_at, json.dumps(trace.events))
    )
    conn.commit()
    conn.close()
    return {'status': 'ok'}

@app.get('/compare')
def compare(run1: str, run2: str):
    conn = sqlite3.connect(DB_PATH)
    r1 = conn.execute('SELECT payload FROM runs WHERE run_id=?', (run1,)).fetchone()
    r2 = conn.execute('SELECT payload FROM runs WHERE run_id=?', (run2,)).fetchone()
    conn.close()
    if not r1 or not r2:
        raise HTTPException(404, 'One or both run IDs not found')

    prompt = f'''Compare these two agent run traces and list concrete behavioral differences:
Run 1: {r1[0]}
Run 2: {r2[0]}
Return a bullet list of differences and a one-sentence verdict.'''
    resp = client.chat.completions.create(
        model=os.environ.get('MONKEYCODE_MODEL', 'gpt-4o-mini'),  # replace with a real model name from MonkeyCode
        messages=[{'role': 'user', 'content': prompt}],
        temperature=0.2
    )
    return {'diff': resp.choices[0].message.content}

To use it, send each run’s events to /traces as soon as the run finishes. Then hit /compare?run1=<old>&run2=<new> to get a natural-language diff. I deliberately keep the prompt minimal; you can extend it to include expected behavior or project-specific invariants.

To feed this service, add a small hook to your agent. At the end of a run, collect every tool call, input, and output into a list and POST once:

import requests
from datetime import datetime

def post_run(run_id, events):
    requests.post('https://your-free-server/traces', json={
        'run_id': run_id,
        'created_at': datetime.utcnow().isoformat(),
        'events': events
    })

Why does this work? Because the trace document is a lossy but sufficient summary. You don’t need every token of the conversation; you need the tool calls, their arguments, and the final visible result. Those are the places where agents regress. The LLM’s job is to look at two such documents and speak like a careful code reviewer: “Here’s what changed, here’s what might break.”

The whole pipeline fits on a free server. The SQLite database stays small for hundreds of runs. The token allowance, while finite, is large enough for a focused debugging session. If you stay disciplined—only diff runs that actually matter—you can get weeks of use out of the free tier.

What this is not good for: real-time alerting, high-frequency regression detection, or teams that need hundreds of comparisons per day. The free server may have cold starts, and the model access has a token cap you must budget. If your agent runs produce megabytes of trace data per hour, you need something heavier than a single-node SQLite box.

Deploying this service is no different from any FastAPI app. MonkeyCode's free server option supports Python apps, so you can push the code, set the two environment variables, and you’re live. The server gives you a public URL; use that as your trace endpoint.

But for a solo dev or a small team that just wants to stop squinting at logs, this loop is a solid baseline. It forces you to structure your traces, and it gives you a repeatable way to answer “what changed between yesterday and today?”.

If you’re already spending hours manually diffing agent logs, the free tier is enough to try this loop for a week. See for yourself.