Log the Decision, Not Just the Response

A free model endpoint returns text. It does not return context. When the output is wrong, you need evidence. This tutorial builds a lightweight audit ledger for every model call. It records routing decisions, latency, and validation results. You can ship it in about an hour.

Why a ledger, not a log file

Log files are prose. They are hard to query. They disappear under rotation. An audit ledger is structured data. Every call becomes one row in SQLite. You can ask questions later.

"Which model failed validation?" "What was the latency at 2 AM?" A log file cannot answer those questions. A ledger can.

AI turned every developer into a reviewer. Most reviewers have no evidence. The ledger is that evidence.

What the ledger records

Each row captures the full decision path:

• request_hash: a stable fingerprint of the prompt
• route: which feature triggered the call
• model: which endpoint served it
• decision: accepted, flagged, or error
• latency_ms: wall-clock time for the call
• prompt_tokens and completion_tokens: rough estimates
• validation: the output check result
• error_class: the exception type, if any
• response_preview: the first 200 characters

That is enough to reconstruct an incident. It is also enough to spot trends.

Stage 1: Create the schema

SQLite ships with Python. No database server is required. Create the database and run this DDL:

CREATE TABLE model_call_ledger (
  id INTEGER PRIMARY KEY,
  ts TEXT NOT NULL DEFAULT (datetime('now')),
  request_hash TEXT NOT NULL,
  route TEXT NOT NULL,
  model TEXT NOT NULL,
  decision TEXT NOT NULL,
  latency_ms INTEGER,
  prompt_tokens INTEGER,
  completion_tokens INTEGER,
  validation TEXT NOT NULL,
  error_class TEXT,
  response_preview TEXT
);

Verify: run this command:

sqlite3 ledger.db ".schema model_call_ledger"

You should see the full table definition. Stage 1 is done.

Stage 2: Wrap the model call

The wrapper is the core artifact. It records before and after the call. It records success and failure. It never returns without writing a row.

import hashlib
import sqlite3
import time

DB_PATH = "ledger.db"

def open_ledger():
    conn = sqlite3.connect(DB_PATH)
    conn.execute("PRAGMA journal_mode=WAL;")
    return conn

def record_call(conn, entry):
    conn.execute(
        """
        INSERT INTO model_call_ledger
        (request_hash, route, model, decision, latency_ms,
         prompt_tokens, completion_tokens, validation, error_class,
         response_preview)
        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
        """,
        (
            entry["request_hash"], entry["route"], entry["model"],
            entry["decision"], entry["latency_ms"], entry["prompt_tokens"],
            entry["completion_tokens"], entry["validation"],
            entry["error_class"], entry["response_preview"][:200],
        ),
    )
    conn.commit()

def estimate_tokens(text):
    return max(1, len(text) // 4)

Now the wrapper itself. The client.complete line is a placeholder. Swap in your endpoint's SDK or HTTP client.

def call_with_ledger(prompt, route="default", model="free-model"):
    conn = open_ledger()
    request_hash = hashlib.sha256(prompt.encode()).hexdigest()[:16]
    start = time.perf_counter()
    try:
        text = client.complete(prompt, model=model)  # your endpoint call
        latency_ms = int((time.perf_counter() - start) * 1000)
        validation = validate_output(text)
        record_call(conn, {
            "request_hash": request_hash,
            "route": route,
            "model": model,
            "decision": "accepted" if validation == "ok" else "flagged",
            "latency_ms": latency_ms,
            "prompt_tokens": estimate_tokens(prompt),
            "completion_tokens": estimate_tokens(text),
            "validation": validation,
            "error_class": None,
            "response_preview": text,
        })
        return text
    except Exception as exc:
        latency_ms = int((time.perf_counter() - start) * 1000)
        record_call(conn, {
            "request_hash": request_hash,
            "route": route,
            "model": model,
            "decision": "error",
            "latency_ms": latency_ms,
            "prompt_tokens": estimate_tokens(prompt),
            "completion_tokens": 0,
            "validation": "none",
            "error_class": type(exc).__name__,
            "response_preview": str(exc),
        })
        raise

Verify: run one call. Then query the ledger:

sqlite3 ledger.db "SELECT decision, latency_ms, validation FROM model_call_ledger ORDER BY id DESC LIMIT 3;"

You should see one row per call. The decision column tells the truth.

Stage 3: Record the validation outcome

A response can be fast and wrong. The ledger must capture that. Plug in any output check. A JSON schema check works. A regex works. A length check works.

def validate_output(text):
    if len(text) < 10:
        return "too_short"
    if not text.strip().endswith("."):
        return "no_terminal_punctuation"
    return "ok"

This is a toy validator. Replace it with your real checks. The key is the result lands in the validation column. A flagged call still gets a row. That is the point. You want to see the failures.

Verify: send a prompt that produces a short reply. Then run:

sqlite3 ledger.db "SELECT validation, decision FROM model_call_ledger ORDER BY id DESC LIMIT 1;"

Expect too_short and flagged.

Stage 4: Ask the ledger questions

This is where the pattern pays. Three queries cover most incidents.

Slowest calls:

SELECT request_hash, model, latency_ms, validation
FROM model_call_ledger
ORDER BY latency_ms DESC
LIMIT 10;

Validation failures by model:

SELECT model, validation, COUNT(*)
FROM model_call_ledger
WHERE validation != 'ok'
GROUP BY model, validation
ORDER BY COUNT(*) DESC;

Error rate per hour:

SELECT strftime('%Y-%m-%d %H:00', ts) AS hour,
       COUNT(*) AS total,
       SUM(decision = 'error') AS errors
FROM model_call_ledger
GROUP BY hour
ORDER BY hour DESC
LIMIT 24;

Verify: run each query. You should see real numbers. If a query returns nothing, your ledger is empty. Go back to Stage 2.

Stage 5: Deploy next to the traffic

The ledger belongs next to the traffic. Run it on MonkeyCode's free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. This tutorial uses MonkeyCode's free model access as the endpoint under test. The pattern works with any HTTP model endpoint. The ledger does not care which provider serves the text.

Deploy the service:

python -m venv .venv
. .venv/bin/activate
pip install fastapi uvicorn
uvicorn ledger_service:app --host 0.0.0.0 --port 8000

Add a health route that reports the last recorded call:

from fastapi import FastAPI
import sqlite3

app = FastAPI()

@app.get("/health")
def health():
    conn = sqlite3.connect("ledger.db")
    row = conn.execute(
        "SELECT ts, decision, validation FROM model_call_ledger ORDER BY id DESC LIMIT 1;"
    ).fetchone()
    return {"last_call": row}

Verify: from your machine, run:

curl http://YOUR_SERVER:8000/health

You should see JSON with the latest call. If you see null, no call has reached the server yet. Fix the routing before you fix the model.

Limitations

The ledger is not a retry layer. It does not cache. It does not benchmark. It records, and that is its job. Token counts are estimates, not provider invoices. The response preview is truncated. Store full payloads elsewhere if you need them.

SQLite handles one writer at a time. A single service is fine. A fleet of workers will contend. Move to Postgres when you outgrow SQLite.

Who should not use this

Do not use this for a single script you run once. Do not use it when you need centralized dashboards. Do not use it if you already have a full observability stack. The ledger fills the gap between "no visibility" and "enterprise monitoring."

That gap is exactly where free endpoints live. They are free, but they are not transparent. A ledger makes them auditable. Start with one table. Add it before the next silent failure. The next 2 AM incident will be a query, not a mystery.