DEV Community

Dakota Huang
Dakota Huang

Posted on

Measure Every Token: A SQLite-Backed Call Tracker for Free Model Endpoints

Free model endpoints are not free of mystery. You get a response, but no dashboard. No latency percentiles. No error rate. No usage history. This tutorial builds a lightweight call tracker. It logs every request to SQLite. It generates daily reports. It alerts on error spikes. You can run it on a free server with cron.

This tutorial uses MonkeyCode's free model access for the example calls and MonkeyCode's free server option for scheduling. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why Track Calls

A free tier hides your usage. You might hit a rate limit without warning. You might see latency creep up. You might miss a 5% error rate. Tracking gives you numbers. Numbers turn guesses into decisions.

A tracker is not a proxy. It wraps your existing calls. It records metadata. It does not change behavior. You can add it to any Python script.

Stage 1: Set Up the Project

Create a directory and a virtual environment.

mkdir llm-usage-ledger && cd llm-usage-ledger
python -m venv venv
source venv/bin/activate
pip install requests
Enter fullscreen mode Exit fullscreen mode

Create a file named tracker.py. This file will hold the logging logic.

# tracker.py
import sqlite3
import time
import os

DB_PATH = os.environ.get("LLM_LEDGER_DB", "calls.db")

def init_db():
    conn = sqlite3.connect(DB_PATH)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS calls (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            timestamp TEXT NOT NULL,
            model TEXT,
            prompt_chars INTEGER,
            response_chars INTEGER,
            latency_ms INTEGER,
            status_code INTEGER,
            error TEXT
        )
    """)
    conn.commit()
    conn.close()

def log_call(model, prompt, response, latency_ms, status_code, error=None):
    conn = sqlite3.connect(DB_PATH)
    conn.execute("""
        INSERT INTO calls (timestamp, model, prompt_chars, response_chars, latency_ms, status_code, error)
        VALUES (?, ?, ?, ?, ?, ?, ?)
    """, (time.isoformat(), model, len(prompt), len(response) if response else 0, latency_ms, status_code, error))
    conn.commit()
    conn.close()
Enter fullscreen mode Exit fullscreen mode

Verify Stage 1.

python -c "import tracker; tracker.init_db(); print('ok')"
ls -la calls.db
Enter fullscreen mode Exit fullscreen mode

You should see a new calls.db file.

Stage 2: Wrap Your LLM Call

Create call.py. It defines a function that calls the endpoint and logs the result.

# call.py
import requests
import time
import os
from tracker import init_db, log_call

init_db()

def call_llm(prompt, model="free-model"):
    endpoint = os.environ["LLM_ENDPOINT"]
    api_key = os.environ["LLM_API_KEY"]
    start = time.time()
    try:
        resp = requests.post(
            endpoint,
            headers={"Authorization": f"Bearer {api_key}"},
            json={"model": model, "messages": [{"role": "user", "content": prompt}]},
            timeout=60,
        )
        status = resp.status_code
        if status == 200:
            data = resp.json()
            response = data["choices"][0]["message"]["content"]
            error = None
        else:
            response = None
            error = resp.text[:200]
    except Exception as e:
        status = 0
        response = None
        error = str(e)
    latency_ms = int((time.time() - start) * 1000)
    log_call(model, prompt, response, latency_ms, status, error)
    return response, status
Enter fullscreen mode Exit fullscreen mode

Verify Stage 2.

export LLM_ENDPOINT="https://your-endpoint.example/v1/chat/completions"
export LLM_API_KEY="your-key"
python -c "from call import call_llm; print(call_llm('Say hello'))"
sqlite3 calls.db "SELECT * FROM calls;"
Enter fullscreen mode Exit fullscreen mode

You should see one row with the prompt length, latency, and status.

Stage 3: Generate a Daily Report

Create report.py. It queries the last 24 hours and prints summary statistics.

# report.py
import sqlite3
import os
from datetime import datetime, timedelta

DB_PATH = os.environ.get("LLM_LEDGER_DB", "calls.db")

def generate_report(days=1):
    conn = sqlite3.connect(DB_PATH)
    since = (datetime.now() - timedelta(days=days)).isoformat()
    cur = conn.execute("""
        SELECT COUNT(*), AVG(latency_ms),
               SUM(CASE WHEN status_code != 200 THEN 1 ELSE 0 END)
        FROM calls WHERE timestamp >= ?
    """, (since,))
    total, avg_latency, errors = cur.fetchone()
    total = total or 0
    errors = errors or 0
    print(f"Calls in last {days} day(s): {total}")
    print(f"Average latency: {avg_latency:.0f} ms" if avg_latency else "Average latency: n/a")
    if total:
        print(f"Errors: {errors} ({errors/total*100:.1f}%)")
    else:
        print("Errors: 0 (0.0%)")
    conn.close()

if __name__ == "__main__":
    generate_report()
Enter fullscreen mode Exit fullscreen mode

Verify Stage 3.

python report.py
Enter fullscreen mode Exit fullscreen mode

You should see the call you made in Stage 2.

Stage 4: Alert on Error Spikes

Create alert.py. It checks the last hour and sends a webhook if the error rate exceeds 20%.

# alert.py
import sqlite3
import os
import json
import urllib.request
from datetime import datetime, timedelta

DB_PATH = os.environ.get("LLM_LEDGER_DB", "calls.db")
WEBHOOK_URL = os.environ.get("ALERT_WEBHOOK")

def check_alerts():
    conn = sqlite3.connect(DB_PATH)
    since = (datetime.now() - timedelta(hours=1)).isoformat()
    cur = conn.execute("""
        SELECT COUNT(*), SUM(CASE WHEN status_code != 200 THEN 1 ELSE 0 END)
        FROM calls WHERE timestamp >= ?
    """, (since,))
    total, errors = cur.fetchone()
    total = total or 0
    errors = errors or 0
    if total and errors / total > 0.2:
        message = f"LLM error rate {errors/total*100:.0f}% in the last hour ({errors}/{total})"
        print(message)
        if WEBHOOK_URL:
            payload = json.dumps({"text": message}).encode()
            req = urllib.request.Request(WEBHOOK_URL, data=payload, headers={"Content-Type": "application/json"})
            urllib.request.urlopen(req)
    conn.close()

if __name__ == "__main__":
    check_alerts()
Enter fullscreen mode Exit fullscreen mode

Verify Stage 4.

export ALERT_WEBHOOK="https://hooks.example.com/your-webhook"  # optional
python alert.py
Enter fullscreen mode Exit fullscreen mode

No output means the error rate is below the threshold.

Stage 5: Schedule with Cron

A tracker only helps if it runs continuously. Use a free server with cron. Add these lines to your crontab.

crontab -e
Enter fullscreen mode Exit fullscreen mode
0 * * * * cd /path/to/llm-usage-ledger && python alert.py >> alerts.log 2>&1
0 9 * * * cd /path/to/llm-usage-ledger && python report.py >> reports.log 2>&1
Enter fullscreen mode Exit fullscreen mode

The first line checks alerts every hour. The second generates a daily report at 9 AM.

Verify Stage 5.

crontab -l
cd /path/to/llm-usage-ledger && python alert.py && python report.py
Enter fullscreen mode Exit fullscreen mode

You should see the report output.

Limitations

This tracker measures what you send through call_llm. It does not capture calls made by other services or libraries. It stores prompt and response lengths, not content. That keeps the database small but limits debugging.

The alert threshold is a fixed 20%. A real system needs adaptive thresholds based on baseline error rates. The webhook is a simple POST. You may need retries and authentication for production.

Who should not use this? Teams that need centralized logging across multiple servers. This is a single-node tracker. Teams that need token-accurate billing should use a library like tiktoken instead of character counts.

The Bottom Line

You cannot optimize what you do not measure. A free model endpoint gives you responses, not visibility. This tracker gives you a ledger of every call. It runs on a free server. It alerts when things break.

Start with one script. Wrap your calls. Let the data accumulate. Then decide if your free tier is actually free enough.

Top comments (0)