Migration Diary: Building a Token Ledger Before You Trust a Free Model Tier

If your side project is quietly burning money on a paid AI endpoint, the fastest fix is not prompt tinkering; it is moving the entire inference loop onto a free model tier with a free server to match. But unmetered free credits are a silent failure waiting to happen. This diary records how I migrated a small pull-request summarizer to MonkeyCode's free model access and its free server option, and why a simple token ledger made the cutover boring.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The Cost That Grew While Nobody Looked

The bot summarized PR descriptions for my team's repository. It cost about $7 per month, which felt harmless until the team doubled and the bill doubled with it. I considered optimizing prompts, caching responses, and even sampling fewer diffs, but the math always pointed to the same conclusion: the fixed endpoint cost was the problem. Moving to MonkeyCode meant trading that monthly bill for a 10M token allowance and a server I did not have to pay for.

Before writing any code, I wrote down the constraints. The bot was stateless, idempotent, and tolerated a retry after a short failure. Those three properties made it a good candidate for a free tier because free infrastructure usually comes with longer cold starts and no guaranteed SLA. If any of those properties had been false, the migration would have been a poor idea.

Step 0: Measure Your Actual Burn Rate

The first mistake people make is migrating before they know their token consumption per day. I parsed my old API call logs and built a tiny script that tallied prompt and completion tokens by day, so the 10M budget had a real number to compare against.

# token_ledger.py - assumes one JSON line per call
def daily_tokens(log_path):
    import json, time
    from collections import defaultdict

    totals = defaultdict(int)
    with open(log_path) as f:
        for line in f:
            entry = json.loads(line)
            day = time.strftime("%Y-%m-%d", time.localtime(entry["timestamp"]))
            totals[day] += entry.get("prompt_tokens", 0) + entry.get("completion_tokens", 0)
    return totals

if __name__ == "__main__":
    usage = daily_tokens("calls.jsonl")
    avg = sum(usage.values()) / max(len(usage), 1)
    peak = max(usage.values() or [0])
    print(f"average daily: {avg:.0f}, peak daily: {peak}")

This script is an illustrative example, not a production tool, but it showed that my bot averaged 140K tokens per day. Ten million tokens meant roughly seventy days of headroom, which was far better than the monthly billing cycle I was escaping. The peak day was 260K, so I knew the budget could absorb a busy week without immediate danger.

Step 1: Pick a Stateless Worker for the Free Server

My old bot lived inside a long-running container with a database connection and in-memory caches. The free server option was not the right home for that container, so I split the bot into a scheduler and a worker. The worker received a PR URL, fetched the diff, called the model, and stored the summary in the repository's existing storage layer.

Every worker run had to be re-runnable without side effects beyond writing the summary. I made the worker fetch the PR diff freshly each time instead of trusting a cached version. This design decision removed the biggest risk of free infrastructure: losing a stateful process while it was mid-request.

# deploy.sh - minimal staging for the free server option
git clone https://github.com/yourname/pr-summarizer-bot
eed pr-summarizer-bot
python -m venv .venv
.venv/bin/pip install -r requirements.txt
systemctl --user restart pr-summarizer-worker

The exact deployment commands depend on your account's free server setup, but the principle is universal: the worker must be killable at any moment and still produce a correct result on the next attempt.

Step 2: Wire the Model Endpoint With Environment Variables

MonkeyCode exposes a model endpoint that speaks the same OpenAI-compatible protocol I was already using, so the code changes were limited to configuration. I stored the base URL and key in environment variables rather than hardcoding them into the source, which made it easy to switch back if the free tier disappointed me.

export MONKEY_BASE_URL="https://api.monkeycode.example/v1"  # replace with real endpoint from docs
export MONKEY_API_KEY="your-key-here"
export MONKEY_MODEL="free-model"                           # replace with actual model id

Those placeholder values matter because I am not going to guess at the current endpoint or model name. Read the project README for the live values, and pin the model version you test against. A free tier can rotate models without warning, so you want a way to freeze the exact identifier your prompts were tuned for.

Step 3: Build a Two-Way Switch Before You Need It

A free token allowance can be exhausted at three in the morning, and that is the worst time to discover your bot has no fallback. I wrote a small wrapper that tried the free endpoint and dropped back to a paid endpoint only when the free one raised a rate-limit or quota error. The fallback path also wrote a metric so I could see how often the safety net was being used.

import os
from openai import OpenAI

free_client = OpenAI(
    base_url=os.environ["MONKEY_BASE_URL"],
    api_key=os.environ["MONKEY_API_KEY"],
)

fallback_client = OpenAI()  # uses your old paid credentials

def summarize(diff_text):
    try:
        return free_client.chat.completions.create(
            model=os.environ["MONKEY_MODEL"],
            messages=[{"role": "user", "content": diff_text}],
            max_tokens=600,
        )
    except Exception:
        print("fallback triggered", flush=True)
        return fallback_client.chat.completions.create(
            model="gpt-4o-mini",  # your previous paid model
            messages=[{"role": "user", "content": diff_text}],
            max_tokens=600,
        )

This is a simplified example, but the pattern is solid: the free tier remains the primary path, the paid endpoint is the emergency brake, and the printed fallback message feeds into my monitoring.

Step 4: Watch the Ledger With a Cron Check

I added a five-line shell script that polls a local usage endpoint and sends a notification when the remaining tokens dip below a threshold. It is intentionally crude because I needed something that would run anywhere, even on the free server where dependencies were minimal.

#!/bin/bash
# quota_watch.sh
used=$(curl -s http://localhost:8080/usage | jq -r '.tokens.used')
limit=10000000
remaining=$((limit - used))
echo "tokens remaining: $remaining"
if (( remaining < 50000 )); then
    echo "WARNING: budget nearly exhausted" | mail -s "PR bot token budget" [e-mail]
fi

The localhost:8080/usage endpoint does not exist in every setup, so treat this script as a template rather than a ready-made command. The important habit is to check the ledger regularly instead of assuming the allowance will last forever.

The Leftovers That Almost Broke the Cutover

Three leftovers were not in the migration checklist. First, the old endpoint returned a different default timestamp format, so my daily aggregation script needed a timezone fix. Second, the free model occasionally returned a refusal that the paid model never produced, and my retry logic treated it as a success because the HTTP status was 200. Third, the worker's previous container had a stable hostname, while the free server gave me a random one on each redeploy, which broke a monitoring alert that relied on the hostname.

Each leftover took less than thirty minutes to fix, but they were invisible until the first run after the cutover. Write a test that exercises the full pipeline before you delete your old endpoint credentials.

Who Should Not Use This Approach

Free model access and a free server are not a universal replacement for a paid setup. If your workload requires sub-second latency, has strict compliance requirements, or processes user data that cannot leave your infrastructure, this migration is wrong for you. Similarly, if your token consumption is already above two million tokens per day, the 10M allowance will vanish before you finish debugging the first deploy.

A stateless, idempotent, and low-frequency workload is the ideal candidate. For anything else, keep the paid tier and treat the free one as a burst buffer, not a home.

The Boring Migration Is the Good Migration

A migration succeeds when nothing surprising happens after you flip the switch. The token ledger, the fallback wrapper, and the stateless worker design made that possible. The old API key is still in my password manager, but it is no longer the default route, and the monthly bill has companies of a number I no longer care about.

If you have a similar bot and want to test the free tier yourself, the MonkeyCode repository is a reasonable starting point. Read its documentation for the current allowance and server constraints, and build your own ledger before you make the leap.