A Small LLM Service That Costs Nothing to Run: A Case Study

Most LLM architecture advice assumes you already have infrastructure. Agents, reasoning ledgers, evals — all of it presumes a server that is already paid for. This case study goes the other way: a small, boring, always-on job that costs nothing to run.

The project is an incident-summary bot. It watches a health endpoint, and when the endpoint changes state, it asks a free-tier model to write a plain-language note. Here is the background, the implementation, the measurements, and the limits.

Background: the problem

You run a hobby API. It goes down at 3 a.m. The alert fires, you restart the process, and now you owe the world a sentence: what broke, for how long, and what you did. Writing that sentence is the most skippable chore in operations, so it gets skipped. The goal here was to automate only that chore. Detection stays dumb — a plain HTTP check. The model writes the note.

MonkeyCode is the infrastructure in this case study: an open-source project that bundles model access and a server option. The free tier includes a 10-million-token allowance and a free server, as of this writing. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Quotas and terms change, so verify the current numbers before you rely on them.

Why this workload fits a free tier

Characteristic	This project	Why the free tier works
Call frequency	Only on state changes	The token allowance lasts
Output size	2–3 sentences	Tiny per-call cost
Latency tolerance	Minutes	A slow response changes nothing
Data sent to the model	Public endpoint names	No secrets to leak

Flip any row and the free tier stops being the right answer. High frequency, long outputs, sub-second latency, or private data: pay for it, or pick a different tool.

Implementation

Step 1: the state machine

The bot needs to know when the endpoint changed state. A cron job runs every five minutes; each run fetches the endpoint and compares the result with the last known state, stored in a small JSON file.

# check.py — run every 5 minutes via cron
import json
import urllib.request
from datetime import datetime
from pathlib import Path

URL = "https://your-hobby-service.example/health"
STATE_FILE = Path("state.json")
EVENTS_FILE = Path("events.jsonl")

def current_state() -> str:
    try:
        with urllib.request.urlopen(URL, timeout=10) as resp:
            return "up" if resp.status == 200 else "down"
    except Exception:
        return "down"

state = json.loads(STATE_FILE.read_text()) if STATE_FILE.exists() else {"state": "up"}
new_state = current_state()

if new_state != state["state"]:
    with EVENTS_FILE.open("a") as f:
        f.write(json.dumps({
            "from": state["state"],
            "to": new_state,
            "at": datetime.now().isoformat(),
        }) + "\n")
    STATE_FILE.write_text(json.dumps({"state": new_state}))

Step 2: the summary generator

When an event is appended, the next step turns it into a note. The prompt asks for a fixed shape, so the output stays parseable.

# summarize.py — read the latest event, ask the model, append an incident note
import json
import os
import urllib.request
from datetime import datetime

event = json.loads(open("events.jsonl").readlines()[-1])

if event["at"] in open("incidents.md").read():
    raise SystemExit("already summarized")

prompt = f"""The service changed state from {event['from']} to {event['to']} at {event['at']}.
Write a 2-3 sentence incident note for a status page. Use this exact shape:
- Status:
- Window:
- Likely cause (clearly labeled as a guess):
- Action taken:
"""

body = json.dumps({
    "prompt": prompt,
    "max_tokens": 200,
}).encode()

req = urllib.request.Request(
    os.environ["MONKEYCODE_API_URL"],
    data=body,
    headers={
        "Authorization": f"Bearer {os.environ['MONKEYCODE_API_KEY']}",
        "Content-Type": "application/json",
    },
)

with urllib.request.urlopen(req, timeout=60) as resp:
    note = json.loads(resp.read())["text"]

with open("incidents.md", "a") as f:
    f.write(f"## {event['at']}\n{note}\n\n")

with open("usage.jsonl", "a") as f:
    f.write(json.dumps({
        "at": datetime.now().isoformat(),
        "approx_tokens": len(note.split()),
    }) + "\n")

The request shape above is the common completion pattern. Field names differ between providers, so check the current docs before copying. If the response includes a usage object, record that instead of the word count.

Step 3: scheduling

Two cron lines on the free server:

*/5 * * * * cd /path/to/bot && python3 check.py
*/5 * * * * cd /path/to/bot && python3 summarize.py

The point is that the server stays on while you sleep. That is the part a laptop cron cannot give you.

Step 4: deployment

Deploy by cloning the repo onto the free server, setting the two environment variables, and adding the cron lines. No Docker, no database. If the server disappears, the state file is the only thing you lose.

Results: what to measure

Here is a measurement plan, not a fixed number, because your endpoint and your prompts will differ. Record three things: state changes, approximate tokens spent, and whether the note was usable.

# measure.py — summarize what the bot actually used
import json

events = sum(1 for _ in open("events.jsonl"))
tokens = sum(json.loads(line)["approx_tokens"] for line in open("usage.jsonl"))

print(f"state changes: {events}")
print(f"approx tokens spent on notes: {tokens}")

What good looks like: a 200-token note you would not rewrite. What bad looks like: a note that hallucinates a cause. The fix is in the prompt — "likely cause" must be labeled as a guess, or removed.

Lessons learned

1. Free tiers reward narrow workloads. The bot works because it does one thing, rarely. A generic assistant endpoint would burn the allowance in days.
2. The server is the real constraint. Tokens are easy to give away; an always-on machine is what changes what you can build. Treat the server as small and disposable.
3. Prompt shape beats prompt length. The fixed output shape made notes usable without a parser.
4. Idempotency is not optional. Cron overlaps happen. The timestamp guard prevents duplicate notes.
5. Free means changeable. Quotas, endpoints, and model availability move. Pin what you can and re-check the docs.

Limitations and who should not use this

This setup is wrong for high-throughput summarization — a token allowance will not survive it. It is wrong for private data: do not send customer logs into a free prompt. It is wrong for latency-critical paths, and for workloads that need a model outside the free access. It is also wrong if you cannot tolerate hobby-grade maintenance. The bot is a tool, not a platform.

Running it yourself

If you want to run this yourself, MonkeyCode is open source, and its free tier includes model access and a server — the docs list the current limits. Start with the dumbest version: one endpoint, one state file, one cron line. Let the model write the note. Keep the judgment for yourself.

MonkeyCode provides free models that can run this workflow.