DEV Community

Dakota Huang
Dakota Huang

Posted on

What Did the Model Decide? A Decision Log Tutorial for a Free Server

#python #ai #llm #tutorial

A raw response log tells you what a model said. A decision log tells you what your system did about it. Free servers restart without warning. Metered calls cost tokens on every retry. You need both facts on disk. This tutorial builds a working decision log from zero. Every stage has a verification step. Total time: about 40 minutes.

The stack is small: Python, FastAPI, and MonkeyCode's free model access plus free server tier.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why a decision log, not a response log

Agent memory is a trending topic. Most guides cover what to store. Few cover how to verify it survives a restart. Free servers kill processes. Your log must live on disk, not in memory.

Raw responses are also expensive to replay. Every retry burns tokens. A decision record stores the prompt hash, the response, and the action you took. One record. One source of truth.

Response log vs cache vs decision log

Artifact Answers Survives restart
Response log What did the model say? Depends
Cache Can we skip this call? Depends
Decision log What did we do about it? Yes, if on disk

A cache prevents duplicate work. A response log supports auditing. A decision log does both, plus it records the action. For metered model calls, the action is the expensive part. Store it.

What we build

A small Python service with three endpoints:

  • POST /decide — accepts a prompt, calls a model, writes a decision record.
  • GET /decisions — returns recent records.
  • GET /health — liveness check.

Records append to decisions.jsonl. The service runs on MonkeyCode's free server. Model calls use MonkeyCode's free model access. The current free tier includes 10 million tokens for new users.

Stage 1 — Get credentials

  1. Create a MonkeyCode account.
  2. Open the dashboard. Copy the API key and the model endpoint URL.
  3. Export them as environment variables. 
export MONKEY_API_KEY="your-key"
export MONKEY_MODEL_URL="https://your-endpoint-from-dashboard"
export MONKEY_MODEL="model-id-from-dashboard"
Enter fullscreen mode Exit fullscreen mode

Verification:

test -n "$MONKEY_API_KEY" && echo "key set"
Enter fullscreen mode Exit fullscreen mode

Stage 2 — Scaffold the project 

mkdir decision-log && cd decision-log
python -m venv .venv && source .venv/bin/activate
pip install fastapi uvicorn httpx
Enter fullscreen mode Exit fullscreen mode

Create main.py:

from fastapi import FastAPI

app = FastAPI()

@app.get("/health")
def health():
    return {"status": "ok"}
Enter fullscreen mode Exit fullscreen mode

Verification:

uvicorn main:app --port 8000
Enter fullscreen mode Exit fullscreen mode

In another terminal:

curl -s http://localhost:8000/health
# {"status":"ok"}
Enter fullscreen mode Exit fullscreen mode

Stage 3 — Define the decision record

A decision record has seven fields:

  • id — UUID.
  • ts — ISO timestamp.
  • prompt_hash — SHA-256 of the normalized prompt.
  • prompt — the original prompt.
  • response — the model's text.
  • action — what your code did with the response.
  • tokens — usage from the API response, if present. 
import hashlib
import uuid
from datetime import datetime, timezone

def make_record(prompt: str, response: str, action: str, tokens: int | None):
    return {
        "id": str(uuid.uuid4()),
        "ts": datetime.now(timezone.utc).isoformat(),
        "prompt_hash": hashlib.sha256(prompt.encode()).hexdigest(),
        "prompt": prompt,
        "response": response,
        "action": action,
        "tokens": tokens,
    }
Enter fullscreen mode Exit fullscreen mode

Verification:

python - <<'PY'
from main import make_record
r = make_record("hi", "hello", "echo", 3)
assert len(r["prompt_hash"]) == 64
print("record ok")
PY
Enter fullscreen mode Exit fullscreen mode

Stage 4 — Write the decide endpoint 

import json
import os
import httpx
from fastapi import FastAPI, Request

LOG_PATH = "decisions.jsonl"

def append_record(record: dict):
    with open(LOG_PATH, "a") as f:
        f.write(json.dumps(record) + "\n")

@app.post("/decide")
async def decide(req: Request):
    body = await req.json()
    prompt = body["prompt"]
    action = body.get("action", "none")
    async with httpx.AsyncClient(timeout=60) as client:
        r = await client.post(
            os.environ["MONKEY_MODEL_URL"],
            headers={"Authorization": f"Bearer {os.environ['MONKEY_API_KEY']}"},
            json={
                "model": os.environ["MONKEY_MODEL"],
                "messages": [{"role": "user", "content": prompt}],
            },
        )
        r.raise_for_status()
        data = r.json()
    response = data["choices"][0]["message"]["content"]
    tokens = data.get("usage", {}).get("total_tokens")
    record = make_record(prompt, response, action, tokens)
    append_record(record)
    return record
Enter fullscreen mode Exit fullscreen mode

Verification:

curl -s -X POST http://localhost:8000/decide \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"Reply with the word ok","action":"log"}'
cat decisions.jsonl
Enter fullscreen mode Exit fullscreen mode

Stage 5 — Survive a restart

Free servers kill processes. The file is the state. Load it on startup. Expose the count.

import json
from pathlib import Path

@app.get("/decisions")
def decisions():
    records = []
    if Path(LOG_PATH).exists():
        for line in Path(LOG_PATH).open():
            records.append(json.loads(line))
    return {"count": len(records), "records": records[-10:]}
Enter fullscreen mode Exit fullscreen mode

Verification — the restart test:

# terminal 1: start the server
uvicorn main:app --port 8000

# terminal 2: write one record
curl -s -X POST http://localhost:8000/decide \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"say hi","action":"echo"}'

# terminal 1: press Ctrl-C, then start again
uvicorn main:app --port 8000

# terminal 2: the record must still be there
curl -s http://localhost:8000/decisions
Enter fullscreen mode Exit fullscreen mode

If the count includes the pre-restart record, the log works.

Stage 6 — Deploy to the free server

  1. Push the project to a Git repository.
  2. Connect the repository to MonkeyCode's free server.
  3. Set the three environment variables in the dashboard.
  4. Deploy. The server exposes the app at a public URL.

Verification:

curl -s https://your-app-url/health
curl -s https://your-app-url/decisions
Enter fullscreen mode Exit fullscreen mode

A file is fine for one instance. It is not a database. Redeploys may wipe the disk. For durable storage, add a free-tier database and point append_record at it.

Who should not use this

  • Multi-user services. There is no auth here. Add a token check before exposing /decide.
  • High throughput. JSONL append works for hobby load. It will not scale.
  • Long-term storage. Free servers can recycle disks. Export records periodically.
  • Production agent memory. This is a scaffold, not a memory system.

Final check

Run the full loop once more. Start. Decide. Restart. Verify. If the record survives, the log works.

The model call is metered. The decision log makes every retry count. MonkeyCode's free tier is enough to test this today.

Top comments (0)