Weekend Build Log: A Release-Note Classifier With Exactly One Model Call per Batch

It is Friday, 11 p.m. Your open-source repo has 40 untagged releases.

You want a board that says breaking, feature, or fix. You do not want auth, dashboards, or a vector store.

This is a build log for the smallest useful version. One SQLite table. One batched model call. One HTTP endpoint. No UI.

What the demo actually does

• Reads release notes from a local JSON file.
• Sends 20 entries per request, each with a stable id.
• Rejects any response whose ids do not match the input.
• Upserts labels into SQLite with a confidence score.
• Serves GET /labels/{tag} as plain JSON.

That is the entire scope. Ship it, use it, then stop.

Step 0: Cut scope before you write a line

Most weekend projects die in the planning doc. Write the cuts down first.

| Tempting | Shipped | Why |
| --- | --- |
| Web dashboard | JSON endpoint | You can read JSON in a terminal |
| Auth | None | Local and read-only |
| Postgres | SQLite file | One file, zero services |
| Streaming | Batched call | 40 rows is not a stream |
| Fine-tuning | Prompt + schema | No labeled data yet |
| Retry forever | Two retries | Failures should be visible |

The table is the artifact. Everything below implements this table.

Step 1: One table, one strict schema

Start with the store. Labels are append-only per release id.

CREATE TABLE IF NOT EXISTS labels (
  release_id TEXT PRIMARY KEY,
  tag        TEXT NOT NULL,
  kind       TEXT NOT NULL,
  confidence REAL NOT NULL,
  model      TEXT,
  created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

Now define the response contract. Make it strict enough to fail loudly.

SCHEMA = {
    "type": "object",
    "properties": {
        "items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "id": {"type": "string"},
                    "kind": {"enum": ["breaking", "feature", "fix", "chore"]},
                    "confidence": {"type": "number", "minimum": 0, "maximum": 1},
                },
                "required": ["id", "kind", "confidence"],
            },
        }
    },
    "required": ["items"],
}

Four labels are enough. Every extra label costs you agreement between runs.

Step 2: Batch the prompt with hard ids

Model output drifts when you ask for alignment by position. Give every row an id instead.

def build_prompt(batch):
    lines = [
        "Classify each release note. Return JSON only.",
        "Allowed kinds: breaking, feature, fix, chore.",
        "Copy the id exactly. Never invent ids.",
        "",
    ]
    for row in batch:
        lines.append(f"id={row['id']} :: {row['note'][:400]}")
    return "\n".join(lines)

Truncate the note body at 400 characters. Long changelogs add tokens, not accuracy.

Keep batches at 20. Smaller batches mean more calls; larger batches mean sloppier answers.

Step 3: Validate ids, not just JSON

Most tutorials stop at json.loads. That is where the bugs live.

def parse_items(raw, expected_ids):
    text = raw.strip()
    if text.startswith("```

"):
        text = text.split("

```")[1]
        text = text.split("\n", 1)[1] if "\n" in text else text
    data = json.loads(text)
    got = {item["id"] for item in data["items"]}
    if got != set(expected_ids):
        raise ValueError(f"id mismatch: {got ^ set(expected_ids)}")
    return data["items"]

Treat an id mismatch as a failed call. Retry once with the same prompt and a lower temperature.

If it fails twice, write the batch to dead_letter.json and move on. Silent partial results are worse than a gap.

Step 4: Test offline before you spend a token

Write a fake client. It returns canned JSON and costs nothing.

class FakeClient:
    def __init__(self, payload):
        self.payload = payload
        self.calls = 0

    def complete(self, prompt):
        self.calls += 1
        return self.payload


def test_batch_roundtrip(tmp_path):
    batch = [{"id": "r1", "note": "Drop Node 16 support"}]
    fake = FakeClient('{"items":[{"id":"r1","kind":"breaking","confidence":0.9}]}')
    items = parse_items(fake.complete(build_prompt(batch)), ["r1"])
    assert items[0]["kind"] == "breaking"
    assert fake.calls == 1

Add one more test with a wrong id. The test must raise, not patch over the gap.

Run it with pytest -q. Green offline means the schema, parsing, and retry path work.

Step 5: Put one endpoint on the free server

from fastapi import FastAPI
import sqlite3

app = FastAPI()

@app.get("/labels/{tag}")
def labels(tag: str):
    con = sqlite3.connect("labels.db")
    rows = con.execute(
        "SELECT release_id, kind, confidence FROM labels WHERE tag=? ORDER BY created_at DESC",
        (tag,),
    ).fetchall()
    return {"tag": tag, "items": [
        {"id": r[0], "kind": r[1], "confidence": r[2]} for r in rows
    ]}

Run uvicorn app:app --host 0.0.0.0 --port 8000. Hit it with curl localhost:8000/labels/v1.2.0.

A read-only endpoint has no auth surface. Keep it that way until someone else asks.

Where a free tier fits here

This project has two costs: model calls and hosting. Both are small, but both are recurring.

MonkeyCode offers free model access and a free server option. Those availability claims come from the operator, so verify the current allowance and terms on the product page before you plan around them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The fit is narrow and honest. You get one place for the batched call and the tiny endpoint while you decide whether the project deserves a paid home. If your workload grows past a weekend, re-evaluate.

The offline test above matters more than the provider. It keeps your retry logic honest no matter which client you swap in later.

What this build skipped on purpose

• No UI. Terminal JSON is enough for 40 rows.
• No embeddings. Keyword matching finds duplicates fine at this size.
• No scheduled refresh. Run the script by hand until it annoys you.
• No multi-label output. One kind per release keeps the table queryable.
• No prompt versioning. Add it when the second person edits the prompt.

Each skip is reversible. That is the point of a scope cut.

Limitations and who should skip this

• Classification quality follows the model. Spot-check 10 rows before trusting the board.
• Release notes written in mixed languages will drift. Add an explicit language instruction.
• Free-tier limits can change. A monthly quota is not a capacity plan.
• This is not for compliance reporting. Breaking-change detection needs human sign-off.

If your releases are already tagged by hand, skip this. If you need per-commit attribution, this is the wrong layer.

Also skip it if you cannot accept an occasional dead_letter.json row. Perfect recall is not on the menu.

The 90-minute checklist

1. Write the cuts table. Ten minutes.
2. Create the SQLite table and the schema constant.
3. Build the prompt with explicit ids and truncation.
4. Write parse_items with the id-set check.
5. Add the fake-client test and run pytest -q.
6. Wire the real client and label one batch of 20.
7. Serve one endpoint and curl it once.

Stop after step seven. The next feature is a different weekend.

If you want to try this shape, run the offline test first, then point the client at whichever free model access you already have and read its quota terms before you batch anything large.