Ship an AI Feature Today With a Four-Rung Degradation Ladder

The cheapest way for a solo founder to ship an AI feature is to assume the model will be slow, throttled, or unaffordable before the week ends — and to encode that assumption in code rather than in a spreadsheet. The artifact below is a four-rung degradation ladder backed by a daily token ledger that refuses calls instead of warning about them. It is under a hundred lines of stdlib Python, its tests run offline with no API key, and it deploys to a free server with two environment variables.

This is not a forecasting exercise. A forecast answers "how many tokens will I use"; a ladder answers "what does the user see when the answer is zero". For an indie product, the second question is the one that decides whether the site stays up overnight while nobody is watching the dashboard.

Why a budget alert loses to a ledger

Alerts arrive after the call. A dashboard email at 3 a.m. does not stop the request that is already in flight, and by the time a human reads it the bill exists. Provider-side usage pages also lag, tokenization differs between models, and retries silently double the count.

A local ledger inverts the order: the call is refused before it is made. The number may be approximate, but the ceiling is structural. That trade — precision for an enforced boundary — is the right one when the budget is zero.

Step 1 — Fix the rung contract first

Each rung is a named behaviour with a known cost and a known quality. Write the table before writing any prompt; it is the part reviewers actually argue about.

Rung	Kind	Token cost	When it answers	User-visible quality
cache	free	0	identical prompt seen before	exact repeat
primary	model	~900	budget allows the best answer	full quality
cheap	model	~250	primary is refused or errors	shorter, acceptable
static	free	0	nothing else is affordable	templated, honest

The costs are config values, not measurements — they are the numbers the operator is willing to bet per request. Recalibrate them from one week of real usage rather than guessing twice.

Step 2 — Make the ledger refuse, not warn

# ledger.py — stdlib only, Python 3.11+
import json
from datetime import datetime, timezone
from pathlib import Path


class Ledger:
    def __init__(self, path: Path, daily_cap: int) -> None:
        self.path = path
        self.daily_cap = daily_cap

    @staticmethod
    def _today() -> str:
        return datetime.now(timezone.utc).strftime("%Y-%m-%d")

    def _state(self) -> dict:
        if not self.path.exists():
            return {"day": self._today(), "spent": 0}
        state = json.loads(self.path.read_text())
        if state.get("day") != self._today():
            return {"day": self._today(), "spent": 0}
        return state

    def reserve(self, cost: int) -> bool:
        state = self._state()
        if state["spent"] + cost > self.daily_cap:
            return False
        state["spent"] += cost
        tmp = self.path.with_suffix(".tmp")
        tmp.write_text(json.dumps(state))
        tmp.replace(self.path)
        return True

Two details matter more than they look. The day key is UTC, so the reset is not tied to whatever timezone the free server happens to boot in. The write goes through a temp file and an atomic replace, because a container that restarts mid-write should not come back with a truncated ledger and a reset budget.

Step 3 — Inject the transport so the ladder is testable

# ladder.py — the ladder itself; the provider client is a parameter
import hashlib
import json
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Rung:
    name: str
    cost: int
    kind: str  # "cache" | "model" | "static"


DEFAULT_RUNGS = (
    Rung("cache", 0, "cache"),
    Rung("primary", 900, "model"),
    Rung("cheap", 250, "model"),
    Rung("static", 0, "static"),
)


class JsonCache:
    def __init__(self, path: Path, limit: int = 512) -> None:
        self.path = path
        self.limit = limit
        self.data = json.loads(path.read_text()) if path.exists() else {}

    def get(self, key: str) -> str | None:
        return self.data.get(key)

    def put(self, key: str, value: str) -> None:
        self.data[key] = value
        for stale in list(self.data)[: max(0, len(self.data) - self.limit)]:
            del self.data[stale]
        self.path.write_text(json.dumps(self.data))


class Ladder:
    def __init__(self, ledger, transport, cache, rungs=DEFAULT_RUNGS,
                 fallback="The assistant is offline right now."):
        self.ledger = ledger
        self.transport = transport          # (rung_name, prompt) -> str
        self.cache = cache
        self.rungs = rungs
        self.fallback = fallback

    def respond(self, prompt: str) -> dict:
        key = hashlib.sha256(prompt.encode()).hexdigest()[:16]
        for rung in self.rungs:
            if rung.kind == "cache":
                hit = self.cache.get(key)
                if hit:
                    return {"rung": rung.name, "text": hit, "spent": 0}
                continue
            if rung.kind == "static":
                return {"rung": rung.name, "text": self.fallback, "spent": 0}
            if not self.ledger.reserve(rung.cost):
                continue
            try:
                text = self.transport(rung.name, prompt)
            except Exception:
                continue  # cost stays reserved: failed calls can still bill
            self.cache.put(key, text)
            return {"rung": rung.name, "text": text, "spent": rung.cost}
        return {"rung": "static", "text": self.fallback, "spent": 0}

Note the deliberate pessimism: a reservation is not refunded when the transport throws. Providers can bill partial output on a dropped stream, and a ledger that assumes refunds is a ledger that under-counts exactly when things are going wrong.

Step 4 — Test the sad paths before the happy one

The happy path is the least interesting thing this code does. The tests that matter prove the ceiling holds when the budget is gone and when the provider is down.

# test_ladder.py
import tempfile, unittest
from pathlib import Path
from ladder import Ladder, Ledger, Rung, JsonCache
from ledger import Ledger as _Ledger  # same class, imported for clarity

RUNGS = (Rung("cache", 0, "cache"), Rung("primary", 900, "model"),
         Rung("cheap", 250, "model"), Rung("static", 0, "static"))


class LadderCase(unittest.TestCase):
    def setUp(self):
        self.dir = Path(tempfile.mkdtemp())
        self.cache = JsonCache(self.dir / "cache.json")

    def make(self, cap, transport):
        return Ladder(Ledger(self.dir / "ledger.json", cap), transport,
                      self.cache, RUNGS, "offline")

    def test_cap_forces_static(self):
        ladder = self.make(1000, lambda rung, prompt: "ok")
        self.assertEqual(ladder.respond("first")["rung"], "primary")
        self.assertEqual(ladder.respond("second")["rung"], "static")

    def test_cache_hit_is_free(self):
        ladder = self.make(1000, lambda rung, prompt: "ok")
        ladder.respond("same")
        second = ladder.respond("same")
        self.assertEqual(second["rung"], "cache")
        self.assertEqual(second["spent"], 0)

    def test_primary_failure_falls_to_cheap(self):
        def flaky(rung, prompt):
            if rung == "primary":
                raise RuntimeError("503")
            return "cheap answer"
        result = self.make(5000, flaky).respond("hello")
        self.assertEqual(result["rung"], "cheap")


if __name__ == "__main__":
    unittest.main()

Run it with python -m unittest -q test_ladder.py. Three tests, no network, no key. In the second test the cap is 1000 while primary costs 900, which is why the second distinct prompt can only reach static — the assertion is about the ceiling, not about the model.

Step 5 — Rehearse on a list, then deploy

Before pointing real traffic at this, replay a short list of the prompts your product actually receives and print which rung would answer each one under a shrunken cap.

# rehearse.py — dry run, no provider call
from pathlib import Path
from ladder import Ladder, Rung, JsonCache
from ledger import Ledger

PROMPTS = ["summarise this changelog", "summarise this changelog",
           "explain this stack trace", "draft a release note"]

ladder = Ladder(Ledger(Path("rehearsal.json"), 1200),
                lambda rung, prompt: f"SIMULATED via {rung}",
                JsonCache(Path("rehearsal-cache.json")))
for prompt in PROMPTS:
    result = ladder.respond(prompt)
    print(f"{result['rung']:>8} | {result['spent']:>4} | {prompt}")

A rehearsal that shows every prompt landing on primary means the cap is decorative. The useful outcome looks like primary for the first prompt, cache for the repeat, and static or cheap once the rehearsal budget drains — that is the shape a user will actually experience on a bad day.

Deployment is two variables and one process: DAILY_TOKEN_CAP and LEDGER_PATH on a mounted volume, with the ledger file outside the container image. On a small free host the cache and ledger are plain files, so nothing needs a database. A single curl against your own page after deploy, with the cap set to 0, is the fastest proof that the static rung renders instead of a 500.

Free model access and a free server option are what make this workflow realistic for one person, because the cliff arrives at zero spend rather than at the first invoice. MonkeyCode is the tool this build used for the model calls and the deployment; at the time of writing the operator advertises a 10 million token free allowance plus a free server option, and both belong in your own notes as a figure to re-check on the pricing page before you rely on them — allowances change, and a hard-coded quota is worse than no quota. The ladder does not depend on that number: set DAILY_TOKEN_CAP to whatever the current terms support.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What this ladder cannot do

• Estimate honestly. 900 and 250 are bets. Real tokenization varies by prompt length, language, and output style.
• Stop provider-side spend. If the same key is used by another script, the ledger knows nothing about it. One key per ledger is the only safe mapping.
• Expire cache entries. There is no TTL in JsonCache; a stale answer can outlive the fact it described. Add timestamps if the content is time-sensitive.
• Guarantee consistency under concurrency. Two processes can both pass reserve before either writes. A file lock or a single-writer queue fixes it; a solo deploy with one worker usually does not need it.

Who should not use this

Teams that need per-customer quotas, invoices that reconcile to the token, or auditable cost attribution should use provider-side budgets and a real billing pipeline instead. Products where a degraded answer is worse than no answer — anything medical, legal, or security-decisioning — should skip the cheap and static rungs entirely and return an explicit failure. And anyone whose workload is unbounded by nature, such as open-ended agent loops, needs a hard iteration cap before a token cap means anything.

For the solo case the trade is simple: accept approximate counting, accept a templated fallback, and ship today with a ceiling you can explain to yourself in one sentence. If you want to try the ladder on a zero bill, the free model access on MonkeyCode is the path this build took — the code above is the same either way.