Ship the Feature That Survives a 429

A solo founder demoed her triage feature on a Saturday. Twenty seconds in, the model endpoint returned 429. The screen showed a spinner, then a red box. The buyer closed the tab.

The fix took an hour. The lesson took longer. Free capacity fails in public, not in staging.

Free model access and a free server are real options on a solo budget. They are also borrowed capacity. The bill stays at zero because someone else sets the ceiling. MonkeyCode offers free model access and a free server option, which is the setup this workflow targets. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The point is not to avoid those limits. The point is to ship a product that degrades in a readable way when they bite. That is a design choice you can make in one evening.

Three tiers, one return shape

Every AI call in an indie app should return the same object. Mine carries two fields: text and tier. The tier records what actually happened.

That single field turns a demo into something you can operate. You can count it, chart it, and alert on it. Without it, every failure looks like a blank screen.

Tier one is the cache. Same ticket, same body, no network. Tier two is the model. Tier three is plain code, and it always answers.

The ordering matters more than the technology. Cheap and certain runs first. Slow and borrowed runs last.

Tier	Path	Cost	Failure mode
cache	local store	none	stale text
model	free endpoint	free quota	429, timeout, empty reply
rules	pure function	none	shallow but honest

The router

The router below is a single function. It never throws at the caller. Point model.summarize at whichever free endpoint you use; the pipeline only needs a promise and a timeout. The example is untested in your environment, so run it before you trust it.

// tiers.js — Node 18+
import { createHash } from "node:crypto";

const keyOf = (t) =>
  createHash("sha1").update(t.ticketId + t.body).digest("hex");

const MIN_CHARS = 40;

function isUsable(text, ticket) {
  if (typeof text !== "string") return false;
  const t = text.trim();
  if (t.length < MIN_CHARS) return false;
  if (t.includes(ticket.body.slice(0, 30))) return false; // echo guard
  return true;
}

export function createPipeline({ store, model, rules, log = () => {} }) {
  return async function answer(ticket) {
    const key = keyOf(ticket);
    const hit = await store.get(key);
    if (hit) return { text: hit, tier: "cache" };

    try {
      const text = await model.summarize(ticket.body, { timeoutMs: 4000 });
      if (isUsable(text, ticket)) {
        await store.set(key, text);
        return { text, tier: "model" };
      }
      log("unusable_output");
    } catch (err) {
      log(`model_failed:${err.status ?? err.code ?? "unknown"}`);
    }

    return { text: rules(ticket.body), tier: "rules" };
  };
}

The echo guard earns its line. A 200 with the prompt repeated back is worse than a clean 429. The timeout earns its line too; a hung socket will hold your request open far longer than a rate limit ever will.

The fallback is a product decision

Most teams write the fallback as a stub that returns an apology. That is why the feature looks broken when quota runs out. Write the fallback as the version of the feature you can defend on your worst day.

// rules.js
const SIGNALS = [
  [/refund|chargeback|invoice/i, "billing"],
  [/log ?in|password|2fa/i, "auth"],
  [/crash|traceback|500/i, "bug"],
];

export function rules(body) {
  const tag = SIGNALS.find(([re]) => re.test(body))?.[1] ?? "general";
  const line = body.split("\n").map((s) => s.trim()).find(Boolean) ?? "No body";
  return `[${tag}] ${line.slice(0, 140)}`;
}

This returns a labeled line, not a summary. A support agent reads [billing] and routes it in two seconds. The label is also an honest signal that the model did not answer.

Write this file before you write the prompt. If you cannot describe the feature without the model, it is not ready for a free tier. You have built a dependency, not a product.

Test the bad day, not the good one

Happy-path tests pass all week and teach you nothing. The four cases that kill a demo are rate limit, timeout, empty reply, and the same ticket arriving twice.

// tiers.test.js
import { describe, it, expect, vi } from "vitest";
import { createPipeline } from "./tiers.js";
import { rules } from "./rules.js";

function memoryStore() {
  const m = new Map();
  return { get: async (k) => m.get(k), set: async (k, v) => void m.set(k, v) };
}

const ticket = {
  ticketId: "t-1",
  body: "Please refund invoice 88, the card was charged twice.",
};

const good = "Billing issue: duplicate charge on invoice 88. Refund queued for review.";

describe("answer()", () => {
  it("falls back to rules on 429", async () => {
    const model = {
      summarize: vi.fn().mockRejectedValue(
        Object.assign(new Error("slow down"), { status: 429 }),
      ),
    };
    const answer = createPipeline({ store: memoryStore(), model, rules });
    const out = await answer(ticket);
    expect(out.tier).toBe("rules");
    expect(out.text).toContain("billing");
  });

  it("falls back on a hang", async () => {
    const model = { summarize: vi.fn().mockRejectedValue({ code: "ETIMEDOUT" }) };
    const answer = createPipeline({ store: memoryStore(), model, rules });
    await expect(answer(ticket)).resolves.toMatchObject({ tier: "rules" });
  });

  it("rejects an empty reply", async () => {
    const model = { summarize: vi.fn().mockResolvedValue("   ") };
    const answer = createPipeline({ store: memoryStore(), model, rules });
    await expect(answer(ticket)).resolves.toMatchObject({ tier: "rules" });
  });

  it("calls the model once per identical ticket", async () => {
    const model = { summarize: vi.fn().mockResolvedValue(good) };
    const answer = createPipeline({ store: memoryStore(), model, rules });
    await answer(ticket);
    const second = await answer(ticket);
    expect(second.tier).toBe("cache");
    expect(model.summarize).toHaveBeenCalledTimes(1);
  });
});

Run it with two commands. No API key, no network, no waiting on a quota window.

npm i -D vitest
npx vitest run tiers.test.js

That last test is the money test. It tells you how much model traffic your cache removes. On repetitive support queues, the number is often larger than the founders expect.

What the free box still needs

Free servers restart. Processes vanish without a warning. Keep the cache and any job state somewhere that survives a restart, such as a file or a small managed database. A module-level Map is not storage; it is a memory leak with good manners.

Point uptime checks at your own edge, not at the free box. A monitor that pings the box becomes traffic, and traffic becomes a quota problem you created for yourself.

Log the tier on every request from day one. After a week, count the mix. That ratio is your real capacity plan, and it beats any estimate you would have written on a napkin.

Limits and who should skip this

The ladder hides failures. That is the point, and it is also the danger. Never use it where a wrong but confident answer costs money or health. Billing disputes, medical intake, legal advice, and security decisions need an error, not a guess.

Skip this if you need a latency guarantee, a signed data agreement, or volume well past a free tier. Skip it if your fallback output would mislead a user who cannot see which tier answered. Free capacity is a starting line, not a contract, and the ceiling can move.

If you are shipping alone this weekend, build tier three first. Then let a free model handle the happy path and watch what the ladder catches.