We built one AI tutor for five completely different subjects — here's the routing layer that keeps it from falling over

Most "AI tutor" apps are a chat window bolted onto a system prompt. We wanted something that actually holds state — knows what you've mastered, adapts difficulty, remembers you tomorrow — across five very different domains: a general subject tutor, exam prep for India's JEE/NEET/UPSC (each with its own official syllabus and negative-marking rules), a language-learning track that goes CEFR A0 through C1, an AI career coach with mock interviews, and a competitive "arena" mode.

That range turned out to be the actual engineering problem. Not "can an LLM explain photosynthesis" — every model can do that — but: what happens when you have 60+ distinct AI-backed features, each with different cost/quality tradeoffs, and one of them starts failing at 2am?

The thing that actually mattered: a feature registry, not a prompt library

Early on, model calls were scattered — each feature picked its own model, its own retry logic (often none), its own idea of what "thinking budget" meant. A cost audit found thinking tokens (billed at the output rate) were the single largest leak in the whole system. That's what forced the redesign.

Now every AI-backed feature is one entry in a single registry file:

\js
export const FEATURES = {
chat_message: { sparks: 2, chain: LITE_FIRST, stream: true },
generate_quiz: { sparks: 1, chain: LITE_FIRST, cacheTtlMs: 30 * 60_000 },
km_build_concepts: { sparks: 8, chain: FLASH_FIRST },
// ...60+ more, one line each
};
\\

Each entry declares its price, its model chain (primary + fallbacks), and its generation config. The router walks the chain on failure:

• Lite-primary features fall back up to Flash at the same price — availability beats margin during a Lite incident.
• Flash-primary features fall back down to Lite at the same price — a degraded answer beats an error page, and the value delta isn't worth the ledger churn.
• Anything that sends images/PDF/video gets no fallback at all — nothing in the app has ever sent media to the smaller model, so its multimodal quality is unverified. Failing loud (with an automatic refund) beats silently returning garbage.

If a cheaper step in the chain ends up answering, the router refunds the price difference automatically. Response caching is an allowlist, not a heuristic — only deterministic-ish generators (quizzes, reviews, curricula) get a TTL; personalized chat is explicitly never cached. And the client-side paywall keeps its own copy of the prices — a test asserts the two stay byte-identical, so drift is a CI failure instead of a support ticket.

None of this is exotic. It's the boring, unglamorous plumbing that every AI product eventually needs and almost none ship on day one — because day one is about the prompt, not the failure mode of the prompt.

The bug that taught us the most: mastery stuck at 0%

Here's the one that stung. We had a whole "mastery" system — spaced repetition, progress bars, the works — and users' mastery scores just... never moved. Not broken-looking, just flat.

The cause: the only thing that triggered a quiz (and therefore a mastery write) was the user typing a hidden keyword like "quiz" or "check" into chat. Practice mode and Review mode — the two surfaces actually designed for this — had no write-back path at all. The UI was doing real work and throwing the result away.

The fix wasn't a clever algorithm, it was making the invisible contract visible: a shared masteryRules\ + pathProgress\ engine, and an actual "Check my understanding" button instead of a magic word nobody was told about. The lesson generalizes past this one bug — if a core feedback loop depends on the user knowing an undocumented trigger, it doesn't exist.

Being honest about failure instead of hiding it

The other pattern worth mentioning: for a while, every AI call that errored was caught and swallowed — console.error(e)\ and render null\. From the user's side, that's just a blank screen with no explanation, indistinguishable from the app being broken. We replaced every one of those call sites with a shared error-state component that actually tells the user what happened and gives them a retry path. Small change, but it's the difference between "this app is broken" and "this app hit a snag" in the user's head — and those get very different reactions.

Where it stands

• 693 tests across 63 files, running on every change
• React 19 + Firebase/Firestore + Cloud Run backend, Gemini 2.5 Flash/Pro/Lite behind the router above
• Five learning modes sharing one credit system, one mastery engine, one sync layer across devices

It's live at vararuchi.com if you want to poke at

I'd genuinely rather have three people tell me what's broken than a hundred silent signups, so — try to break something, and tell me what you find.

Comments

[Alex Shev]

The routing layer is the product here. A general tutor only works if the system can decide which subject logic, examples, and evaluation style apply before it starts answering confidently.