I'm building Finovaa, a budgeting app with an AI coach. It's launching on iOS and Android soon.
The scariest bug in a money app isn't a crash. It's the coach confidently telling you that you have $240 left when you have $24. A crash is obvious. A wrong number said in a friendly voice gets believed.
So the coach is built on one rule, and four pieces of code enforce it.
The rule: decide locally, let the model only rephrase
Every coaching surface has to be correct, instant and useful before any model is called. The model's only job is to reword a decision the app has already made. If the model is slow, down, over quota, or wrong, the user still gets the right answer.
1. A pure signal engine decides what to say
Detectors for budget pace, unusual spending, recurring charges, goal gaps and month-end shortfall run as pure, synchronous functions over data the screen has already loaded. Their results are ranked by severity: risk > watch > info > win.
Two things make this work:
-
They're pure and clock-injected.
nowis passed in, so every detector is an ordinary unit test. No mockingDate, no flaky "it fails on the 31st". - Returning nothing is a normal outcome. A coach that always has something to say is noise. Most days, most signals are silent.
2. A deterministic sentence ships first
Each signal builds a real sentence from its own facts, synchronously, before any network call. The free tier gets exactly this, so nobody ever sees an empty card or a spinner where advice should be. The numbers in it come from the same engine the rest of the screen uses, so the card can never disagree with the chart above it.
3. The model gets the facts, as plain lines
For Pro users, the app asks the model to reword the verdict. The facts go in as short, labelled lines, something like:
Budget: Groceries, monthly, limit £520
Day 5 of 30; 25 days left including today
Spent: £83 (16% of the limit)
Safe to spend per day for the rest: £17
One small detail that mattered: a projection from the first few days of a month is one purchase extrapolated over thirty days. The screen doesn't show it that early, so the model isn't handed it either. Anything you don't want the model to say, don't give it.
4. A grounding check throws out invented numbers
Before a reply is shown, every number in it has to appear in the facts we sent. If one doesn't, the reply is dropped and the deterministic sentence stays on screen.
const numbersIn = (text: string): string[] =>
(text.match(/\d+(?:[.,]\d+)*/g) ?? []).map(n => n.replace(/,/g, ''));
export const isGroundedReply = (reply: string, facts: string): boolean => {
const text = reply.trim();
if (!text || text.length > MAX_REPLY_CHARS) return false;
if (/undefined|null|NaN|\{\{/.test(text)) return false;
const allowed = new Set(numbersIn(facts));
return numbersIn(text).every(n => allowed.has(n));
};
It's deliberately blunt. It doesn't try to understand the sentence; it just refuses any figure it didn't supply. That also catches the embarrassing failures: a template that leaked {{amount}}, or a reply that says "undefined".
(The real version also normalises Arabic-Indic digits, because the app ships in Arabic and "١٧" should match "17".)
What this buys
- The free tier is complete. The AI is an upgrade to the wording, not a dependency of the feature.
- Outages are invisible. If the model fails, the user sees a correct sentence instead of an error.
- The tests are boring. The logic that matters is pure functions with plain inputs and outputs.
- Trust. The coach can be warm, blunt or funny, but it can't be wrong about your money in a way the app wasn't.
The trade-off is that the model can't come up with an insight the engine didn't detect. For money, I'll take that trade every time.
If you're curious what this looks like in the app, the waitlist is open at finovaa.app. I'd also love to hear how other people are keeping LLM output honest in apps where the numbers matter.
Top comments (1)
Deciding synchronously and treating the model strictly as a stylistic formatter is the cleanest way to guarantee uptime and correctness.One subtle edge case we hit with token-set grounding in financial pipelines: prompt metadata leaks into the allowed numeric set. If the context contains "Day 5 of 30" or "1% of the limit6", integers like 5, 30, and 16 become valid numbers. If the model hallucinates "You spent £5 on lunch", the check passes because 5 exists in the prompt, even though it represented a calendar day rather than a monetary amount.To prevent that, we bind numbers to typed slots before generation, or have the model output structured keys and interpolate them on the client. Scoping the allowed set by entity type (monetary amounts checked only against currency figures, counts checked against calendar days) stops dates from validating currency slips.