Teaching our on-device assistant to say "I don't know"

Familia Sync — Sun, 20 Sep 2026 21:55:11 +0000

We're building FamiliaSync, an offline-first family organizer where all data stays encrypted on your own devices. That constraint extends to the assistant: every query has to run on the phone, no cloud calls allowed. Here's what happened when we replaced our hand-written intent router with a small on-device classifier.

A smarter first step

Every request to the FamiliaSync assistant now starts with a small intent classifier trained on-device. It replaces the hand-written pattern rules that used to decide what you meant. The pipeline behind it is unchanged in spirit: your text is routed to the right tool, or handed to Needle 2 — the on-device model bundled with the app — when a request needs open-ended reasoning.

The training corpus was researched, not generated in bulk: 105 intents backed by roughly 2,400 example utterances carrying the slots real commands need — dates, times, assignees, quantities, amounts, categories, and recurrence. Thirty-five of those intents map to real tools in the app.

Training the 'no' cases

Here's the part we didn't expect to matter most: 70 of the 105 intents are documented-only. They cover things the assistant can't do yet — notes, stores, loans, meter readings — and they exist so that an unsupported request falls through to the model for a plain-language answer instead of being forced into an unrelated tool. Training the 'no' cases turned out to be as important as training the 'yes' cases.

A confidence gate sits on top: if the classifier isn't sure, it doesn't route at all — the model handles the request instead. And everything happens on-device. Training is lazy and local, inference is local, and no query ever leaves the phone.

Real phrasing broke it within hours

Shipping the router was the easy part. Real queries surfaced three misroutes the same day, and each one taught us something:

"what are todays plan" fell below the confidence threshold — and a stray keyword let the model invent a meal called "Sri, Ram" from family names in the prompt. Loose keywords were stripped, and 'today's plan' phrasings now map to listing events.
"what is my meal plan for today" confidently matched the wrong tool — a read question nearly overwrote the shopping list, stopped only by the confirmation gate. That tool now requires an explicit generate signal, and the corpus grew question-form examples.
The meal-plan answer that did work dumped raw record IDs and the entire week. It now renders meal names and accepts filters like 'today' or 'lunch'.

From commands to conversation

The bigger shift: the assistant can now ask back. Tell it 'create a reminder' and instead of guessing, it asks what to remind you about — then completes the action from your reply. Tools can request a missing argument mid-flow, and partially collected details carry over between turns.

Follow-ups exposed gaps we didn't know existed. When people answer in their own words, dates arrive as 'friday 6pm', 'tonight', or '19th spetember 2026' — so the date parser now handles casual formats, including typo-tolerant month names. And when someone answers 'anyone' or 'doesn't matter' to a member prompt, the event is created unassigned rather than silently picking the first family member.

An assistant that knows when it doesn't know is more trustworthy than one that always guesses.

Conclusion

Nine commits in one day: a trained intent router, honest misroutes fixed in the open, and an assistant that asks follow-up questions — all running entirely on-device, with settings copy updated in four languages. Every query stays on your phone. That's the point.