For the past few months I've been building a fitness coach on my own: an iPhone app, an Apple Watch app and a small server for sync. One rule shaped everything: your questions to the coach, and its answers, should never leave your phone. Health data is personal, and most fitness apps send all of it to a server.
So the coach's language model runs on the iPhone itself. Here's what that taught me.
Why on-device
A cloud model would have been easier. It's bigger, smarter and someone else keeps it running. But it means every meal, every night's sleep and every question goes to a server. Running the model on the phone keeps that data where it already is, works offline, and costs nothing per question.
The price is size. A phone can hold a model of about 1.3 GB comfortably, which is small by today's standards.
Distilling a model of my own
Off-the-shelf small models were too general. So I distilled one: a larger "teacher" model produced thousands of worked examples (a meal described in plain words, turned into foods and amounts; a question about your day, answered from your data), and a small model was fine-tuned on them. It took three rounds, each one measured against the last on the same frozen test set before anything shipped.
Where it went wrong: numbers
The small model turned out to be very good at reading. "Chicken salad and a coffee", "two slices of pizza", "a bowl of soup and some bread": it picked out the foods and amounts reliably.
It was bad at numbers, in two ways.
Arithmetic. Asked to estimate a portion's weight, its answer was within 25% of the right weight 73% of the time. When the app worked the weight out instead, from the food and the amount the model had read, it was within 25% 92% of the time, on the same meals.
Putting real numbers on the wrong question. This one was sneakier. Asked how my weight was going, the model replied with my weight, correctly, and then added my recovery score as if it explained something. Every number was real, so a simple "is this number in the data?" check let it through. It was still wrong.
The fix: the app does the numbers, the model reads and advises
I split the work:
The model reads. It understands what you typed and picks the foods and amounts out of it.
The app calculates. Weights, calories, what's left for the day: all deterministic code.
For any question with a number in it, the answer leads with the app's figures, and the model only adds advice after them, with no numbers of its own.
And numbers became a release gate: ten questions with known answers ("what's left to eat today?", "how did I sleep?", "how far am I from my goal weight?"). The model alone got 7 of 10. With the app supplying the numbers, 10 of 10. One wrong number blocks a release.
What testing on a real phone found
The project has over 2,600 automated tests. Then I sent 30 ordinary meals through the app on my own phone, and found three bugs the tests had missed, including one where "2 naan" came to 180 servings because two food tables disagreed about what a serving was. A safety check caught that one before anything was saved, which is exactly why it exists.
The lesson I'd pass on: test your AI on the device, with real messages, before you trust any number it produces.
Where it is now
It's in public beta. You can log meals by message, get a train-or-rest answer from your sleep and heart rate, ask the coach about your day, and log workouts from an Apple Watch.
It's free on TestFlight if you'd like to try it, and I'd love to hear what breaks: https://testflight.apple.com/join/eMrHKRk6
Built with .NET MAUI for iOS, Swift for the Apple Watch, ASP.NET for sync, and llama.cpp for the on-device model.
Top comments (0)