Most AI apps pick one model for you and bill you for it. In Fud AI (an open-source calorie tracker) I went the other way: the user brings their own key.
Why BYOK
Food analysis is a vision + reasoning task that almost every modern model can do. Locking users to one provider means one bill, one rate limit, one point of failure. Letting them choose means cost control, resilience, and privacy — requests go device → provider directly.
The provider routing
Fud AI routes across 13 providers behind one interface — Gemini, OpenAI, Claude, Grok, Groq, Mistral, OpenRouter, Together, Hugging Face, Fireworks, DeepInfra, plus any OpenAI-compatible endpoint, plus local Ollama. On supported iPhones it can also fall back to on-device Apple Intelligence, so text/voice logging works with no network at all.
Lessons
- A thin provider abstraction + a configurable fallback chain pays off fast.
- Transient overloads (429/503/529) should auto-retry with backoff so spikes resolve invisibly.
- On-device fallback is a genuine privacy and resilience feature, not just a nice-to-have.
Code: https://github.com/apoorvdarshan/fud-ai · App: https://fud-ai.app
Top comments (0)