The latest articles on DEV Community by Falaq (@falaq_ai). https://dev.to/falaq_ai https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4008686%2F42704624-f48e-4dc8-b7e4-c7ea8516983a.png https://dev.to/falaq_ai en Falaq Mon, 29 Jun 2026 19:47:35 +0000 https://dev.to/falaq_ai/what-i-learned-building-offline-ai-for-android-2pi https://dev.to/falaq_ai/what-i-learned-building-offline-ai-for-android-2pi <p>I built Orb because I wanted AI on my phone that still worked when the network disappeared.</p> <p>Most AI apps are simple from the app side: send the prompt to a server, stream the response back, and keep the real complexity in the cloud. Orb is the opposite. It runs local models on Android after a one-time model download, so the app has to deal with the annoying mobile parts directly.</p> <p>A few things stood out.</p> <h2> The model download is part of the product </h2> <p>If your app needs a 1-3 GB model before it becomes useful, the download flow is not plumbing. It is onboarding.</p> <p>Users need to know:</p> <ul> <li>why the file is large</li> <li>whether they need Wi-Fi only for setup</li> <li>if the download can resume</li> <li>how much storage it will use</li> <li>what works offline after that</li> </ul> <p>This is also why I pulled part of the download work into a React Native native module. Large model files need progress, resume, and background-friendly behavior. A normal "just fetch it" flow is not enough.</p> <h2> Offline has to be obvious </h2> <p>A privacy app cannot just say "local-first" and hope people believe it.</p> <p>The product has to make the boundary clear:</p> <ul> <li>no account</li> <li>no cloud chat backend</li> <li>no subscription</li> <li>no ads</li> <li>works without Wi-Fi after setup</li> </ul> <p>For Orb, the point is simple: once the model is downloaded, chat, image/document workflows, speech-to-text, text-to-speech, and local reasoning should not require an internet connection.</p> <h2> Mobile inference is not one performance target </h2> <p>Android hardware is all over the place.</p> <p>Some devices can use GPU acceleration. Some Snapdragon devices may have experimental NPU/HTP paths. Many devices need CPU fallback. The same model can feel completely different across phones.</p> <p>So the app cannot be marketed like every phone gets the same experience. The honest version is: use the best local path available, fall back safely, and keep model choices realistic.</p> <h2> Privacy is a workflow, not a slogan </h2> <p>The strongest use cases are not "replace ChatGPT." They are moments where sending data to a remote service feels wrong or impossible:</p> <ul> <li>private notes</li> <li>personal documents</li> <li>travel with no signal</li> <li>journaling</li> <li>studying offline</li> <li>screenshots/photos you do not want uploaded</li> </ul> <p>Local AI is weaker than cloud AI in many ways, but it wins when the private/offline boundary matters.</p> <h2> Orb </h2> <p>Orb is my attempt at making this feel normal on Android: local models on-device, image and document upload, speech-to-text, text-to-speech, no account, no subscription, no ads, and no cloud chat backend.</p> <p>It is Google Play only for now:<br> <a href="https://play.google.com/store/apps/details?id=com.falaq.orb" rel="noopener noreferrer">https://play.google.com/store/apps/details?id=com.falaq.orb</a></p> android ai privacy reactnative