The Neural Engine in your iPhone runs 35 trillion operations per second. Apple uses it for photo processing. You can use it to run vision AI models that analyze images, read documents, and answer questions about anything your camera sees — without uploading a single photo to any server.
Off Grid is a free, open-source app that runs vision AI entirely on your iPhone. No internet connection after the initial model download. No account. No data leaving your device. This guide covers how to set it up, which vision models work best on iOS, and what to expect.
What Vision AI Can Do on Your iPhone
Vision AI means the model can see images, not just read text. You give it a photo and a question, and it answers based on what it sees.
Vision AI
|
Attachments
|
Text Generation
|
Document analysis. Photograph a receipt, invoice, or contract. Ask the model to extract line items, totals, or specific clauses. No need to type anything out manually.
Homework and math. Point your camera at a math problem or a diagram. The model reads it and walks you through the solution.
Code on a screen. Take a photo of code on a whiteboard or another screen. Ask the model to explain it, find bugs, or convert it to a different language.
Scene description. Photograph anything and ask what it is, what's happening, or for details about specific objects.
Accessibility. Describe images, read text in photos, identify objects. All on device, all private.
What You Need
Minimum hardware: iPhone with 6GB RAM (iPhone 13 Pro and newer). The smallest vision model (SmolVLM 500M) fits comfortably.
Recommended hardware: iPhone 15 Pro or newer (8GB RAM). This lets you run the 2B+ vision models that produce much more detailed and accurate responses.
Which Vision Models to Use
Off Grid automatically downloads the companion file (mmproj) that vision models need alongside the main model. You don't need to configure anything.
SmolVLM 500M — The speed pick. About 600MB total download. Runs in roughly 7 seconds on iPhone 15 Pro. Good enough for document reading, basic scene description, and text extraction. Start here.
SmolVLM 2.2B — The quality pick for 8GB+ iPhones. Much more detailed answers. About 10 to 15 seconds on iPhone 15 Pro.
Qwen3-VL 2B — Strong multilingual vision understanding. If you need to analyze documents in languages other than English, this is the model.
Gemma 3n E4B — Google's mobile-optimized multimodal model. Vision plus audio understanding with selective activation to save memory.
How It Works
Vision-language models combine two systems: a vision encoder that "sees" the image and converts it into tokens the model understands, and a language model that reasons about those tokens alongside your text prompt.
When you attach an image and send a message, Off Grid loads both the main model and its multimodal projector. The image is processed locally through the vision encoder, combined with your text prompt, and the language model generates a response. The entire pipeline runs on your iPhone using Metal GPU acceleration.
How This Compares to Apple Intelligence Visual Features
Apple Intelligence can describe images and extract text through built-in iOS features. But it routes certain tasks through Apple's Private Cloud Compute, it's locked to Apple's own models, and you can't control what runs locally versus on their servers.
Off Grid runs everything locally. Every vision inference happens on your iPhone's processor. You choose the model. The code is open source. And the same app works on Android and macOS — your workflow isn't locked to one ecosystem.
Privacy for Sensitive Documents
This is where local vision AI matters most. Medical documents, insurance forms, tax returns, contracts, personal letters, ID cards, bank statements. Every time you upload one of these to a cloud AI for analysis, a copy exists on someone else's server.
With Off Grid, the image goes from your camera to your iPhone's RAM to the model and back to your screen. No network request. No server. No copy anywhere you don't control.
Tips for Better Results
Good lighting matters. The model can only work with what the camera captures. A well-lit, sharp photo produces much better results than a blurry, dark one.
Crop to the relevant area. If you're analyzing a receipt, crop out everything except the receipt. Less visual noise means more accurate answers.
Be specific with your prompt. "What does this say?" is fine. "Extract all line items with prices from this receipt as a list" is better.
Use the right model size. For simple text extraction, SmolVLM 500M is fast and accurate. For complex reasoning about what's in an image, use a larger model.
Getting Started
- Install Off Grid from the App Store
- Download a vision model (SmolVLM 500M to start — about 600MB)
- Tap the camera icon in chat to capture or select an image
- Type your question about the image
- Get your answer — entirely on device
Off Grid also runs text generation, image generation, voice transcription, tool calling, and document analysis. All locally, all in the same app. Check the GitHub for the latest releases.
Top comments (0)