DEV Community: Mohammed Ali Chherawalla

How to Speed Up Local AI Replies With MTP on Your Android Phone in 2026 (No Setup)

Mohammed Ali Chherawalla — Sun, 25 Oct 2026 12:54:20 +0000

You run a local model on your Android phone and the reply crawls out word by word. You would like the same answer sooner, without swapping to a smaller model and without reading about draft models. Off Grid AI Mobile (OGAM) now supports MTP, multi-token prediction. A model built for it guesses several words ahead, checks them in one pass, and keeps the ones it would have written anyway. You turn it on with one switch. If the model needs an extra file for it, the app downloads that file for you.

What you need

An Android phone with enough RAM for the model you use. Gemma 4 E4B, the smallest catalog model with a published MTP head, lists 6 GB of RAM as its minimum. Gemma 4 12B lists 12 GB.
OGAM 0.0.116 or later. 0.0.116 added the automatic MTP head download behind the switch. The current release, 0.0.118, also downloads the head together with the model on one progress bar. Download OGAM for Android.
A GGUF chat model that runs on llama.cpp. MTP does not apply to the Gemma 4 LiteRT models, which use Google's LiteRT engine.
No account and no payment. MTP is part of the free app.

How MTP makes replies faster

A normal model writes one token per pass through its weights. A model with MTP layers drafts a few tokens at once, then verifies all of them in a single pass. Every token that passes the check is exactly the token the model would have produced on its own, so the answer does not change. You only wait through fewer passes.

Some models carry these draft layers inside their own GGUF file. Others have them published as a separate small file called an MTP head. OGAM's catalog knows the published heads for Gemma 4 E4B (about 99 MB), Gemma 4 12B (about 465 MB) and Qwen 3.8 27B (about 1.37 GB), and matches each head to its model whatever quantization you download. On an Android phone, Gemma 4 E4B is the one most people can run. Gemma 4 12B needs a 12 GB phone, and Qwen 3.8 27B lists 32 GB.

Step 1: Find a model with the MTP badge

Open OGAM and tap Models in the bottom bar.
Look for the MTP badge, with a fast-forward icon, at the start of the badge row on a model card. It means the model has a published MTP head and replies faster with MTP on. A screen reader reads it as "MTP: replies faster".
Tap the model to download it. In 0.0.118 one tap downloads the model, its vision file and its MTP head on one progress bar. For Gemma 4 E4B that is the 4.98 GB model, the 0.99 GB vision file and the 99 MB head.

The same badge shows in the model sheet inside a chat, so you can check the model you are on without leaving the conversation.

If you downloaded the model before 0.0.118, or the head download failed, you do not need to download the model again. The next time the model loads, the app fetches the missing head in the background. That load runs without MTP, and the load after it uses the head.

Step 2: Turn on Speculative Decoding (MTP)

The Settings tab on Android. Model Settings is the first item.

Tap Settings in the bottom bar.
Tap Model Settings (System prompt, generation, and performance).
Open the Text Generation section.
Tap Advanced.
Find Speculative Decoding (MTP) and tap On.

You can do the same from inside a chat. Tap the sliders icon at the top right, open TEXT GENERATION, tap Advanced and set Speculative Decoding (MTP) to On. Both places change the same setting.

If the selected model has a published head that is not on the phone yet, the download starts as soon as you tap On. The progress shows under the switch as Getting MTP head... with a percentage. If it fails, a retry icon appears in its place. Tap it once to try again.

Step 3: Reload the model and chat

MTP is set when llama.cpp builds the model in memory, so it takes effect on the next load. The setting says Requires model reload. Go back to your chat. A banner with a warning icon says the settings changed and offers to reload the model. Tap it, wait for the model to load, and send a message. Changing Draft Tokens later shows the same banner.

To see the speed, turn on Show Generation Details in the same Text Generation section. Each reply then shows its speed in tokens per second, so you can compare it with MTP off.

The in-chat suggestion

When the model file you use carries its own MTP layers, OGAM tells you in the chat instead of waiting for you to find the setting. A card reads This model can reply faster, says the model drafts several tokens at once and checks them in one pass, and offers Turn on speculative decoding and reload the model. Tap it and the app turns MTP on and reloads the model in one step. Tap the close icon to dismiss the card. The card does not appear for models without draft layers, and it does not appear once MTP is on.

Draft Tokens

With MTP on, a Draft Tokens slider appears below the switch. It sets how many tokens the model drafts per step, from 1 to 8. The default is 3, which is llama.cpp's own default. Longer drafts help when the model guesses well and waste work when it guesses badly. Start at 3, and change it only if your measured speed says so.

What this does not do

It does not speed up every model. Only models with MTP layers in their file, or a published MTP head, can draft. On any other model the app does not enable MTP, even with the switch on, because passing it to a model without draft layers can leave the engine stuck.
It can make the first word slower in long chats. With MTP on, llama.rn reprocesses the whole prompt each turn, so from the second message on, the wait before the first token grows with the conversation. This is why MTP is off by default, and why Use fastest settings turns it off. Turn it back on yourself after the speed test if you prefer it.
It does not apply to LiteRT models, remote servers or image models.
An MTP head is not a model. Since 0.0.117, files that only speed up another model are no longer offered as models you can load. If you import one, the app refuses it with "This file only speeds up another model. It cannot run on its own."

FAQ

What is MTP in local AI?
MTP stands for multi-token prediction. A model trained with it has extra layers that guess the next few tokens, and the main model checks those guesses in one pass. Correct guesses are kept, so you get the same reply in fewer passes.

Is MTP the same as speculative decoding?
MTP is one form of speculative decoding. Classic speculative decoding uses a separate small draft model. MTP uses draft layers trained with the model itself, either inside its GGUF or in a published MTP head.

Which Android phones can use MTP?
Any Android phone that runs a model with MTP layers in OGAM can use it. The limit is the model's RAM need: Gemma 4 E4B lists 6 GB as its minimum, and the head adds about 99 MB of storage.

Does Gemma 4 support MTP on Android?
Yes, the GGUF versions. MTP heads for Gemma 4 E4B and Gemma 4 12B are published beside Unsloth's GGUF builds, and OGAM downloads the matching head with the model. The Gemma 4 LiteRT versions do not use MTP.

Why is my first reply slower after turning on MTP?
The model reloads when you change the setting, and with MTP on, llama.rn reprocesses the full prompt each turn. In a long chat that can slow the first token even while the rest of the reply comes faster.

Does MTP need internet?
Only once, to download the MTP head if the model uses one. After that, MTP runs on the phone with no connection.

How to Ask Questions About a 500-Page PDF on Your Mac in 2026 (Page Citations, No Cloud)

Mohammed Ali Chherawalla — Sun, 25 Oct 2026 11:23:27 +0000

You have a 500-page contract, annual report or textbook as a PDF on your Mac. You want to ask "what does it say about termination?" and get an answer you can check, not a guess. Off Grid AI Desktop reads the whole file on your Mac, answers from it, and puts a small page chip such as p. 212 next to each claim. Click the chip and the PDF opens at that page. The file never leaves your Mac, and there is no page limit and no file size cap.

What you need

A Mac with Apple silicon (M1 or later) and macOS 13 or later. The Mac download is built for Apple silicon only.
Off Grid AI Desktop 0.0.55 or later, the current stable release. Download Off Grid AI Desktop for Mac.
A chat model downloaded in the app. Any local text model works. A larger model writes better answers; the documents part works the same with all of them.
No account and no payment. Document chat is part of the free app.

Page citations arrived in 0.0.55. If you used an older build, the old limits are gone: a long attachment no longer ends the reply with "Context is full.", and follow-up questions no longer lose the document.

Step 1: Attach the PDF

Open a chat in Off Grid AI Desktop.
Click the + button at the left of the message box and choose Attach files. A Finder window opens. Select one PDF or several.
Or drag the files from Finder onto the message box. It reads Drop files to attach while you hold them over it.

The app reads PDF, Word (.docx), plain text, Markdown, CSV and JSON files. Each file shows as a small pill in the message box. You can type and send your question straight away. The pill stays usable while the app reads and indexes the file in the background.

Step 2: Watch it read the file

As soon as a chat has a document, a button with a file icon and a number appears in the row under the message box. Its tooltip reads, for example, "1 document in this chat". Click it to open Documents in this chat. Each file has a status line:

Reading... while the app pulls the text out of each page.
Reading scanned pages 12/40 when pages are images with no text layer, for example a scanned contract.
Indexing 45% while it builds the search index.
512 pages when it is ready.

On a Mac, scanned pages are read with Apple Vision, the text recognition built into macOS. Off Grid AI's own measurement puts it at about 0.2 to 0.5 seconds a page, and it reads Latin and Chinese, Japanese and Korean text. For a document in Hindi, Marathi or Nepali, the app uses PaddleOCR instead and downloads its Devanagari model the first time it needs it.

If some pages cannot be read, the status line says so, for example "3 scanned pages not read". The app does not pretend those pages are in its answers.

Step 3: Ask your question

Type the question the way you would ask a colleague who read the document: "What are the termination clauses, and what notice does each one need?"

While the app prepares the answer, the work timeline above the reply shows what it is doing: "Opening 1 document...", "Searching 1 document...", "Ranking 24 passages...". Then the model writes.

How much of the document the model reads depends on your Mac. The app measures how fast your Mac reads text into the model, and compares that with the model's context window.

A document that fits is read whole, every page, on every question.
A document that does not fit, such as a 500-page PDF on a smaller model, is read as a map of the whole document plus the passages that best match your question. The app finds them with a keyword index and a local embedding model (multilingual-e5-small), then re-ranks them with a second local model before the chat model sees them.

Both happen on every turn. A follow-up question searches the document again for that question, so it does not depend on what the previous answer happened to quote.

Step 4: Check the answer against the page

Each claim drawn from the PDF ends with a green chip that names the page, such as p. 212.

Hover over a chip to see the file name and the passage it cites.
Click a chip to open the PDF in the viewer beside the chat, at that page. Use Fit to screen in the viewer if the page is too wide for the panel.

This is the part that makes a long document usable. You do not have to trust the summary. You read the sentence on page 212 yourself.

Work with several documents in one chat

You can attach more files later in the same chat, for example last year's report next to this year's. Every answer reads every included document.

To leave a file out, open Documents in this chat and turn off its switch. The file name is struck through and the button shows a count such as 2/3. Turn the switch back on to include it again. The eye icon next to each file opens it in the viewer.

Use a project for documents you ask about often

If you ask about the same set of files across many chats, such as a policy library or a course reader, put them in a project instead.

Open Projects and create a project.
Under Knowledge base, click Add files and pick the documents. Each file shows its progress while it is indexed.
Start chats inside the project.

Then choose how chats use them in Settings > Text > Project knowledge:

When needed (the default) searches the knowledge base only when a question needs it, so other answers in the project stay fast.
Relevant passages searches it for every question and adds the best matches.
Every answer reads all of it each time.

What this does not do

It does not read Excel, PowerPoint, Pages, Keynote, Numbers, old .doc files or ebooks yet. Export them to PDF first.
It does not make a small model smart. A 1B model can find page 212, but its answer will be weaker than a 9B model's. Pick the largest model your Mac runs well.
On a 500-page file that does not fit the model, the answer comes from the best-matching passages, not every page. A question like "list every date in the document" can miss some. Ask narrower questions, or split the work by chapter.
Pages Apple Vision cannot read, such as poor scans or handwriting, are reported as not read rather than guessed.

Where the document goes

The PDF stays on your Mac. The text, the index and the page map are stored in the app's encrypted database. Nothing is uploaded. If you back up the app, the backup includes the document text. If you delete the chat, the app also deletes the documents no other chat uses.

FAQ

Can I chat with a PDF on my Mac without uploading it?
Yes. Off Grid AI Desktop reads the PDF on your Mac with a local model. The file, its text and the answers stay on your Mac.

Is there a page or size limit for PDFs?
No. The app reads the PDF page by page with no size cap. A document too large for the model is read as a map plus the passages that match each question.

Does it work with scanned PDFs on a Mac?
Yes. Scanned pages are read with Apple Vision, the text recognition built into macOS. Pages it cannot read are listed as not read.

How do I know the answer is really in the PDF?
Each claim carries a page chip such as p. 212. Hover to see the passage, or click to open the PDF at that page.

Does it work offline?
Yes, once you have downloaded a chat model. Reading the PDF, searching it and writing the answer all run on your Mac.

Is it free?
Yes. Document chat with page citations is part of the free Off Grid AI Desktop app, with no account.

How to Retry, Edit and Queue Questions to Local AI on Your Android Phone in 2026 (Every Answer Kept)

Mohammed Ali Chherawalla — Sun, 25 Oct 2026 06:10:02 +0000

You asked the model on your Android phone a question, the answer was close but not right, and you want another try without losing the first one. Or the model is still writing and you already know your next three questions. Off Grid AI Mobile (OGAM) handles both. Every retry and every edit keeps the old answer as a version you can step back to, and anything you type while a reply runs waits in a queue above the box until the model is free. The model runs on your phone, so none of this goes anywhere.

What you need

An Android phone with a chat model downloaded. Any GGUF or LiteRT chat model works, for example Qwen 3.5 2B or Gemma 4 E2B. If you have not set one up yet, start with How to Run Local AI on Your Android Phone in 2026.
OGAM 0.0.117 or later. Versions and the queue arrived in the 0.0.115 beta and are in the stable releases, 0.0.117 and 0.0.118. Download OGAM for Android.
No account and no payment. Both features are part of the free app.

How the message menu works

Every message in a chat has a menu. Long-press the message bubble, or tap the three dots (...) next to its time, and a sheet called Actions opens. The menu does not open while a reply is still being written.

What the sheet offers depends on whose message it is:

On your own message: Copy, Edit and Resend.
On the model's answer: Copy, Edit and Regenerate, plus other actions such as Generate Image when they apply.

Step 1: Get a new answer without losing the old one

Long-press the answer you want to replace, or tap its three dots.
Tap Regenerate.
The chat ends at your question while the model writes a new answer.

When the new answer is done, a small switcher appears under it: a left arrow, a count such as 2/2, and a right arrow. Tap the left arrow to go back to the first answer. Tap the right arrow to return to the new one.

Resend on your own question does the same thing from the other side. It runs the question again and adds the reply as a new version beside the old answer.

A retry runs the turn the same way it ran the first time. If your question asked for an image, Resend draws an image again instead of answering in text.

If a retry fails before it saves an answer, the chat goes back to how it was. Nothing is deleted.

Step 2: Fix your question and ask again

Long-press your question, or tap its three dots.
Tap Edit. The EDIT MESSAGE sheet opens with your text in it.
Change the text and tap SAVE & RESEND. Tap CANCEL to leave it as it was.

The edited question becomes a new version beside the original, and the model answers it. The original question keeps its own answer and every turn you had after it. The switcher appears under your question this time. Step back to the first version and the whole conversation that followed it comes back with it.

This is the part that matters in a long chat. Say you asked a question at turn 3 and kept going to turn 12. Edit turn 3, and turns 4 to 12 are not lost. They stay with the old version of turn 3.

Step 3: Correct an answer by hand

Sometimes the model's answer is nearly right and you only want to fix a number or a name before you build on it.

Long-press the answer and tap Edit.
Change the text and tap SAVE.

On the model's answers the button reads SAVE, not SAVE & RESEND, because nothing is run again. Your corrected text is saved as a new version of the answer, and the model's original stays one tap away in the switcher.

Which version the model reads

When you send your next message, the model reads the chat as you see it on screen: the versions you have selected, and nothing from the versions you stepped away from. To continue from the first answer instead of the second, step back to it with the switcher and then type.

Step 4: Queue your next questions while the model writes

On a phone a long answer can take a while. You do not have to wait to type.

While a reply is running, type your next message.
Tap the send button. The message does not interrupt the model. It goes into the queue, which shows above the text box with a count such as 1 queued and one row per message.
Keep typing and sending. Each message joins the queue.

When the reply ends, everything in the queue for that chat goes to the model together as your next turn. The queue belongs to the chat you wrote it in.

Each queued row has two buttons:

The return arrow sends that one message now. The running reply stops and the message goes on its own. The rest of the queue waits for it.
The trash icon removes that message from the queue.

The x on the right of the queued badge clears the whole queue.

To skip the queue for a message you have not sent yet, long-press the send button instead of tapping it. The reply stops and your message goes at once.

Step 5: Stop without losing what you queued

When the text box is empty, the send button turns into a square Stop button. Tap it and the model stops writing.

A Stop does not throw your queue away, and it does not send it either. The queue holds and the badge reads Paused after Stop. Tap Send next to it when you are ready, or remove the messages you no longer need first. If a reply fails, the queue holds the same way and reads Paused after an error.

Switching apps mid-reply

Since 0.0.117, if you switch to another app while a reply runs, the model keeps working when you come back, and you no longer need to eject and reload it. Since 0.0.116, your place in a chat is kept when you switch chats or leave the app, and your three most recent chats pick up faster when you return.

What this does not do

There is no list of all versions at once. You step through them one at a time with the arrows.
The queue does not send messages one by one. Everything queued in a chat goes as a single turn when the reply ends. Use the return arrow on a row to send one message by itself.
Editing an answer does not teach the model anything. It changes what the model reads in this chat from that point on, and nothing else.
Regenerate uses the model and settings you have now. For a different kind of answer, switch model or change its settings first, then tap Regenerate.

FAQ

Can I regenerate an answer in a local AI app on Android without losing the old one?
Yes. In OGAM, long-press the answer and tap Regenerate. The old answer stays as a version, and the arrows under the new answer switch between them.

What happens to the rest of the chat when I edit an earlier question?
It is kept. The edited question is a new version, and the original question keeps its answer and every turn after it. Step back with the arrows and that thread comes back.

Can I type while the AI is still answering on my phone?
Yes. Messages you send while a reply runs wait in a queue above the text box and go to the model together when the reply ends.

How do I interrupt a long reply with a new question?
Long-press the send button. The running reply stops and your new message goes at once. For a message already in the queue, tap the return arrow on its row.

Does tapping Stop delete my queued messages?
No. The queue holds and shows Paused after Stop. Tap Send when you want them to go, or remove them.

Does any of this need internet?
No. The model, the versions and the queue are all on your phone. Airplane mode works.

How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Your Mac in 2026

Mohammed Ali Chherawalla — Sun, 25 Oct 2026 03:16:44 +0000

Your MacBook Air has 16 GB of memory. The PC under your desk, or the Mac Studio in the office, has a big GPU and already runs Ollama or llama-server with a 30B model loaded. You want to chat with that model from your Mac, in a proper app with chat history, documents and thinking controls, without copying a 20 GB file to a laptop that cannot run it. Off Grid AI Desktop connects your Mac to that server. Its models show up next to your local ones, and the chat tells you which server answers and what it is waiting for.

What you need

A Mac with Apple silicon (M1 or later) and macOS 13 or later. The Mac download is built for Apple silicon only.
Off Grid AI Desktop 0.0.55 or later, the current stable release. Download Off Grid AI Desktop for Mac.
Another computer on your network running Ollama or llama.cpp's llama-server, or any server with an OpenAI-compatible /v1 API.
The server's network address, for example 192.168.1.30.
No account and no payment. Remote servers are part of the free app.

Your Mac does not need to download the model. The model runs in the other computer's memory, and your Mac sends text over your network.

Step 1: Let the server accept calls from your network

Both servers listen only on the computer they run on by default. Open them to your network first.

Ollama listens on port 11434. On the server, set OLLAMA_HOST to 0.0.0.0 and restart Ollama. If the server is a Mac running the Ollama app, run this in Terminal on that Mac, then quit and reopen Ollama:

launchctl setenv OLLAMA_HOST "0.0.0.0"

llama.cpp listens on port 8080. Start llama-server with --host:

llama-server -m qwen3.5-27b-q4_k_m.gguf --host 0.0.0.0 --port 8080

If the server's firewall asks whether to allow incoming connections, allow them. Only do this on a network you trust, or add an API key to the server.

Step 2: Add the server in Off Grid AI Desktop

Open Off Grid AI Desktop and click Settings in the sidebar.
Find the card Remote model server. It reads "Connect to a model server on your network or another trusted host."
Under Saved servers, click the Add server icon.
In Server name, type a name you will recognise in chat, for example Office PC.
In Address, type the server's address with its port: http://192.168.1.30:11434 for Ollama, or http://192.168.1.30:8080 for llama.cpp. You do not need to add /v1. The hint under the field shows the exact address the app will call, for example "Will connect to: http://192.168.1.30:11434/v1/models".
If the server needs a key, paste it into API key (optional). The key is stored in the macOS keychain, not in a settings file.
Click Test connection. The status line reads, for example, "Connected in 14 ms. 6 models found." If it reads "Connection failed.", go back to Step 1.

Step 3: Pick the models and save

Under Text + Vision, search the server's models and select the one you want to chat with. The line under it changes to "Selected: " and the model's name.
Optional: open Other models. If the server also offers image, voice or transcription models, pick them under Image, Voice and Transcription.
Leave Use this server on. Its hint reads "Off keeps it saved; other servers are unaffected."
Click Save. The status reads "Server saved and active."

The app saves a server only after you pick at least one model. If you skip that, it says "Test the connection and select at least one model first."

For a llama.cpp server, you also see Allow screen images. It stays off until you turn it on, and it controls whether Web Use and Computer Use may send screenshots of your Mac to that server. Normal chat does not need it.

Step 4: Chat on the remote model

Open a new chat with Cmd+T. Open the model picker in the chat: the server's models carry a Remote tag, next to the models on your Mac. Pick one and type.

While you chat, the app reports the remote model truthfully:

Which server answers. The chat status shows the remote model and the server name, for example Qwen 3.5 27B · Office PC, instead of a stopped local engine.
Thinking. If the model can think, the thinking switch works on the remote server too. The app asks Ollama for the model's capabilities and asks llama-server for its chat template, so the switch sends the control that server reads. The switch stays available while the server is still loading the model.
Context window. A chat on a remote model uses the remote server's context window, not your Mac's local setting. In the model settings, the Context window row reads "Set by the remote model server." A llama.cpp server reports its window, so you see the real number. Ollama does not report one, so the row reads Not reported and the server decides how long a reply can be.
Several chats at once. Two chats on a remote model run side by side. One no longer waits for the other to finish, as it would on a single local engine.

Run several servers

You can save more than one server, for example Ollama on a gaming PC and llama-server on a Mac Studio. Each one has its own Use this server switch, so you can turn one off without touching the others. Turning one off moves chat, image, voice or transcription only to a server that offers a model for it. Picking an image model on one server does not move your chat to that server.

If a server is off or removed while a chat uses it, the chat says so. It does not quietly answer with a different model.

Use Change model next to a saved server to pick a different model from its list, and Remove to delete it.

When the server is another Off Grid AI Desktop

If the other computer runs Off Grid AI Desktop, use its gateway at port 7878 instead of Ollama. That server tells your Mac what a request is waiting for, and the chat shows it: "Waiting for Office PC · 2 ahead" when other people are in line, or "Office PC is loading" and the model name while it loads. Ollama and llama.cpp do not send this, so with them the chat shows "Waiting for the model server..." until the first words arrive. See how to run a local AI server for your whole team on your Mac.

What this does not do

It does not work outside your network by itself. To reach a server at home from a cafe, use a private network such as Tailscale. See how to use your home AI models from your laptop anywhere.
It does not make the server faster. Reply speed is the server's speed, plus a few milliseconds of network time.
It does not download or manage models on the server. Pull models with ollama pull, or start llama-server with the model you want, then click Test connection again to refresh the list.
With Ollama, the app cannot show a context window, because Ollama does not report one.
Your prompts leave your Mac and go to that server. They stay on your network only if the server is on your network.

FAQ

Can I use Ollama running on another computer from my Mac?
Yes. Set OLLAMA_HOST=0.0.0.0 on the server, then add http://<server-ip>:11434 under Settings > Remote model server in Off Grid AI Desktop. The server's models appear in the model picker with a Remote tag.

Does Off Grid AI Desktop work with llama.cpp's llama-server?
Yes. Start llama-server with --host 0.0.0.0 and add http://<server-ip>:8080 as a server. The app reads the server's context window and thinking support from llama-server itself.

Do I need to download the model to my Mac?
No. The model stays on the server. Your Mac sends the conversation over the network and shows the reply.

Can I switch between local and remote models?
Yes. Local and remote models sit in the same model picker. Pick a local model for offline work and a remote one when you are on the same network as the server.

Is it free?
Yes. Remote servers are part of the free Off Grid AI Desktop app on Mac, with no account.

A Private Microsoft Recall Alternative on Your Mac in 2026 (Local AI Reviews Your Day Overnight)

Mohammed Ali Chherawalla — Sat, 24 Oct 2026 21:35:09 +0000

You want a searchable memory of your workday on your Mac, the kind of thing Microsoft Recall does on Windows. But you do not want your screen sent to a cloud service, and you do not want a model reading screenshots while you are trying to work. Off Grid AI Desktop Pro saves your screen as you work and leaves the heavy part, the AI analysis, for the night. By default it runs from 3:00 to 8:00, it can wake your Mac to do it, and the model runs on the Mac itself. In the morning your Day view has the journal, timeline and to-dos from yesterday.

Microsoft Recall is a Windows feature for Copilot+ PCs, so it is not available on a Mac. This is the closest thing you can run on Apple silicon today.

What you need

A Mac with Apple silicon. The Mac download is built for Apple silicon only.
Off Grid AI Desktop 0.0.55 or later. Overnight analysis is in the current stable release. Download Off Grid AI Desktop for Mac.
Off Grid AI Pro. Screen capture and the Day view are Pro features. Pro is monthly or a one-time lifetime price; the Pro page shows today's price.
A local vision model downloaded in the app. The capture settings recommend a 16K context window for it.
Screen Recording permission for Off Grid AI Desktop in macOS. Accessibility permission is optional, but the app reads exact names, URLs and text better with it.

The Day view on a Mac. This is what the overnight pass fills in.

How it works

Capture has two halves. Taking a screenshot and reading the text on screen is light, so it runs all day. Analyzing each saved moment with a local model is heavy: it is one model pass per frame. Overnight mode splits them. Screenshots are saved as you work, the frames wait in a queue, and the model works through the queue inside the window you choose.

Nothing is lost while frames wait. They are saved to disk as pending and stay there until they are analyzed.

Step 1: Turn on capture and choose Overnight

Open Off Grid AI Desktop and click Replay in the sidebar.
At the top, the capture status shows Paused. Click Resume capture. If it already shows Capturing, skip to Step 2 and pick Overnight there.
The first time you turn capture on, the app asks: "When should Off Grid analyze your day?" Two options appear:
- Overnight (recommended): "Your Mac stays fast all day. Analysis runs 3:00-8:00 and can wake it."
- As it happens: "Your day appears as you go, using some power while you work."
Click Overnight (recommended).
macOS shows a password prompt from Off Grid AI. Enter your Mac password and click OK. This lets the app set a nightly wake at the start of the window.

If you cancel the password prompt, overnight analysis still runs, but only while your Mac is already awake. The app tells you so: "The password prompt was cancelled, so the computer will not wake on its own."

You are asked this question once. To change your answer later, use Settings.

Step 2: Change the hours in Settings

Click Settings in the sidebar.
Scroll to Capture & processing.
Under Processing, find the Analyze row. If it reads "As it happens.", switch its control to Overnight and enter your Mac password when asked. It then reads "Overnight. Your Mac wakes at 03:00 to do it."
Two time fields appear next to it: Analysis starts and Analysis ends. Set them to any hours you are away from the Mac, such as 01:00 to 06:00. A window can cross midnight, such as 22:00 to 06:00.
To go back to analyzing during the day, switch the control from Overnight to As it happens.

When you change the start time, the app moves the nightly wake to match. It may ask for your password again.

Step 3: Leave the Mac on power and the app running

The Mac does the work, so a few things must be true at night:

Off Grid AI Desktop is running. It can sit in the menu bar. If you quit it, nothing is analyzed.
A MacBook is plugged in. The wake is set with the macOS pmset scheduler. While frames are waiting inside the window, the app keeps the Mac awake, then lets it sleep again when the queue is empty or the window ends.
The Mac can be locked. A locked Mac stops new screenshots, but it does not stop the analysis of frames already saved. That is the point of the overnight window.

Analysis does wait if you paused capture yourself. The Day view says so instead of promising a run that will not happen.

Step 4: Read your day in the morning

Click Day in the sidebar. The journal, timeline, time spent by app and to-dos are built from what the model read overnight.

During the day, the Day view shows how much is waiting, for example "42 moments waiting · analyzed tonight at 03:00". If you want the results now, click the Analyze now button (the play icon) next to that line. It analyzes today's pending frames straight away.

If you work on battery with As it happens selected, the Day view offers the switch once: "On battery. Analyze overnight instead?" Click Overnight to switch, or dismiss it.

Check what happened overnight

The Capture & processing card has a Pipeline health panel with Screen access, Accessibility text, Vision model, Frame queue, Last frame and Last observation. The queue counts update as frames move.

If some frames failed, a panel reads, for example, "3 frames need another pass". It groups them by the real reason, with a count for each, and says how many "will retry on their own" and how many "need you". Frames stay saved, and a retry button sits next to them.

A frame can also be too large for the active model's context. Those show as "frames are waiting for a larger model". There is no retry button for them, because retrying would not help. Raise the context or switch to a larger model, and they are analyzed on their own.

What this does not do

It does not run when the app is closed. Off Grid AI Desktop must be running at night.
It does not replace a wake schedule you already set. macOS keeps one repeating wake. If your Mac already wakes at another time, the app says "Your Mac already wakes every day at" that time and does not change it. You can remove that schedule or set the window to start at your existing wake time.
It does not send your screen anywhere by default. Analysis uses a local model. One exception: if you have no local model installed and you selected a remote chat server in the app, capture analyzes with that server instead. Install a local vision model if you want every frame to stay on the Mac.
It does not capture everything. Apps you add under Excluded apps in Capture & processing are not saved, and nothing is saved while capture is paused.

FAQ

Is there a Microsoft Recall for Mac?
Microsoft Recall itself is Windows only. Off Grid AI Desktop Pro gives you the same kind of searchable screen history on a Mac with Apple silicon, analyzed by a local model on the Mac.

Does my Mac need to be awake all night?
No. The app asks macOS to wake the Mac at the start of the window, keeps it awake while frames are waiting, then lets it sleep. Keep a MacBook plugged in so the wake and the work are not cut short.

Why does Off Grid AI ask for my Mac password?
Setting a repeating wake with pmset needs administrator rights, so macOS asks. The app only removes a wake it set itself, and it asks again when you turn the wake off.

Can I see my day before the overnight run?
Yes. Click Analyze now next to the waiting count in the Day view, or Re-process today under Today's captures in Settings. Live capture pauses while that pass runs.

Will it slow down my Mac during the day?
With Overnight selected, the model work waits for the window, so daytime use is limited to taking screenshots and reading on-screen text. With As it happens, analysis runs while you work and uses more power.

Download Off Grid AI Desktop for Mac | Explore Pro

How to Run a Local AI Server for Your Whole Team on Your Mac in 2026 (Ollama Alternative, No Cloud)

Mohammed Ali Chherawalla — Sat, 24 Oct 2026 04:13:39 +0000

You have one Mac with plenty of memory and a team that wants to use local AI. Buying everyone a big machine is expensive, and sending your code and documents to a cloud API is the thing you were trying to avoid. Off Grid AI Desktop can turn that one Mac into a shared AI server. Its gateway has a Team mode that answers several people at once, gives each person their own API key, and keeps the Mac awake while it works. Every request stays on your network.

What you need

A Mac with Apple silicon (M1 or later) and macOS 13 or later. The Mac download is built for Apple silicon only.
Off Grid AI Desktop 0.0.55 or later. Team mode is in the current stable release, 0.0.55. Download Off Grid AI Desktop for Mac.
A chat model downloaded in the app, for example Qwen 3.5 9B.
The other computers on the same network as the Mac, such as your office Wi-Fi.
No account and no payment. The gateway and Team mode are part of the free app.

More unified memory means more people at once. With Auto, the app opens up to 4 requests at once on a Mac with 32 GB or less, up to 8 on a Mac with up to 96 GB, and up to 16 above that, such as a Mac Studio with 128 GB. Inside that cap, the real number comes from the memory left over after the model loads, so each request still has room for a long conversation.

Step 1: Open the Gateway

Open Off Grid AI Desktop and click Gateway in the sidebar.
The screen reads "One OpenAI-compatible API for every model on this Mac." Under Base URL you see http://127.0.0.1:7878/v1. That address is for the Mac itself.
For other computers, use your Mac's network address with the same port. Find it in System Settings > Wi-Fi > Details next to your network, under IP address. If it reads 192.168.1.20, your team uses http://192.168.1.20:7878/v1.

The gateway listens on port 7878 on every network interface, so other devices on the network can reach it as soon as the app runs. If the macOS firewall is on and asks whether to allow incoming connections for Off Grid AI Desktop, click Allow.

Step 2: Switch from Just me to Team

On the right side of the Gateway screen, find Clients.
The switch reads Just me and Team. Just me is the default: one request at a time, at full speed.
Click Team. Its tooltip reads "Several clients at once; extras wait their turn."
A Requests at once menu appears. Leave it on Auto, which shows the maximum your Mac allows in brackets, for example Auto (8). Or pick a lower number, such as 2 at once, if you want each answer to come back faster.

The change applies to the next request. If someone is mid-answer when you change a setting, the app waits for that answer to finish before it restarts the engine, for up to a minute, so nobody gets a reply cut in half.

When every slot is busy, new requests wait in line in the order they arrived. When the line is full, the gateway answers with HTTP 429 and tells the client to retry in 2 seconds, so a client backs off instead of hanging.

Step 3: Give each person an API key

By default the gateway does not ask for a key. On a shared network, turn keys on.

Under API keys, turn on the switch. Its label changes to API key required.
In the field that reads "New key name: phone, laptop, or service", type a name for one person or tool, for example priya-laptop.
Pick an Expiry: 7 days, 30 days, 90 days, Custom date and time or No expiry.
Click Create key.
Copy the key with the copy icon. The app says "Copy this key now. It will not be shown again."
Repeat for each person or tool. The app's own hint is "Use one key per client."

Each key in the list shows whether it is Active, Expired or Revoked, and when it was last used. If a laptop is lost, click the revoke icon next to that key. Requests with that key are refused from then on, and nobody else needs a new one. The keys are stored in the macOS keychain, not in a settings file.

Step 4: Connect your team's tools

The gateway speaks the OpenAI API, so most tools that let you set a base URL and an API key can use it. The Quick start section on the Gateway screen has copyable code for curl, Python, JavaScript and image generation. From another computer, the Python version looks like this:

from openai import OpenAI

client = OpenAI(base_url="http://192.168.1.20:7878/v1", api_key="THE_KEY_YOU_COPIED")
resp = client.chat.completions.create(
    model="local",
    messages=[{"role": "user", "content": "Summarise this changelog in two lines."}],
)
print(resp.choices[0].message.content)

A request that names local or no model runs on the chat model loaded on the Mac. The Endpoints list on the Gateway screen shows what else the server offers: chat with images, text to image, speech to text, text to speech, embeddings and the model list.

To use the Mac's models from the Off Grid AI phone app, follow the guide to use Off Grid AI Desktop from your phone and give the phone its own key.

Step 5: See who is using it

Open Settings and find AI activity. Every request is listed with its model and timing, and a request sent with a key shows that key's name beside its source. A request sent without a key shows no name.

For scripts and monitoring, GET /v1/gateway/status returns the mode, the requests-at-once setting, and for each part of the server how many requests are running, waiting and refused. It needs the same key as every other route.

Keep it serving

In Team mode the Mac stays awake while it is answering, and is allowed to sleep a minute after the last request ends. In Just me mode it does not hold the Mac awake.

To serve without the desktop window, start the app from Terminal in server-only mode, with Team mode for that run only:

"/Applications/Off Grid AI Desktop.app/Contents/MacOS/Off Grid AI Desktop" --server-only --gateway-mode team --gateway-clients auto

--gateway-clients takes auto or a number. These flags apply to that launch and do not change the settings you chose in the app. The environment variables OFFGRID_GATEWAY_MODE and OFFGRID_GATEWAY_CLIENTS do the same.

What this does not do

It does not make one answer faster. Each extra person shares the same GPU and memory bandwidth, so with four people typing at once, each answer streams slower than it would for one person.
Image generation runs one picture at a time even in Team mode. A second image request waits for the first.
It does not reach people outside your network. There is no relay or cloud tunnel. Remote staff need a VPN into your office network.
It does not stop a MacBook sleeping when you close the lid. For a shared server, leave the lid open or use a Mac mini or Mac Studio.
Without the API key switch on, anyone on the same network can use the models. Turn keys on before you share the address.

FAQ

Is Off Grid AI Desktop an alternative to Ollama for a team server?
Yes, for teams whose tools speak the OpenAI API. It serves chat, vision, image generation, speech and embeddings from one address, with per-person API keys and a request log built in.

How many people can use one Mac at once?
With Auto, up to 4 requests at once on a Mac with 32 GB or less, up to 8 up to 96 GB, and up to 16 above that, limited further by the memory your model leaves free. More people can send requests; the extra ones wait in line.

Does my team's data leave the office?
No. Requests go from each computer to the Mac on your network and the models run on the Mac. Nothing is sent to a cloud service.

Can I revoke one person's access?
Yes. Click the revoke icon next to their key under API keys. Their requests are refused and every other key keeps working.

Is Team mode free?
Yes. The gateway, Team mode and API keys are part of the free Off Grid AI Desktop app on Mac, with no account.

How to Find AI Models Your Android Phone Can Actually Run in 2026 (No Failed Downloads)

Mohammed Ali Chherawalla — Fri, 23 Oct 2026 15:51:45 +0000

You search Hugging Face for a model, wait for a 5 GB download on your Android phone, and then it will not load: too big for your RAM, or a format the app cannot run yet. Off Grid AI Mobile (OGAM) now filters that out before you tap download. Its model search lists only models your phone can load, shows one result per model instead of dozens of copies, and marks the files that only just fit.

What you need

An Android phone. What search shows depends on your phone's RAM, so an 8 GB phone and a 16 GB phone get different results.
OGAM 0.0.117, the current stable release. The search changes arrived in 0.0.116 and 0.0.117. Download OGAM for Android.
An internet connection while you search and download. After the download, the model runs on the phone with no connection.
No account and no payment. Search is part of the free app.

How search decides what your phone can run

The rule is the same one the app uses everywhere it offers a model, and it lives in the app's shared model code.

An Android phone gives one app well under its total RAM, and a model needs more memory than its file size while it runs: the weights, plus the cache that holds your conversation. The app's phone budget comes from a measurement in the code: a model with a 4.64 GB file needed 6.4 GB to load at a 4K context, about 1.4 times its size. From that:

A file under 36% of your RAM runs well.
A file between 36% and 50% of your RAM is marked Tight.
A file over 50% of your RAM is not offered.

Phone RAM	Runs well	Tight	Hidden
8 GB	under 2.9 GB	2.9 to 4 GB	over 4 GB
12 GB	under 4.3 GB	4.3 to 6 GB	over 6 GB
16 GB	under 5.8 GB	5.8 to 8 GB	over 8 GB

Search results from Hugging Face do not carry file sizes, so the list judges them by parameter count at 4-bit (about 0.6 GB per billion parameters). A 13B model on an 8 GB phone works out at about 7.8 GB and does not appear.

Size is not the only check. Some new models need a newer llama.cpp engine than the one inside your version of the app. Every upload of those models is left out of search, so you do not download something that fails to load. Files that only speed up another model (draft and MTP files) are not offered as models, and image generators stay on the Image tab instead of showing in chat search.

Step 1: Search

Open OGAM and tap Models in the bottom bar.
Stay on the Text tab and tap the Search models field.
Type a model or maker, for example qwen, gemma or lfm. Results appear half a second after you stop typing, or when you tap search on the keyboard.

Curated models that match come first. These are the models in the app's catalog, with the same fit rules as the picks at the top of the screen. Hugging Face results follow, up to 30 per search, and a model already in the curated list is not repeated.

Step 2: Read the cards

Each search result is one model, not one upload. When several people uploaded the same model, you see it once, led by the most trusted upload, with a +N uploads chip that counts the others.

Cards are short. Beyond the name, maker and parameter count, a card can show:

MoE with an active size, such as 3B active: a mixture-of-experts model. The active size sets its speed, so a large MoE model can reply like a much smaller one. It still needs RAM for its full size.
Newer: [model name]: a newer model replaces this one. Older models are listed last.
Research licence: the model's licence does not allow commercial use.

Uploads with safety training removed, such as "abliterated" or "uncensored" copies, are not shown.

Step 3: Narrow the list with filters

Next to the search field are two icons.

Tap the lightning icon to sort. Smart is the default. For You puts the best fit for your RAM first. Size, Downloads and Newest do what they say.
Tap the sliders icon to open the filters:
- Org: the maker. The list is OpenBMB, Qwen, Llama, Google, Microsoft, Mistral, DeepSeek, HuggingFace, NVIDIA and Liquid AI. Pick a maker without typing anything and the app lists that maker's curated models, then its Hugging Face models.
- Type: Text, Vision or Code. Vision searches models that read images.
- Source: LM Studio, Official, Verified or Community, by who uploaded the files.
- Size: < 1B, 1-3B, 3-8B or 8B+ parameters.
- Quant: a quantization, such as Q4_K_M or Q8_0.
Tap Clear to remove all filters.

A dot on an icon means a sort or filter is on.

Step 4: Pick a size and download

Tap a result. The detail screen shows the description, downloads and likes, then Sizes.
Sizes shows up to three files in plain words: Recommended (the best file for your phone, listed first), Smaller and Best.
Tap All files to see every file your phone can run, and Fewer sizes to go back. Files over your phone's limit are not in either list.
A file marked Tight loads, but leaves little room for other apps and a long chat. Pick the next size down if you switch apps often.
Tap download. When it finishes, the app makes it your active model and offers Chat now.

If you already have a GGUF file on the phone, tap the upload icon (Import local file) next to the filters instead.

What this does not do

It does not test speed. A model that fits can still be slow on an older chip. To tune speed after the download, use Use fastest settings on the Model Settings screen.
It does not check free storage while you search. The app checks space when the download starts and tells you how much it needs.
The fit check uses total RAM, not what is free right now. If other apps hold a lot of memory, a Tight model can still fail to load, and the app keeps your current model loaded when that happens.
Hugging Face results show at most 30 per search. Make the search more specific or add a filter to see others.
Search needs a connection. Downloaded models do not.

FAQ

Why can't I find a model I saw on Hugging Face?
It is too big for your phone's RAM, it needs a newer engine than your app version has, or it is an image generator, which lives on the Image tab. Uncensored and abliterated uploads are also hidden.

What does Tight mean on Android?
The file is between 36% and 50% of your phone's RAM. It can load, but there is little memory left, so a long chat or switching apps can cause trouble. A smaller size is steadier.

What is the biggest model an 8 GB Android phone can run?
Search offers files up to about 4 GB on an 8 GB phone, and files under about 2.9 GB run well. In practice that means models up to about 4B parameters at 4-bit run comfortably.

What does "3B active" mean on an MoE model?
The model is a mixture of experts and uses 3 billion parameters per word it writes, so it runs at about the speed of a 3B model. It still needs RAM for all of its parameters.

Why do some models show "+3 uploads"?
Three other people uploaded the same model. The app shows the most trusted upload and counts the rest, so the list does not fill with copies.

Can I search for Liquid AI models?
Yes. Open the filters, tap Org and pick Liquid AI, or type lfm in the search field.

How to Compare Local AI Models Side by Side on Your Mac in 2026 (One Question, Up to Six Models)

Mohammed Ali Chherawalla — Fri, 23 Oct 2026 12:53:34 +0000

You have a few local models on your Mac and you want to know which one writes the better answer for your work. Copying the same prompt into one chat after another is slow, and by the third model you have lost track of what the first one said. Off Grid AI Desktop now splits its chat into two to six panes. Each pane can run on its own model, and one question goes to all of them at once, so the answers arrive next to each other in one window.

What you need

A Mac with Apple silicon (M1 or later) and macOS 13 or later. The Mac download is built for Apple silicon only.
Off Grid AI Desktop 0.0.55 or later. Split view is in the current stable release, 0.0.55. Download Off Grid AI Desktop for Mac.
At least one chat model downloaded in the app, such as Qwen 3.5 9B.
For more than one model: a model server the app can reach. That can be Ollama or LM Studio running on the same Mac, a llama.cpp server, or another computer running Off Grid AI Desktop on your home network.
No account and no payment. Split view is part of the free app.

How the panes choose a model

Every pane has a Model for this pane picker at the top. It offers two kinds of choice:

Active model: the model loaded in Off Grid AI Desktop itself, the one you picked under Active models. Every pane set to this uses the same model.
A model from a saved server: each entry shows the model name and the server it comes from, so you can tell two copies of the same model apart.

So to compare different models, put one pane on the Active model and point the others at models served by Ollama, LM Studio or another machine. A pane that uses a server does not start the app's own engine on your Mac.

Step 1: Add a model server

Open Chat from the sidebar.
Click the gear icon at the top right (Settings) and choose the Remote tab.
Click Add server.
Type a Server name, for example "Ollama on this Mac".
Type the Address. Ollama on the same Mac answers at http://127.0.0.1:11434, and LM Studio at http://127.0.0.1:1234. The app adds /v1 when needed, and the hint under the field shows the exact address it will call.
If the server needs one, type its API key. The key is stored in the system credential store, not in a settings file.
Click Test connection. On success the status line reads "Connected in" followed by the time in milliseconds and the number of models found.
Under Text + Vision, search for and select a model.
Leave Use this server on and click Save.

Saving a new server with a chat model also moves the app's chat to that server. If you want the Active model to stay on your downloaded model, click the chip icon at the top right (Active models) and pick your local model again.

Repeat for each server. Every chat model a saved server offers shows up in the pane pickers.

Step 2: Split the chat

In Chat, click the split icon at the top right. Its tooltip reads Open tabs side by side.
The chat opens as panes, one for each chat tab you had open, up to six. With fewer than two tabs open, you still get two panes, and the extra pane starts on a new chat.
Use the minus and plus buttons next to the pane count to change the number of panes. Their tooltips read Remove the last pane and Add a pane. The count runs from 2 to 6.

The layout follows the count: two or three panes sit in one row, four make a two by two grid, and five or six make a grid of three columns and two rows. The split controls stay in the top right pane, so they do not move as you add or remove panes. Active models and Settings also show once, in that pane, because they apply to the whole app.

Step 3: Pick a model for each pane

Click Model for this pane in the first pane and choose Active model.
In each other pane, choose a model from a server.
The picker reads the servers' model lists each time you open it, so a model you add to a server in Settings shows up without closing the split.

The app remembers each pane's choice, so the next time you split, every pane starts on the model you gave it last time.

Step 4: Ask one question

With split view on, the panes share one input at the bottom of the window. This is the default, shown as One input for every pane in the top right controls.

Type your question in the shared input and send it.
Every pane sends the same question to its own model, and the answers stream in side by side.
Files you attach to the shared input go to every pane too, so you can compare how each model reads the same PDF or screenshot.
To stop, click Stop in the shared input. It stops every pane that is still answering.

For follow-ups that should go to one model only, switch to An input in each pane. Each pane then gets its own input, and what you type there goes to that pane alone. Switch back to send to all of them again.

Each pane keeps its own chat tabs, its own gallery and its own tasks. Press Command-T to start a new chat in the pane you are typing in; the other panes keep their chats.

Step 5: Go back to one chat

Click the same split icon again. Its tooltip now reads One chat. Your pane chats are saved like any other chat.

The window now narrows to 450 pixels wide, so a two pane comparison fits next to your editor or browser.

What this does not do

It does not score or rank the answers. You read them and decide.
It does not load several models inside Off Grid AI Desktop at once from the split view. Every pane on Active model uses the same in-app model. Different models come from servers.
Running several models on one Mac takes memory for each of them, and they share the same unified memory. If the models together are larger than your RAM, the Mac slows down or a model fails to load. Use smaller models, fewer panes, or a second computer for the bigger ones.
If a pane's server is turned off or removed, that pane does not quietly answer with another model. It tells you the server is off or gone and asks you to pick another model for that pane.
Split view is not saved across restarts. When you quit and reopen the app, Chat opens as one chat; the pane model choices are kept for the next split.

FAQ

How do I compare two local LLMs on my Mac?
Install Off Grid AI Desktop, add Ollama or LM Studio as a server under Settings > Remote, then click Open tabs side by side in Chat. Set one pane to Active model and the other to a model from the server, and send one question.

How many models can I compare at once?
Up to six panes. Each pane can use a different model, as long as your Mac or your servers can hold them in memory.

Does comparing models send my prompt to the cloud?
No. Each pane sends the question only to its own model: the one inside the app, or the server address you typed. If every server is on your Mac or your home network, the prompt never leaves them.

Is the split view free?
Yes. It is part of the free Off Grid AI Desktop app on Mac, with no account.

Can I ask a follow-up to just one model?
Yes. Switch the top right control to An input in each pane, and each pane gets its own input for its own follow-ups.

How to Run Qwen 3.5 and Gemma 4 Faster on Your Android Phone in 2026 (One-Tap Tuning, No Cloud)

Mohammed Ali Chherawalla — Fri, 23 Oct 2026 09:53:52 +0000

You have Qwen 3.5 or Gemma 4 running on your Android phone and you want replies to come faster. You do not want to guess whether the CPU, the GPU or the NPU is quicker on your chip, or how many threads to set. Off Grid AI Mobile (OGAM) now does that for you. Tap Use fastest settings, and the app runs the model you use on each part of your phone that can run it, measures how fast each one writes, and keeps the winner. The test takes 30 seconds or less, and everything runs on the phone.

What you need

An Android phone with 6 GB of RAM or more for the full test. Below 6 GB the app stays on the CPU, because the GPU and NPU buffers do not fit, and the button only resets the CPU settings.
A Snapdragon chip to compare backends. On a Snapdragon phone the app tests the CPU, the NPU (Hexagon) and the GPU (Adreno). On Google Tensor, Exynos and MediaTek Dimensity phones, the GPU backend is not available (it runs on Adreno only), so the app keeps the CPU and sets its threads and batch size for you.
OGAM 0.0.116 or later. The button arrived in 0.0.116 and is in the current stable release, 0.0.117. Download OGAM for Android.
A GGUF chat model already downloaded and selected, such as Qwen 3.5 2B, Qwen 3.5 4B or Gemma 4 E2B.
No account and no payment. This is part of the free app.

What the test does

When you tap the button, OGAM unloads your chat model, then loads it on each backend your phone can run, one at a time. On each one it reads a 64-token prompt and writes 16 tokens, then records the writing speed in tokens per second. The whole run has a 30-second budget. A backend that fails to load, runs out of time, or quietly falls back to the CPU counts as failed and cannot win.

The fastest backend that ran becomes your Inference Backend. If none of the accelerated backends ran, you stay on the CPU, which works on every Android phone. The app also sets the rest of the speed settings to match the winner:

CPU Threads and Batch Size go back to Auto. Auto threads skip your phone's efficiency cores. The code notes a measurement on a OnePlus CPH2707 where five threads ran slower than four, so a lone prime core next to a larger performance cluster is left out too.
On the CPU, Flash Attention is on and the KV Cache Type is q8_0.
On the NPU, batch size is 128, the value from llama.cpp's own Snapdragon guide. On the NPU and the GPU, flash attention is off and the cache is f16, because those are the settings proven safe on those backends today.
Speculative Decoding (MTP) is turned off.

One more speed rule runs without any tap. On Android, when you run on the CPU or the NPU, the app downloads the Q4_0 version of a model when its repository has one, because llama.cpp repacks Q4_0 best for ARM CPUs and the NPU runs it natively.

Step 1: Load the model you want to tune

Open OGAM and tap Models in the bottom bar.
Download or select the GGUF model you chat with, for example Qwen 3.5 4B or Gemma 4 E2B. The test runs on the model that is selected now. If no model is selected, there is nothing to measure.

Step 2: Open the speed settings

The Settings tab on Android. Model Settings is the first item.

Tap Settings in the bottom bar.
Tap Model Settings (System prompt, generation, and performance).
Open the Text Generation section.
Tap Advanced to show the speed controls.

You can also do this from inside a chat. Tap the sliders icon at the top right to open Chat Settings, open TEXT GENERATION, and tap Advanced. Both places change the same settings.

Step 3: Tap Use fastest settings

Tap Use fastest settings. It is the first control under the advanced sliders, above CPU Threads.
The button changes to Measuring... Cancel while the test runs. Keep the app open.
When the button reads Use fastest settings again, the test is done.
Look at Inference Backend below it. The selected option, CPU, GPU or NPU (Beta), is the one that measured fastest on your phone.

If you tap Measuring... Cancel, nothing is written and your settings stay as they were.

Step 4: Check the speed on a real reply

Go back to your chat and send a message. The model loads again on the new backend.
To see the speed, turn on Show Generation Details in the same Text Generation section. Each reply then shows the model, the context used and the speed under it.

Run the test again after you switch to a different model. A backend that wins for Qwen 3.5 may not win for Gemma 4 on the same phone.

Gemma 4 on LiteRT

OGAM also offers Gemma 4 E2B and E4B in Google's LiteRT format, listed as Gemma 4 LiteRT on the Models screen. These models use a different engine, so the Use fastest settings button does not appear for them. Under Advanced you get an Acceleration switch with GPU and CPU instead. Since 0.0.117, a LiteRT model on the GPU loads faster after the first time, and if a GPU model fails to load, the app moves it to the CPU after one try instead of retrying in a loop.

What this does not do

It does not check answer quality, only speed. The NPU (Beta) option itself warns that some models, Gemma among them, can fall back to the CPU or produce invalid output there. If replies look wrong after the test picks the NPU, set Inference Backend to GPU or CPU. If the NPU does not start at all, the app runs the model on the CPU.
It does not run on its own. The test runs only when you tap the button, and every setting stays editable afterwards. Tap the reset icon next to CPU Threads or Batch Size to return either one to Auto later.
It turns off Speculative Decoding (MTP). With MTP on, llama.rn reprocesses the whole prompt each turn, so the first word gets slower from the second message on. Turn it back on yourself if your model has an MTP head and you prefer it.
It does not test image generation or speech models. Those have their own settings.
It does not apply to remote servers. Chats that run on a computer over Wi-Fi use that computer's settings.

FAQ

Is the GPU or the NPU faster for local AI on Android?
It depends on the chip and the model, which is why OGAM measures it instead of guessing. On a Snapdragon phone with 6 GB or more, the app tests the CPU, the Hexagon NPU and the Adreno GPU on your model and keeps the one that writes fastest.

Why does my Pixel or Samsung Exynos phone stay on the CPU?
The GPU backend in llama.rn runs on Qualcomm Adreno GPUs only, not Mali, and the NPU backend needs a Snapdragon Hexagon NPU. On those phones the app tunes CPU threads and batch size instead.

How long does Use fastest settings take?
30 seconds or less. A backend that has not finished when the budget runs out counts as failed, and you can cancel at any time.

How many CPU threads should I use for llama.cpp on Android?
Leave CPU Threads on Auto. OGAM counts your performance cores and skips efficiency cores, and on phones with one prime core beside a larger performance cluster it leaves the prime core out, because more threads measured slower there.

Does tuning need internet?
No. The test loads and runs the model you already downloaded, on the phone. Nothing is sent anywhere.

Do I need to run it again after an app update?
Run it again when you change models or update the app. Each run replaces the last result.

How to Pick the Best Local AI Model on Your Android Phone in 2026 (Fast, Balanced or Best, No Cloud)

Mohammed Ali Chherawalla — Fri, 23 Oct 2026 06:52:02 +0000

You want a local AI model that runs well on your Android phone, and you do not want to read quantization tables to find it. Off Grid AI Mobile (OGAM) now opens the Models screen with three picks worked out for your phone's RAM: Fast, Balanced and Best. Tap one, it downloads, and it becomes the model your chats use. After the download, the model runs on your phone's CPU, GPU or NPU, and nothing you type is sent anywhere.

The same three picks are on the Image tab for image generation and on the Speech tab for voice transcription.

What you need

An Android phone. The picks scale with RAM, so a 6 GB phone and a 16 GB phone see different models.
OGAM 0.0.116 or later. The picks arrived in 0.0.116 and are in the current stable release, 0.0.117. Download OGAM for Android.
Wi-Fi for the download. Chat models are hundreds of megabytes to a few gigabytes. After that, no connection is needed.
No account and no payment. The model picks are part of the free app.

How OGAM chooses Fast, Balanced and Best

The picks come from one rule that lives in the app's shared model catalog. It reads your phone's total RAM and each model's download size, then keeps only the models that run well on your phone.

An Android phone does not give one app all of its RAM, and a model needs more memory than its file size while it runs (weights, plus the cache that holds your conversation). The app's phone budget is measured from that: a file under about 36% of your RAM runs well, a file between 36% and 50% is marked Tight, and anything larger is not offered. On a 12 GB phone, that is under 4.3 GB to run well and up to 6 GB at most.

From the models that run well:

Best is the highest-quality chat model in the catalog that fits.
Balanced is the next best.
Fast is a model that runs at the speed of a 4B model or faster, and faster than Balanced. A mixture-of-experts model counts by its active parameters, so a large model with a small active part can be the Fast pick.

Picks come from model makers people know, such as Qwen, Gemma, Llama, Mistral and Phi. A model that a newer one replaces is never picked, and neither is a model your app's engine is too old to load. As an example from the release notes, a 12 GB phone gets Qwen 3.5 0.8B, 2B and 4B.

On Android, when you run on the CPU or the NPU, the app downloads the Q4_0 version of a model if one exists, because it runs faster there.

Step 1: Open the picks

Open OGAM and tap Models in the bottom bar.
Stay on the Text tab. At the top, under For this phone, you see three small cards side by side: Fast, Balanced and Best.
Each card shows the model name and its download size.

Step 2: Download one and switch to it

Tap the download icon on the card you want.
The card shows the download progress as a percentage.
When the download finishes, the model becomes your active chat model. The card gets a highlighted border and a chat icon.
Tap the chat icon (Chat now) to open a chat with that model.

If you already downloaded a pick earlier, its card shows a circle icon instead. Tap it to switch to that model without downloading again.

If you download a model from the list below the picks instead, the app also makes it your active model when it finishes and shows "[model name] is ready" with Later and Chat now.

If you switch to a model that does not fit in the memory that is free right now, the app keeps your current model loaded, so you do not lose your chat.

Step 3: Pick an image model and a speech model

Tap the Image tab. The same three cards appear under For this phone. On this tab they take the place of the old RAM banner.
Tap the download icon on a card. When it finishes, the app says the model is ready, and the pick becomes the image model your chats use.
Tap the Speech tab. The picks here are multilingual speech-to-text models only, so they work whatever language you speak.

Image and speech models have no quality score in the catalog, so these picks go by size: Best is the largest that runs well, Fast the smallest with at least 2 billion parameters, and Balanced the one in between.

Step 4: Choose a size in plain words

When you tap a model in the list to open its details, the Sizes section shows up to three choices instead of quantization names:

Recommended: the file that suits your phone best. It is listed first.
Smaller: the smallest file below the recommended one, for when storage or memory is short.
Best: the largest file above the recommended one, for the highest quality your phone can run.

Every file is still there. Tap All files under the list to see them all, and Fewer sizes to go back. Files too big for your phone are hidden, and files that only just fit carry a Tight label.

Step 5: Get new models without an app update

New models are added to the catalog between app releases. To get them:

On the Models screen, tap the refresh icon at the top right, next to the downloads icon.
A sheet opens. It names each new model, for example "2 new models", and the new models appear in the list straight away.
If nothing changed, the sheet says Up to date. If your phone is offline, it says it could not check and asks you to check your connection.

The picks are worked out again with the new models included, so a better model for your phone can move into the Best slot.

What this does not do

It does not test speed on your phone. The picks are based on RAM, file size and the catalog's quality ranking, not a benchmark run on your device. To tune speed, use Use fastest settings on the Model Settings screen, which tests each backend your phone can run in 30 seconds or less.
It does not look at free storage when it picks. The app checks free space when the download starts and tells you how much it needs if space is short.
It does not pick from all of Hugging Face. The picks come from the curated catalog. You can still search Hugging Face from the Text tab, and the search hides files your phone cannot run.
A slot can be empty. On a phone with little RAM, there may be fewer than three models that run well, and the row shows only the slots it can fill.

FAQ

What is the best local AI model for an Android phone with 8 GB of RAM?
Open the Text tab in OGAM and look at the Best card: that is the highest-quality catalog model that runs well in 8 GB. On 8 GB, files under about 2.9 GB run well, so the picks are small models at 4-bit.

Which is faster on Android, Fast or Balanced?
Fast is always the quicker of the two. It runs at the speed of a 4B model or faster, while Balanced is the next-best quality model and can be larger.

Does picking a model need internet?
Only for the download and for checking for new models. After the download, chat, image generation and transcription with that model run on the phone with no connection.

Why does a model show Tight?
Its file is between about 36% and 50% of your phone's RAM. It can load, but there is little room left for other apps and a long conversation, so a smaller size may be steadier.

Can I still choose a different model?
Yes. The list below the picks has the full curated catalog, and you can search Hugging Face. The picks are a starting point, not a limit.

Why did my picks change?
The picks follow the catalog. After you tap the refresh icon and new models arrive, a newer or better model that fits your phone can take a slot.

How to Delete AI Chats and Generated Images From Your Phone in Off Grid AI in 2026

Mohammed Ali Chherawalla — Tue, 13 Oct 2026 10:23:32 +0000

An image experiment can leave several drafts in your phone's chat history and Gallery. OGAM (Off Grid AI Mobile) lets you delete an unwanted image, remove selected images, or delete a conversation and clean up its associated generated images.

Use the control for the material you want to remove. Clearing a missing Gallery record is different from deleting an image file, and deleting a chat is different from removing a downloaded AI model. The steps below help you keep the useful result while removing the drafts you no longer need.

Get OGAM for Android or iPhone | Mobile beta 114

What would you like to do with Off Grid AI?

Have a feature or use case you would like us to support? Tell us what you want to do and which device you use.

Write to support@getoffgridai.co, join our Slack community, or talk to us on Reddit.

Check the installed version and save what you need

This guide follows mobile beta 114, which includes cleanup for chats deleted from Home or a project, deletion of synced image files by their saved path, and a visible state for missing images. The source release can appear before a store update. Check your installed version if the controls or messages differ.

Open an image you want to keep and use its save or share control before deletion. On iPhone, this opens the share route for the file. On Android, the save route can copy it to Pictures/OffgridMobile when storage access is available. Confirm that the exported copy opens where you saved it.

The copy outside OGAM is separate from the Gallery item. Deleting the Gallery item does not promise to erase copies in Photos, Files, cloud backups, messages, or another app. Decide whether you want to remove a draft from OGAM or remove other copies too.

You do not need a loaded AI model to inspect a saved image or delete an existing record. Keep model downloads separate from this cleanup task.

Delete one image from Gallery

From Home, open Image Gallery. Find the unwanted image and open it. Check the picture before selecting the delete control, then read Delete Image and confirm Delete.

The app attempts to remove the saved image file and removes its Gallery record after success. If it reports Image was not deleted, the record stays visible. Read the error before trying again; a failed delete is not a completed cleanup.

This path also uses the saved location for images received through sync. Such an image may use a different location or file extension from one generated directly on the phone. You do not need to move it into a guessed folder first.

Start with an unwanted test image. Return to Gallery and check that the expected item is gone while the image you wanted to keep remains available.

Remove several drafts together

Use Gallery's selection control, or press and hold an image to enter selection mode. Select the unwanted drafts and check the displayed count. Use the delete control for the selected images and confirm Delete Images.

For example, if one useful illustration sits among six rejected drafts, open the useful one first and save it. Then select the six drafts. Check the selection count before confirming rather than using Select all and trying to remember what you wanted to keep.

If some files cannot be removed, OGAM reports Some images were not deleted and keeps their records. A completed bulk action can therefore have a partial result. Inspect the remaining Gallery entries before assuming the whole selection was removed.

Understand "Image is not on this device"

A Gallery record can exist while its image file is unavailable. The release shows Image is not on this device instead of treating a blank preview as a normal image. Its Remove control asks whether to remove that record from Gallery.

Removing this unavailable record is a list cleanup. It does not recover the image and is not proof that another device's copy was deleted. If you still need the image, look for a known exported copy or check the paired source device before removing the record.

When sync has already supplied a usable local copy, the app can point the Gallery record to that copy if the original path is gone. This helps an existing file remain usable; it does not recreate a file that no device still has.

What you see	Meaning	Useful next step
A normal preview	The file can be displayed	Save it or delete it deliberately
Image is not on this device	The current file is unavailable	Locate a retained copy, or remove the stale record
Image was not deleted	The file removal failed	Inspect the error and retained record
Some images were not deleted	A cleanup had a partial result	Check the remaining items

Delete a conversation and its generated images

On Home, find the conversation in Recent. Swipe its row to reveal the delete action. Select it, read Delete Conversation, and confirm the title before deleting. Project chat lists also have a delete action for the selected conversation; inspect its confirmation before proceeding.

The cleanup first cancels image generation owned by that chat and waits for the request to end. It then attempts to remove the generated image files associated with the conversation. This prevents that request from saving another result after cleanup has started.

If some image files cannot be removed, the conversation can still be deleted while those image records stay in Gallery. The app reports that result with Some images were not deleted. Open Gallery and inspect those records separately.

Only use this action when the conversation itself is no longer needed. If you want to keep its prompts and discussion, delete the unwanted Gallery images instead. Save any required answer text or exported files before removing the conversation.

Check paired devices and independent copies separately

If you use Pro sync, do not treat a phone cleanup as a guaranteed local-only operation. Conversation changes can be shared with trusted devices. Check the intended sync state and retain anything important before deleting a conversation you still use elsewhere.

At the same time, a deleted chat is not a global eraser for independent exports or backups. Inspect those locations separately when they are part of your cleanup. An offline paired device also cannot confirm a current transfer or deletion until the relevant connection and sync complete.

These are saved-data controls. They do not remove the downloaded text, image, or speech models that let you start another local task. Use the model or download manager only when you intend to remove model files too. For a computer, follow the separate Desktop chat and image cleanup guide.

Clean up one finished image experiment

Open OGAM, save the one result you need, then remove a small set of unwanted drafts. Read the outcome and check Gallery again. Once the result is clear, repeat the same process for the next finished conversation.

How to Organise Work and Personal Logins in Off Grid AI Vault in 2026

Mohammed Ali Chherawalla — Tue, 13 Oct 2026 10:22:44 +0000

Two accounts at the same website are easy to confuse. A login called "Portal" tells you little when one account is for work and the other is personal. OGAD (Off Grid AI Desktop) Pro lets you give Vault entries clear titles, place them in folders, mark favorites, and store several website URLs for one login.

Start with a few accounts you use each week. You can find the right entry on the computer and choose it from the paired browser extension without moving your password collection into a cloud password account.

Get OGAD | Desktop beta 122

What would you like to do with Off Grid AI?

Have a feature or use case you would like us to support? Tell us what you want to do and which device you use.

Write to support@getoffgridai.co, join our Slack community, or talk to us on Reddit.

Prepare the Vault before sorting it

Use desktop beta 122 with Pro active. Open Vault and unlock it. If this is your first use, complete Vault setup and keep its recovery information somewhere you can access if the computer is unavailable. For browser autofill, use the early-access extension paired with this Desktop and keep Vault unlocked.

These organization controls do not need a chat model. You are editing saved records, not asking AI to inspect passwords. Do not paste passwords, recovery phrases, or exported password files into a chat to sort them.

The illustration above shows an example login. Your entry names and available fields will depend on your saved items and installed version. The procedure below uses the beta's edit form.

Give each account a useful title

Suppose you have a work account and a personal account at the same design service, plus a client portal. Use titles such as "Design service - work," "Design service - personal," and "Northwind portal - project account."

The title should identify the purpose. Keep the actual account identity in Username / Email. A clear title helps you choose, but it does not change which account the password belongs to.

Select a login in Vault.
Choose Edit.
Check Title, username, and the saved website address.
Change the title to identify the account's purpose.
Select Save, then reopen the entry to check it.

Do one entry first. If it is correct, repeat the same naming rule for the related accounts. Renaming a Vault entry does not rename your account at the service.

Put related entries in a folder

The edit form has a Folder field. Type a useful folder name or choose an existing suggestion. For example, use Work/Email, Work/Client portals, and Personal. Keep the set small enough that you can choose a folder without thinking about a complex filing system.

Save the entry, then use Folder filter above the list to choose that folder. All folders restores the full list. No folder helps find entries that have not been assigned yet.

A folder is an organization label. It does not create a second encrypted Vault, a different unlock secret, or separate permissions for a colleague. Only place accounts in this Vault when they belong in your intended local password collection.

If a login seems to disappear after editing, check the folder filter and search text before recreating it. An active filter can hide a valid entry. Clearing the filter is safer than making a duplicate with another password.

Mark the few logins you use most

Open the entry's edit form and enable Favorite, then Save. Favorites are placed ahead of other entries in the relevant list. Use this for the accounts you open frequently rather than marking every entry.

For a normal workday, you might favor your work email and current client portal while leaving occasional accounts in their folders. The folder answers "Which group?" The favorite answers "Which do I use most?"

Remove the favorite mark when a project ends. That keeps the top of the list useful without deleting a login you still need for records or occasional access.

Add another website address to the same login

Some accounts use more than one legitimate sign-in address. In the edit form, check Primary URL, then choose Add URL for each additional address. Save the entry and review the grouped websites in its detail view.

Only add an address where those same credentials are valid. A second account with a different username or password needs its own login entry, even if both accounts use the same website. Adding a URL is not a way to grant that account access to a new service.

Situation	Useful record choice
Same account and password, another valid sign-in address	Add a URL to the existing login
Work and personal accounts on one website	Keep separate login entries
Old address no longer used	Remove that URL after checking the replacement
Another project uses a different account	Create or retain its separate entry

Check the actual address in the browser before filling. A similar-looking page title does not establish that the page is the service you intended.

Choose the right login in the browser

Open the service's normal sign-in page in the paired browser. Use the key control in the field or open the matching Vault entry in the panel. When several saved logins match the site, select the intended account. Check the filled username before submitting the form.

The extension can also offer to save a login you typed when Desktop confirms that it is not already saved. Complete the sign-in normally, review any save offer, and inspect the resulting record. A multi-page form can make the username less obvious, so check that field rather than trusting the title alone.

If no entry appears, confirm that OGAD is running, the pair is connected, and Vault is unlocked. Then check the saved URLs. The website still receives the credentials you submit to it; local Vault storage does not remove the website from the sign-in process.

Review imported entries in small groups

After importing passwords and notes, start with one folder's worth of accounts. Check the title, username, URLs, and any notes. Test an important login before removing its old source record. Keep exported CSV files out of shared folders because they can contain readable secrets.

Use No folder to find unsorted entries, assign a folder, then mark only the frequent ones as favorites. Do not merge entries just because their titles match. Two imported records can refer to different accounts or contain different current passwords.

You can keep login organization separate from password quality. Use Vault's password audit for weak or reused passwords, and change a password at the actual website before updating the saved value. Editing a stored password alone does not change the website's password.

Organise the accounts you will use tomorrow

Open OGAD and choose a small group of work and personal logins. Give each a clear title, assign its folder, and check one browser sign-in. Once that group is correct, apply the same simple rules to the rest of your Vault.

DEV Community: Mohammed Ali Chherawalla

How to Speed Up Local AI Replies With MTP on Your Android Phone in 2026 (No Setup)

What you need

How MTP makes replies faster

Step 1: Find a model with the MTP badge

Step 2: Turn on Speculative Decoding (MTP)

Step 3: Reload the model and chat

The in-chat suggestion

Draft Tokens

What this does not do

FAQ

Related

How to Ask Questions About a 500-Page PDF on Your Mac in 2026 (Page Citations, No Cloud)

What you need

Step 1: Attach the PDF

Step 2: Watch it read the file

Step 3: Ask your question

Step 4: Check the answer against the page

Work with several documents in one chat

Use a project for documents you ask about often

What this does not do

Where the document goes

FAQ

Related

How to Retry, Edit and Queue Questions to Local AI on Your Android Phone in 2026 (Every Answer Kept)

What you need

How the message menu works

Step 1: Get a new answer without losing the old one

Step 2: Fix your question and ask again

Step 3: Correct an answer by hand

Which version the model reads

Step 4: Queue your next questions while the model writes

Step 5: Stop without losing what you queued

Switching apps mid-reply

What this does not do

FAQ

Related

How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Your Mac in 2026

What you need

Step 1: Let the server accept calls from your network

Step 2: Add the server in Off Grid AI Desktop

Step 3: Pick the models and save

Step 4: Chat on the remote model

Run several servers

When the server is another Off Grid AI Desktop

What this does not do

FAQ

Related

A Private Microsoft Recall Alternative on Your Mac in 2026 (Local AI Reviews Your Day Overnight)

What you need

How it works

Step 1: Turn on capture and choose Overnight

Step 2: Change the hours in Settings

Step 3: Leave the Mac on power and the app running

Step 4: Read your day in the morning

Check what happened overnight

What this does not do

FAQ

Related

How to Run a Local AI Server for Your Whole Team on Your Mac in 2026 (Ollama Alternative, No Cloud)

What you need

Step 1: Open the Gateway

Step 2: Switch from Just me to Team

Step 3: Give each person an API key

Step 4: Connect your team's tools

Step 5: See who is using it

Keep it serving

What this does not do

FAQ

Related

How to Find AI Models Your Android Phone Can Actually Run in 2026 (No Failed Downloads)

What you need

How search decides what your phone can run

Step 1: Search

Step 2: Read the cards

Step 3: Narrow the list with filters

Step 4: Pick a size and download

What this does not do

FAQ

Related