OGAD (Off Grid AI Desktop) includes catalog entries for Gemma 4 E4B and Gemma 4 12B. You can download a supported model package, select it in the app, and run local chat on your computer.
Choose based on the resources available and the task you need to do. A catalog download size is not the same as the RAM needed while the model is running, and a larger name is not a measured promise of a better answer.
Download OGAD | Desktop beta 122 release notes
What would you like to do with Off Grid AI?
Have a feature or use case you would like us to support? Tell us what you want to do and which device you use.
Write to support@getoffgridai.co, join our Slack community, or talk to us on Reddit.
Compare the catalog packages first
The current catalog uses Q4_K_M primary weights for these entries. It also identifies companion vision-projector files. Check the current download details in the app because a complete package can contain more than one file.
| Catalog entry | Approximate primary weight size | Approximate vision-projector size |
|---|---|---|
| Gemma 4 E4B | 4.98 GB | 990 MB |
| Gemma 4 12B | 7.12 GB | 175 MB |
These are catalog file sizes, not benchmark results or total runtime-memory requirements. Allow additional disk space for companion downloads, temporary files, and future model work. Other applications also need memory while you use the model.
The desktop beta 122 release is the release scope for this guide. Use the app's current compatibility and download state when choosing a model for your computer.
Install the model through Models
- Install or update OGAD.
- Open Models and the text-model catalog.
- Find Gemma 4 E4B or Gemma 4 12B.
- Check the displayed resource guidance and available disk space.
- Choose Download and wait for all required files to finish.
- Choose Use and confirm that the intended model becomes active.
Download one candidate first. Keeping both is reasonable if you have the space and want to compare them, but two simultaneous downloads do not make the first useful result arrive sooner.
If you already have compatible files, use the Storage scan-folder workflow to reuse them. Check the discovered entry and its companion files rather than assuming any file with “Gemma” in the name is the package you intended.
Test a small text task
Open a new chat and confirm the active model. Ask it to summarize a short paragraph you provide, identify explicit requirements, or draft a simple checklist. Keep the first request small enough that you can judge the answer yourself.
For example, provide five project facts and ask for a three-step plan that does not add new deadlines. Check whether the response preserves the facts and marks missing information instead of filling gaps.
Then try the work you actually care about. A model that answers a greeting has loaded successfully, but that says little about its ability to follow your document-review or coding instructions.
Check vision separately
If you want to ask about an image, confirm that the app identifies the selected model as vision-capable and that the matching projector is available. Attach a simple image with a result you can verify.
Ask for visible facts first: objects, labels, or the broad structure of a chart. Do not begin with a question that requires exact small text or a numerical conclusion you cannot check.
A successful text reply does not validate the image path. If an image is rejected or ignored, inspect the selected model, projector, attachment, and error message before you change unrelated settings.
Understand the additional companion download
This desktop release includes model-specific companion handling for the supported Gemma entries, including multi-token prediction files used by the runtime. Keep the companion paired with the correct primary model instead of mixing files from different entries.
Multi-token prediction is a runtime technique, not an automatic speed guarantee for your computer. Actual behavior depends on hardware, context, the model, and the job. This guide does not claim measured tokens per second or a percentage improvement.
Let the app manage the supported package where possible. If a companion download fails, use the download state and retry controls rather than substituting an unrelated file with a similar name.
Fix loading and download problems
| Symptom | Check first |
|---|---|
| Download does not finish | Network, free space, and download error |
| Model cannot load | Available memory and complete required files |
| Replies use the wrong model | Active selection, including any remote endpoint |
| Vision fails while text works | Matching projector and attachment support |
| Work slows with long conversations | Context size, memory pressure, and other active work |
Close unnecessary heavy applications and shorten the test context when diagnosing memory pressure. An external drive can provide storage capacity, but it does not add RAM.
Try the model's matching MTP option
After a normal reply works, inspect the model's available MTP companion in Models. Use the E4B companion for E4B and the 12B companion for 12B. Finish its download, then open Settings > Text > Advanced > Speculative decoding and select MTP when the app offers it.
Leave the draft-token limit on Auto for the first comparison. In desktop beta 122, that uses two proposed tokens per MTP step. More draft tokens can create more rejected work. Keep the main model, context, and prompt unchanged when comparing Off and MTP.
Use a real task such as rewriting a saved project update. Check both the time and the retained facts. The companion is a performance option for a compatible main model, not a replacement for the main model or its vision file.
Use local chat after setup
Once the app and required model files are downloaded, local chat with local input can run without internet. Live websites, connected email, cloud documents, and other online tools still need their own connections. Keep those dependencies separate from the model itself.
Local chat and model management do not by themselves require the Pro assistant workflows. Features such as God, connected work tools, and other Pro capabilities have their own requirements.
Download OGAD, try the Gemma entry that fits your resources, and test it on one real task. Compare answer quality and responsiveness on your own computer before deciding which model to keep active.
Top comments (0)