A new model lands on Hugging Face, you pull the GGUF, and your Linux box refuses to load it because the llama.cpp inside your app is two weeks old. The usual fix is to clone llama.cpp, rebuild with -DGGML_VULKAN=ON or -DGGML_CUDA=ON, and point your tools at the new llama-server. Off Grid AI Desktop skips that. It downloads newer llama.cpp, stable-diffusion.cpp and whisper.cpp builds in the background, checks that each one starts on your machine, and uses it the next time a model loads. On a machine with an NVIDIA GPU, the CUDA builds and the CUDA libraries update the same way.
To show how far this moves: the 0.0.55 AppImage and .deb ship with llama.cpp b11819 and stable-diffusion.cpp master-987. On 29 October 2026 the Linux engine index already offered llama.cpp b11946 and stable-diffusion.cpp master-1017, so a machine on 0.0.55 picks those up with no new package.
What you need
- An x64 Linux machine. The documented target is Ubuntu 24.04 or newer. The release ships an AppImage (
off-grid-ai-0.0.55.AppImage) and an amd64 package (off-grid-ai_0.0.55_amd64.deb); there is no Linux ARM build. - Off Grid AI Desktop 0.0.55 or later, the current stable release. Download Off Grid AI Desktop for Linux.
- An internet connection when an update downloads. Chat, images and transcription still run offline once the engines and models are on disk.
- For CUDA: an NVIDIA GPU with the NVIDIA kernel driver loaded, plus the app's optional NVIDIA download. Without one, the app updates its Vulkan and CPU builds instead.
- No account and no payment. Engine updates are part of the free app.
Which engines update on Linux
Each engine comes in more than one build, and the app only installs the builds your machine can use:
| Engine | What it runs | Linux builds |
|---|---|---|
| llama.cpp | Chat and vision models | Vulkan, CPU, CUDA |
| Prism (a llama.cpp fork) | Bonsai models | Vulkan, CPU, CUDA |
| stable-diffusion.cpp | Image generation | Vulkan, CUDA |
| whisper.cpp | Transcription | Vulkan, CPU, CUDA |
The Vulkan builds run on AMD, Intel and NVIDIA graphics. The CUDA builds install only when the app finds an NVIDIA device node (/dev/nvidia0 or similar), which the proprietary NVIDIA driver creates and the open-source nouveau driver does not.
A nightly job builds each new upstream release, checks it and publishes it, so a new llama.cpp tag reaches the index without a new app version.
Step 1: Find the engine settings
- Open Off Grid AI Desktop and click Settings in the sidebar.
- Open the Engines card. Its summary reads "Keep chat, image and transcription engines current, and manage GPU components."
- At the top is the NVIDIA panel, Use your NVIDIA GPU. Without an NVIDIA driver it says "No NVIDIA driver was found." Below it is the Keep engines up to date switch.
Step 2: Leave Keep engines up to date on
The switch is on by default. With it on, the app checks the engine index 1 minute after launch and then every 6 hours. For each engine it is missing a newer build of, it:
- Downloads only the files it does not already have.
- Checks each file's size and SHA-256 hash, and checks that the index itself carries a valid Ed25519 signature.
- Starts the new engine once, with a 30-second limit, to prove it runs on your machine.
- Makes it the first engine the app looks for.
The line under the switch tells you where it is: "Checking for newer engines...", a download count, "Checking the new engine starts...", and then "Engines are up to date. A new engine is used the next time a model loads."
Under that line, the app lists the version of each engine it installed, for example Chat: b11946 · Images: master-1017-be48649 · Transcription: v1.9.5. When the GPU and CUDA builds of one engine are on different versions, each gets its own entry, such as Chat (cuda): b11946.
Step 3: Check for an update now
You do not have to wait 6 hours. Under the version list, click the round-arrow button, Check for engine updates.
Then reload your chat model: switch to another model and back, or restart the app. A running model keeps its engine until then, so nothing changes mid-answer.
Step 4: Let the CUDA pack update itself
On a machine with an NVIDIA GPU, the large part is the CUDA pack: the CUDA builds of each engine plus the NVIDIA runtime libraries they need, including the cuBLAS and cuDNN libraries for GPU speech. On 29 October 2026 the current Linux pack, 2026-10-28-auto-r8, was about 7 GB. You do not need the CUDA toolkit installed; the pack brings its own libraries.
- If you have never installed it, the Use your NVIDIA GPU panel offers it with a Download button. You choose whether to download it.
- If you already have an older pack, the app downloads the new one in the background. Your current pack keeps CUDA running until the new one is fully in, so you never lose GPU support halfway through.
- While a download runs, the panel shows its progress. The button there becomes Pause, then Resume. Pausing also stops the check, unpack and install steps, so a paused pack does not keep using the disk.
- If the download or unpack fails, the button reads Try again. A retry after a failed unpack skips the part already downloaded.
- When the panel reads "Installed. Restart the app to use it.", click Restart app.
If you serve kept models from the Gateway
If you keep several models loaded through the Gateway, a new llama.cpp does not restart them while they answer requests. A card appears instead, "Engine update ready for kept models", with a Restart models button. They keep serving on the old build until you click it.
When an update fails
If a download breaks off or a new engine does not start, the status line turns red and says why, for example "The new llama engine did not start" followed by the error. The app keeps using the engine it had and tries again at the next check. The previous release and the engines that shipped inside the app stay on disk as fallbacks.
What this does not do
- It does not update the app itself. New screens and settings still come with app releases; this covers the engines that run the models. The AppImage and
.debkeep their own update path. - It does not let you pick a specific llama.cpp build or pass your own build flags. You get the newest build the index publishes for Linux x64. Turning the switch off keeps the engines on the version this app release shipped with, and the status line says so.
- It does not add CUDA to a machine without an NVIDIA GPU, or to one running nouveau. AMD and Intel graphics run the Vulkan build.
- It does not install GPU drivers. Vulkan needs a working Vulkan driver from your distribution, and CUDA needs the NVIDIA driver.
FAQ
How do I update llama.cpp in Off Grid AI Desktop on Linux?
You do not need to do anything. With Keep engines up to date on, the app installs newer builds in the background. To check at once, open Settings > Engines and click Check for engine updates.
How do I know which llama.cpp version I am running?
Open Settings > Engines. The line under the status lists each installed engine and its version, such as Chat: b11946.
Do I have to rebuild llama.cpp myself to run a brand-new model?
No. If the model needs a newer llama.cpp than your app shipped with, the engine update brings a prebuilt Vulkan, CPU or CUDA build. Reload the model once the status reads "Engines are up to date."
Does the update include the CUDA build for my NVIDIA GPU?
Yes, once the NVIDIA download is installed. The CUDA builds and the NVIDIA libraries update in the background, and the old pack keeps working until the new one is in. Click Restart app when the panel asks.
Is it safe to let the app install new engine builds?
Each update comes from an index signed with the publishing key, and every file is checked against its SHA-256 hash before use. A new engine must start in a 30-second test before the app uses it, and the previous build stays on disk as a fallback.
Top comments (0)