The GPU in a mid-range laptop can run a 7-billion-parameter model fast enough to feel like a chat. That power sits unused while you pay a monthly fee to send every question to a remote server. Off Grid AI Desktop is a free, open-source app that runs Alibaba's Qwen models directly on your Mac or PC.
Free, open-source, AGPL-3.0, runs offline.
The built-in model browser. Download text, vision, image, and voice models from Hugging Face, sized to your machine.
Why Qwen
Qwen is Alibaba's open-weight model family. The weights are public, so you download them once and run them on your own machine.
Qwen has two strengths worth naming. It is strong at coding tasks, which makes it useful for explaining functions, sketching scripts, and reviewing snippets. It is also multilingual, trained across many languages, so it handles non-English text better than many open models its size.
You run it through the Models browser in Off Grid AI Desktop. Find a Qwen GGUF on Hugging Face, download it, and chat. Nothing leaves your disk.
What You Need
Qwen ships in many sizes, from tiny to large. Pick the size your hardware can hold.
| Tier | Mac | Windows / PC | Runs |
|---|---|---|---|
| Minimum | M1, 8 GB unified memory | 8 GB RAM, integrated GPU or CPU | Qwen 1.5B / 3B class, quantized |
| Recommended | M2 or M3, 16 GB+ | 16 GB RAM, NVIDIA GPU with 8 GB VRAM | Qwen 7B class |
| Comfortable | M3 Max or M4, 32 GB+ | 32 GB RAM, 12 GB+ VRAM | Qwen 14B and up, longer context |
You need macOS 13 or later, or Windows 10/11. Keep 5 to 20 GB of free disk per model. Quantized GGUF files shrink the download, which is how a large model fits in consumer memory.
What Off Grid AI Desktop Can Do
Chat is the core, but Qwen plugs into more than a text box.
- Chat with Qwen offline. Tune temperature and context window per conversation. A private alternative to cloud chat.
- Browse and download models. The built-in Hugging Face browser pulls any compatible GGUF. Swap Qwen sizes in seconds.
- Render code and artifacts. When Qwen writes HTML, SVG, Mermaid, or React, it renders live in a sandboxed preview with no network access. Good for the coding work Qwen is built for.
- Chat with your documents. Upload PDFs, notes, or a whole folder and ask questions with cited sources, all on-device.
- Use tools. An agentic loop lets Qwen call built-in local tools like a calculator and date functions.
Which Qwen to Use
Bigger is not always better. A model that fits in memory beats one that spills to disk.
| Your hardware | Recommended Qwen | What to expect |
|---|---|---|
| 8 GB Mac or PC | Qwen 1.5B / 3B, Q4 quant | Quick answers, decent for short coding help |
| 16 GB, decent GPU | Qwen 7B class | Solid coding and multilingual quality, fast enough to chat |
| 32 GB+ with strong GPU | Qwen 14B and up | Best reasoning, longer files, slower first token |
For coding and multiple languages, the 7B class on 16 GB hits a good balance. Move up only if you have the memory to spare.
How Hardware Acceleration Works
Off Grid AI Desktop runs Qwen through a bundled llama.cpp server on your machine. It uses your GPU where it can.
On a Mac, that path is Metal. Apple Silicon shares one memory pool between CPU and GPU, so the GPU reaches the model without copying it across a bus. A 16 GB Mac runs a 7B model comfortably.
On Windows, the app uses CUDA on NVIDIA cards or Vulkan on other GPUs. With no supported GPU, Qwen runs on the CPU. Slower, but it works.
Quantization makes the rest possible. Full Qwen weights are 16-bit floats. Quantized GGUF stores them in 4 or 8 bits, cutting memory roughly in half. The accuracy cost is small. The win is that a model that would not fit now fits.
Keeping It Fast
Match the model to your memory first. If the app offloads layers to the CPU, you went too big. Drop a size.
Use Q4 for general chat and Q8 when you want top quality and have the memory. The larger quant is slower and heavier.
Trim the context window to the task. A long context slows every token and eats RAM. Start a new chat per topic so the model is not re-reading a long history each turn.
Privacy: Stronger Than Cloud Chat
A cloud chat service logs your prompts and may use them to train future models. Code you paste, documents you summarize, all of it lands on a server.
Off Grid AI Desktop keeps Qwen on your disk. No account, no telemetry, no API key. The code is AGPL-3.0, so you can read it and verify nothing leaves. Disconnect from the internet and Qwen keeps answering.
Getting Started
- Open the repo at github.com/off-grid-ai/desktop.
- Download the build for your OS, or clone and build from source.
- Install and launch the app.
- Open the Models browser, search for Qwen, and download a GGUF that fits your hardware.
- Open chat, select the model, and start typing.
git clone https://github.com/off-grid-ai/desktop
cd desktop
npm install
npm run dev
What's Coming
- Cross-device sync, so chats and memory move between your machines, encrypted.
- More bundled models as new open-weight releases ship.
- Unified search across chats, documents, and captured work.
The Off Grid AI Desktop chat, running a local model fully on-device.
FAQ
Q: Is it really free?
Yes. The app is free and open-source under AGPL-3.0. No paid unlock to run Qwen.
Q: Does it work fully offline?
Yes. After a model downloads, you can drop the network and keep chatting.
Q: Which Qwen models can I run?
Any compatible Qwen GGUF from Hugging Face, downloaded through the built-in Models browser. Pick a size that fits your RAM.
Q: What is Qwen good at?
Coding tasks and multilingual text, in particular. It handles non-English well and explains and writes code.
Q: How much RAM do I need?
8 GB runs a small Qwen. 16 GB is comfortable for the 7B class. 32 GB lets you run 14B and larger.
Q: Mac or Windows?
Both. Mac uses Metal on Apple Silicon. Windows uses CUDA or Vulkan, with a CPU fallback.
Run Qwen on your own hardware, completely offline.
Top comments (0)