Mohammed Ali Chherawalla

Posted on Oct 29 Originally published at getoffgridai.co

How to Compare Local AI Models Side by Side on Linux in 2026 (One Question, Up to Six Models)

#ai #linux #llm #ollama

You run Ollama, LM Studio or a llama.cpp server on your Linux box, you have pulled a handful of models, and you want to know which one writes the best answer for your own prompts. Off Grid AI Desktop splits its chat into two to six panes. Each pane runs on its own model, one question goes to all of them, and the answers stream in next to each other in one window.

What you need

A Linux PC with an x64 processor. Off Grid AI Desktop for Linux ships as an x64 AppImage and an amd64 .deb, documented for Ubuntu 24.04 or newer. There is no Linux ARM package.
Off Grid AI Desktop 0.0.55 or later. Split view is in the current stable release, 0.0.55. Download Off Grid AI Desktop for Linux.
At least one chat model downloaded in the app, such as Qwen 3.5 9B. With an NVIDIA GPU and the app's optional NVIDIA download, it runs on CUDA. Without one, it runs on Vulkan or the CPU.
For more than one model: a model server the app can reach. That can be Ollama or LM Studio on the same machine, a llama.cpp llama-server, or another computer on your network running Off Grid AI Desktop.
No account and no payment. Split view is part of the free app.

How each pane chooses its model

Every pane has a Model for this pane picker at the top. It offers two kinds of choice:

Active model: the model loaded inside Off Grid AI Desktop, the one you picked under Active models. Every pane set to this uses that same model.
A model from a saved server. Each entry shows the model name and the server name.

To compare different models, put one pane on Active model and point the others at models from Ollama, LM Studio or another machine.

Step 1: Add a model server

Start your server. On most distributions the Ollama installer sets up a systemd service, so systemctl status ollama shows whether it is running. In LM Studio, start its local server. For llama.cpp, run llama-server with your model.
In Off Grid AI Desktop, open Chat from the sidebar.
Click the gear icon at the top right (Settings) and choose the Remote tab.
Click Add server.
Type a Server name, for example "Ollama on this PC".
Type the Address. Ollama on the same machine answers at http://127.0.0.1:11434, LM Studio at http://127.0.0.1:1234, and llama-server at http://127.0.0.1:8080 unless you set another port. The app adds /v1 when needed, and the hint under the field shows the exact address it will call.
If the server needs one, type its key in API key (optional). The app encrypts the key with your desktop's keyring through the Secret Service, such as GNOME Keyring or KWallet, and stores only the encrypted bytes.
Click Test connection. On success the status line reads "Connected in" followed by the time in milliseconds and the number of models found.
Under Text + Vision, search for and select a model.
Leave Use this server on and click Save.

Saving a new server with a chat model also moves the app's chat to that server. To keep Active model on your downloaded model, click the chip icon at the top right (Active models) and pick your local model again.

Every chat model a saved server offers appears in the pane pickers.

Step 2: Split the chat

In Chat, click the split icon at the top right. Its tooltip reads Open tabs side by side.
The chat opens as panes, one for each chat tab you had open, up to six. With fewer than two tabs open you still get two panes, and the extra pane starts on a new chat.
Use the minus and plus buttons next to the pane count to change the number of panes. Their tooltips read Remove the last pane and Add a pane. The count runs from 2 to 6.

Two or three panes sit in one row, four make a two by two grid, and five or six make three columns and two rows. Active models and Settings show once, in the top right pane, because they apply to the whole app.

Step 3: Pick a model for each pane

Click Model for this pane in the first pane and choose Active model.
In each other pane, choose a model from a server. The list is searchable, which helps when ollama list runs to a dozen tags.
The picker rereads each server's model list every time you open it, so a model you pull with ollama pull while the split is open appears without closing it.

The app remembers each pane's choice. The next time you split, every pane starts on the model you gave it last time. Split view itself does not survive a restart: after you reopen the app, Chat opens as one chat, and your pane choices wait for the next split.

Step 4: Ask one question

With split view on, the panes share one input at the bottom of the window. This is the default, shown as One input for every pane in the top right controls.

Type your question in the shared input and send it.
Every pane sends the same question to its own model, and the answers stream in side by side.
Files you attach to the shared input go to every pane too, so you can see how each model reads the same PDF or screenshot.
To stop, click Stop in the shared input. It stops every pane that is still answering.

For a follow-up to one model only, switch to An input in each pane. Each pane then gets its own input.

Each pane keeps its own chat tabs, its own gallery and its own tasks. Press Ctrl+T to start a new chat in the pane you are typing in; the other panes keep their chats.

Models on one machine share its memory. If the app's model and your server models together are larger than your GPU's VRAM, the overflow moves to system RAM and that pane answers more slowly. Pick smaller quantizations or fewer panes, or put the bigger models on a second machine. If a pane's server is turned off or removed, that pane tells you so and asks you to pick another model, rather than answering with a different one.

Step 5: Go back to one chat

Click the same split icon again. Its tooltip now reads One chat. Your pane chats are saved like any other chat. The window narrows to 450 pixels wide, so it fits beside your editor when you tile windows. On GNOME, Super+Left and Super+Right snap it to half the screen.

FAQ

How do I compare two local LLMs on Linux?
Add Ollama, LM Studio or llama-server as a server under Settings > Remote in Off Grid AI Desktop, then click Open tabs side by side in Chat. Set one pane to Active model, the other to a server model, and send one question.

How many models can I compare at once?
Up to six. Each pane can use a different model, as long as your machine or servers can hold them in memory.

Can I compare Ollama models with each other?
Yes. Add Ollama once as a server, then give each pane a different Ollama model from Model for this pane. If they do not all fit in VRAM, Ollama unloads one model to load the next, which adds a pause before that pane answers.

Can a pane use a model on another Linux machine?
Yes. Add that machine's address under Remote, for example http://192.168.1.20:11434. Ollama listens only on 127.0.0.1 by default, so on the other machine run sudo systemctl edit ollama, add Environment="OLLAMA_HOST=0.0.0.0" under [Service], restart the service, and allow port 11434 through your firewall, for example with sudo ufw allow 11434/tcp.

Does comparing models send my prompt to the cloud?
No. Each pane sends the question only to its own model: the one inside the app, or the server address you typed. If every server is on your machine or home network, the prompt stays there.

Is split view free on Linux?
Yes. It is part of the free Off Grid AI Desktop app, with no account.

DEV Community