At home, for yourself, pick LM Studio or Ollama; for an API on a server, pick Ollama for one user and vLLM for many. The choice changes the memory too: Qwen3.8 27B takes 17.2 GB as GGUF Q4_K_M and 52.8 GB as the original BF16 weights that vLLM runs. Below: a feature table checked against the docs on 2026-10-20, the memory for each format and what goes wrong.
- 17.2 GB — Qwen3.8 27B, GGUF Q4_K_M, 8K context: Ollama, LM Studio, llama.cpp
- 30.6 GB — the same model at FP8, 8K context: vLLM
- 52.8 GB — original BF16 weights, 8K context: vLLM
- 40.2 GB — FP8 with 16 requests at once, 8K each
Computed by the VRAM calculator, data as of 2026-10-19.
How do Ollama, LM Studio, llama.cpp and vLLM differ?
All four run open models, but they split the work differently: LM Studio and Ollama are ready-made programs, llama.cpp is an engine with many flags, and vLLM is a server for an API under load.
- Ollama is a program that downloads a model and runs it with one command, ollama run. It has a REST API and an OpenAI-compatible /v1 address.
- LM Studio is a desktop app: a chat, a model catalog and a local-server switch. For servers there is a windowless mode (llmster) and the lms command-line tool. It runs models on the llama.cpp engine, and on Macs with Apple Silicon on MLX as well.
- llama.cpp is an open C/C++ engine under the MIT licence. It also provides llama-server, with an OpenAI-compatible API and a web interface.
- vLLM is a library and server that serves models through an API. It was built for simultaneous requests and has no web chat.
What they can do in practice is in the table. Its rows come from the projects' READMEs and documentation, checked on 2026-10-20.
| Feature | Ollama | LM Studio | llama.cpp | vLLM |
|---|---|---|---|---|
| Interface | Terminal and API, an app on Windows and macOS; Open WebUI adds a web chat | A window with a chat, a model catalog and a server switch | Terminal and API; llama-server has a web interface | API only, no ready web chat |
| API | Its own REST API and /v1 in the OpenAI format (a subset), port 11434 | OpenAI-compatible /v1 and an Anthropic-compatible one; port 1234 in the examples | OpenAI-compatible chat completions, responses and embeddings, plus Anthropic Messages; --api-key | OpenAI-compatible (chat, completions, responses, embeddings) and Anthropic Messages; started with vllm serve |
| Several users | By default 1 request per model, the rest queue; set with OLLAMA_NUM_PARALLEL | Continuous batching in the llama.cpp engine, 4 requests at once by default | Parallel slots (-np) and continuous batching, on by default | Continuous batching and PagedAttention: built for throughput |
| Weight formats | GGUF, and safetensors on import; the Ollama library and hf.co/… | GGUF; MLX on Macs with Apple Silicon | GGUF, 1.5- to 8-bit quantisation | Hugging Face weights: FP8, INT8, INT4, GPTQ, AWQ. GGUF is “highly experimental” |
| Systems | Windows 10 22H2+, macOS, Linux, Docker; NVIDIA, AMD, Apple Metal, Vulkan | Windows (x64, ARM), Linux (x64, ARM64), macOS on Apple Silicon only | Windows, macOS, Linux, Docker; CUDA, HIP, Metal, Vulkan, SYCL, CPU | Linux; Windows only through WSL; NVIDIA from compute capability 7.5, AMD, Intel |
| Licence | MIT | Free at home and at work (since July 2025) | MIT | Apache 2.0 |
Sources: docs.ollama.com and the Ollama README, lmstudio.ai/docs, the ggml-org/llama.cpp repository on GitHub, docs.vllm.ai. We did not measure generation speed and do not compare it: it depends on the card, the model and the quantisation.
Where to download Ollama, LM Studio, llama.cpp and vLLM
Ollama, llama.cpp and vLLM install with a terminal command; LM Studio is an ordinary app from lmstudio.ai. The commands come from the projects' documentation.
# Ollama: Linux and macOS
curl -fsSL https://ollama.com/install.sh | sh
# Ollama: Windows, PowerShell
irm https://ollama.com/install.ps1 | iex
# llama.cpp: Linux and macOS
curl -LsSf https://llama.app/install.sh | sh
# vLLM: Linux with an NVIDIA card
uv pip install vllm --torch-backend=auto
vllm download-kernels
LM Studio runs on Windows, Linux and Macs with an Apple Silicon chip; it does not run on Intel Macs. llama.cpp has ready builds on the Releases page of the ggml-org/llama.cpp repository, and a Docker image. vLLM needs Linux and, for NVIDIA cards, compute capability 7.5 or higher: an RTX 20-series card or newer.
How much memory does one model take in each format?
The same Qwen3.8 27B takes from 17.2 GB to 52.8 GB, and the difference is in the weight format, not the program. Ollama, LM Studio and llama.cpp work with quantised GGUF files: Q4_K_M stores about 4.8 bits per parameter instead of 16. vLLM takes Hugging Face weights: the original BF16, or compressed FP8, AWQ or GPTQ.
Qwen3.8 27B: 17.2 GB at Q4_K_M and 52.8 GB at BF16 — a 24 GB card and an 80 GB one
| GB | |
|---|---|
| Q4_K_M (GGUF: Ollama, LM Studio, llama.cpp) | 17.2 |
| Q8_0 (GGUF) | 28.9 |
| FP8 (vLLM) | 30.6 |
| BF16 (original weights, vLLM) | 52.8 |
8K context, one request, FP16 cache. Weights are file sizes; data as of 2026-10-19.
The Q4_K_M weights are 3.3 times lighter than BF16 (15.3 GB against 50.9 GB). Q4_K_M fits a 24 GB card. Q8_0 and FP8 need a 32 GB card, and FP8 almost fills it. BF16 needs an 80 GB card.
Qwen3.8 27B at BF16 does not fit a 48 GB card. The BF16 weights are 50.9 GB, and 52.8 GB with the cache and buffers. They will not start on a 48 GB card: you need an 80 GB card or the FP8 build (30.6 GB). The first vLLM start takes 10–30 minutes or more, and the card is billed the whole time. Work out the memory before you rent.
GPT-OSS 20B looks different: it was released in MXFP4 from the start, and its weights are 11.3 GB. With an 8K context it needs 12.6 GB, a 16 GB card. Expanding the weights to BF16 gives about 40.3 GB (estimated from the parameter count) and a 48 GB card. Before you pick a program, check the format the model was released in: the memory depends on it.
What changes when many requests arrive?
All four can serve several requests at once, but every request has its own context cache, and it takes GPU memory. The Ollama documentation says so directly: memory grows as OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH.
Qwen3.8 27B at FP8: one request 30.6 GB, 16 at once at 8K 40.2 GB, at 32K 64.2 GB
| GB | |
|---|---|
| 1 request (8K) | 30.6 |
| 4 requests (8K) | 32.5 |
| 16 requests (8K) | 40.2 |
| 16 requests, 32K (32K window each) | 64.2 |
FP8 weights, FP16 cache. Every request fills its window, so this is an upper estimate. Data as of 2026-10-19.
For one request with an 8K window, the Qwen3.8 27B cache takes 0.64 GB: the model has a hybrid architecture, so it is modest. For 16 requests that is 10.3 GB, and at a 32K window 34.3 GB. One request fits a 32 GB card, four need 48 GB, and sixteen at 32K need 80 GB.
The number of simultaneous requests is set differently in each tool. In Ollama it is the OLLAMA_NUM_PARALLEL variable, in llama.cpp the -np flag (the number of slots), in LM Studio the Max Concurrent Predictions setting. In vLLM it is limited by the memory left after the weights.
What to pick: a PC, a server or maximum control
- One person, a home computer. LM Studio if you want a window with a chat and a model catalog. Ollama if you prefer the command line and need an API for your own programs. On Windows and Mac these two are the easiest.
- No GPU of your own, or the model does not fit. Rent a server with Ollama: the template gives a browser chat and an API. It is enough for you and a small team.
- An API for many users or a service. vLLM: continuous batching, Hugging Face weights, FP8 and AWQ. It needs Linux; on Windows it runs only through WSL.
- You need to squeeze a model into limited memory. llama.cpp: fine control over the cache, layers and experts, described below.
llama.cpp helps when a model barely fits. It can store the K and V cache compressed: the flags -ctk q8_0 -ctv q8_0 make it about half the size. For Qwen3.8 27B at Q4_K_M with a 128K context that is 20.9 GB instead of 24.7 GB, so a 24 GB card is enough instead of 32 GB. Ollama has the same trick (OLLAMA_KV_CACHE_TYPE), but with one cache type for all models. Its documentation calls the precision loss at q8_0 very small.
With MoE models, llama.cpp can keep the experts in system RAM (the --cpu-moe and --n-cpu-moe flags), and a model that does not fit in GPU memory runs in a hybrid CPU and GPU mode. In that mode answers are slower than with the whole model in GPU memory.
Check your own model
Pick a model, a format and a context in the free VRAM calculator: it shows the memory breakdown and which GPU sizes hold it. For vLLM look at the FP8 and BF16 options, for Ollama, LM Studio and llama.cpp at Q4_K_M and Q8_0. The rule of thumb is in the VRAM guide for LLMs.
What can go wrong
- Parallel requests multiply the cache. The first tables are computed for one request. For an API with several people, add memory for the cache of every simultaneous request, especially with a long context.
- GGUF in vLLM is not the main path. The vLLM documentation calls GGUF support highly experimental and requires a separate plugin. For vLLM take FP8, AWQ or the original weights.
- vLLM takes almost the whole card. The --gpu-memory-utilization parameter sets the share, so other jobs on the same card usually cannot run.
- The vLLM key does not guard every address. By the documentation, --api-key is checked only on the /v1, /v2 and /inference paths. If you install vLLM yourself, put the port behind a proxy.
- The model did not fit in GPU memory. Ollama and llama.cpp put some layers in system RAM and still start, but answers become noticeably slower. In Ollama the share shows in the Processor column of ollama ps.
How we counted
Weights are real file sizes: GGUF from Hugging Face repositories, FP8 and BF16 from the models' original repositories. The BF16 size of GPT-OSS 20B is computed from its parameter count. The context cache is computed from the model's config.json; for several requests it is multiplied by their number, each request filling its whole window. The runtime is about 0.75 GB plus the logits buffer. 1 GB here is 1024³ bytes, as on GPUs. Calculator data as of 2026-10-19; the method is in the calculator, and one table for 35 models is in How much VRAM do open LLMs really need?.
The tools' features were checked against their official documentation on 2026-10-20: the README and docs.ollama.com, lmstudio.ai/docs, the llama.cpp README and server documentation, and docs.vllm.ai. Releases are frequent, so open the project page before you install.
No GPU that big?
If the model you want does not fit your own card, the quickest test is a rented GPU for an hour. On gpu.nz (my service, billed by the minute) Ollama and vLLM have ready templates: Open WebUI + Ollama from an 8 GB card and vLLM from 24 GB, both with an OpenAI-compatible /v1 API. The walkthrough with request examples and common mistakes is in Ollama and vLLM on a rented GPU. LM Studio and llama.cpp you install yourself over SSH.
Disclosure: I run gpu.nz, a per-minute GPU rental; the feature table and the memory figures do not depend on that. The original, with a live calculator, is at gpu.nz/en/guides/ollama-lm-studio-llama-cpp-vllm.
Top comments (0)