Ollama Hardware Requirements: RAM, VRAM and GPU

Originally published on DevToolHub.

Ollama runs a model by loading its weights into memory. So the size of the model file is the number that decides what hardware you need. For example, a 4-bit 8B model is about 5 GB on disk. It needs roughly that much free memory plus overhead to run. But push to a 70B model and you are looking at 40 GB or more. This guide gives the real numbers by model size, explains GPU versus CPU, and covers the setting most people miss: context length.

The Ollama hardware requirements come down to one formula. Memory needed is roughly the model file size, plus 1 to 2 GB of overhead, plus the KV cache for your context window. Everything below is that formula applied.

What are the Ollama hardware requirements?

The baseline is simple. First, you need enough free RAM or VRAM to hold the model file plus about 20% headroom. Ollama's model tags like llama3.1 or gemma3 default to Q4_K_M, a 4-bit quantization, so the download sizes on ollama.com are already compressed.

The official README sets the baseline: at least 8 GB of RAM for 7B models, 16 GB for 13B models, and 32 GB for 33B models. That holds up in practice for 4-bit quants at a modest context length. However, larger context windows and running multiple models at once push those numbers higher.

Storage matters too. Models live in ~/.ollama/models, or /usr/share/ollama/.ollama/models on a Linux service install. A single 70B model is about 40 GB on disk. So it is easy to fill a drive if you pull several large models to compare them.

Ollama hardware requirements by model size

These figures assume the default 4-bit quantization and a small-to-moderate context window. Add memory for larger context (see below).

Model size	Approx. file size (Q4)	Minimum RAM (CPU)	Comfortable GPU VRAM	Example GPU
1B–3B	0.7–2 GB	8 GB	4 GB	Most modern GPUs, integrated graphics
7B–8B	4.5–5 GB	8 GB	6–8 GB	RTX 3060, RTX 4060
12B–14B	7–9 GB	16 GB	12 GB	RTX 3060 12GB, RTX 4070
27B–32B	16–20 GB	32 GB	24 GB	RTX 3090, RTX 4090
70B	40–43 GB	64 GB	48 GB	RTX 6000 Ada, 2× RTX 3090
120B+ / MoE	65 GB and up	128 GB+	80 GB+	A100, H100, multi-GPU

Planning to serve a model to more than one request at a time? Then multiply the context memory by the number of parallel slots. Ollama's OLLAMA_NUM_PARALLEL controls that count, and each slot needs its own slice of KV cache. For a broader look at which models are worth running, see the best open-source LLMs roundup.

Do you need a GPU to run Ollama?

No. Ollama runs on CPU alone, and a 7B model on a modern multi-core CPU with fast RAM produces a few tokens per second. That is usable for scripts and batch jobs but slow for interactive chat.

A GPU changes the experience. When a compatible GPU fits the whole model in VRAM, Ollama loads every layer there. So inference is often 10 to 30 times faster than CPU. But when the model does not fit, Ollama does a partial offload. It puts as many layers as possible on the GPU and runs the rest on the CPU. That works, however speed drops sharply because every token now waits on the slow path.

⚠️ Note: a partial offload can be slower than pure CPU in some cases because of the constant copying between GPU and system RAM. If ollama ps shows a CPU/GPU split, either use a smaller model or a smaller quantization so the whole thing fits in VRAM.

For CPU-only inference, RAM bandwidth is the bottleneck, not core count. DDR5 and dual-channel memory help more than extra cores. The DigitalOcean droplet setup guide shows CPU-only sizing for a cloud server.

Which GPUs does Ollama support?

Ollama supports three GPU paths, and the requirements differ:

• NVIDIA: compute capability 5.0 and higher, with driver 550 or newer (570+ for the oldest supported cards). This covers everything from the GTX 900 series through the RTX 50 series, and you can confirm a card on the NVIDIA CUDA GPUs list. NVIDIA is the most tested path.
• AMD: on Linux, ROCm v7 drivers are required. Supported cards include the Radeon RX 9000 and RX 7000 series, parts of the RX 6000 series, Radeon PRO W-series, and Instinct accelerators. Windows support is narrower.
• Apple Silicon: Ollama uses the Metal API on M-series Macs with no extra setup. Unified memory means the GPU can address most of system RAM (more on this below).

To limit Ollama to specific GPUs, set CUDA_VISIBLE_DEVICES (NVIDIA) or ROCR_VISIBLE_DEVICES (AMD) to the device UUIDs. Set CUDA_VISIBLE_DEVICES=-1 to force CPU-only mode. The full compatibility table is in the official GPU documentation.

How context length changes Ollama hardware requirements

Context length is the setting that quietly breaks memory planning. The KV cache holds the attention state for every token in the context window, and it grows with both the window size and the model size. A large context can add several gigabytes on top of the model weights.

Recent Ollama versions set the default context length based on available VRAM rather than a fixed value:

• Under 24 GB VRAM: 4K tokens
• 24–48 GB VRAM: 32K tokens
• 48 GB or more: 256K tokens

You override this with OLLAMA_CONTEXT_LENGTH on the server or num_ctx per request:

OLLAMA_CONTEXT_LENGTH=16384 ollama serve

Two settings reduce the cost. First, enabling flash attention with OLLAMA_FLASH_ATTENTION=1 cuts KV cache memory as context grows. Also, you can cap context per request instead of raising the server default for every model. So if you need long context for agents or coding tools, budget for it in your VRAM math from the start. Running a model across a cluster is a different problem, covered in the deploy an LLM on Kubernetes guide.

How to check if Ollama is using your GPU

Run ollama ps while a model is loaded. The PROCESSOR column tells you exactly where the model is running:

$ ollama ps
NAME               ID              SIZE     PROCESSOR    UNTIL
llama3.1:8b        365c0bd3c000    6.7 GB   100% GPU     4 minutes from now

100% GPU means the whole model is in VRAM. 100% CPU means no GPU acceleration. A split like 35%/65% CPU/GPU means partial offload, and that is your signal to drop to a smaller model or quant.

If you expect GPU use and see CPU, work through three checks. First, confirm the driver meets the minimum version. Then check that nvidia-smi or rocminfo sees the card. Finally, make sure another process is not already holding the VRAM.

Apple Silicon: unified memory changes the math

On an M-series Mac, the CPU and GPU share one pool of memory, so there is no separate VRAM number. A MacBook with 64 GB of unified memory can run models that would need a 48 GB workstation GPU on a PC. This makes high-memory Macs one of the most cost-effective ways to run 70B models locally.

The catch is that macOS reserves part of that memory for the system. By default the GPU can use roughly 65 to 75 percent of total RAM. That is usually fine. But if a large model fails to load on a Mac that seems to have enough memory, the wired memory limit is the likely cause.

Frequently Asked Questions

Q: What are the minimum Ollama hardware requirements?
A: A machine with 8 GB of RAM runs small models (1B–3B) and quantized 7B models on CPU. For a smooth experience with 7B models, use 16 GB of RAM or a GPU with 6–8 GB of VRAM. No GPU is required, but one makes inference 10 to 30 times faster.

Q: How much VRAM do I need for a 70B model in Ollama?
A: About 40 to 43 GB for the 4-bit weights, plus a few GB for context. A single 48 GB GPU or two 24 GB GPUs handles it. On CPU, plan for 64 GB of system RAM and expect slow output.

Q: Can Ollama run on a server with no GPU?
A: Yes. CPU-only inference works for any model that fits in RAM. Expect a few tokens per second for 7B models. Fast, dual-channel DDR5 memory matters more than the number of CPU cores.

Q: Why is my model running on CPU when I have a GPU?
A: Common causes are a driver below the minimum version, too little free VRAM, another process holding VRAM, or an unsupported card. Run ollama ps to confirm, then nvidia-smi to check the card and driver.

Q: Does a bigger context window need more memory?
A: Yes. The KV cache scales with context length and model size and can add several GB. Enable OLLAMA_FLASH_ATTENTION=1 to reduce it, and only raise OLLAMA_CONTEXT_LENGTH as far as your hardware allows.

Quick Summary:

• Memory needed ≈ model file size + 1–2 GB overhead + KV cache for your context window.
• Official minimums: 8 GB RAM for 7B, 16 GB for 13B, 32 GB for 33B, all at 4-bit quantization.
• A 70B model needs about 40–43 GB of VRAM or 64 GB of system RAM.
• No GPU is required, but a GPU that fits the whole model is 10–30× faster than CPU; a partial CPU/GPU split is much slower.
• NVIDIA needs compute capability 5.0+ and driver 550+; AMD needs ROCm v7 on Linux; Apple Silicon uses unified memory and punches above its price for large models.
• Run ollama ps and read the PROCESSOR column to confirm where a model is actually running.

So once you have matched the Ollama hardware requirements to your model, the rest is setup. But if local hardware caps out below the model you need, compare the hosted tier in the Ollama Cloud pricing and limits guide.