Five Ollama Settings You Should Tune Before Running Local Models Seriously

By Nokka | September 11, 2026

This article was written by AI (deepseek-v4.1-flash) through Hermes Agent, reviewed and edited by Nokka.

People install Ollama and hit the same wall: responses are slower than expected even on good hardware.

Most of the time the hardware is not the problem. The defaults are simply not tuned for sustained heavy use [1].

Setting one: Flash Attention

Flash Attention speeds up the attention computation inside the model by keeping data in GPU cache instead of main memory [2].

A common misconception is that this is off by default. In practice Ollama enables it automatically when the system supports it [3].

The environment variable exists to force it on, for cases where auto-detection misses or you want a guarantee [2][3].

The payoff is both speed and quality on long-context work, because the technique manages memory more efficiently.

Setting two: KV cache type

KV cache type controls how the model stores the key/value state of the computation [1].

The important condition: compression only works when Flash Attention is already on. Without it, this option does nothing [3].

Compressing to one of the quantized formats can roughly double the context length you can hold before memory runs out, because the data takes less space.

The trade is a small accuracy loss. On ordinary tasks you will not feel it. On precision work, benchmark before committing.

Setting three: context length

The default context length is usually low to save memory. If you work with long documents or big code files, it will not be enough [4].

Raising it lets the model remember longer conversations, but memory use climbs sharply.

A safe practice is to start at the size your actual work needs. Do not max it out. An oversized context makes every single response slower.

Setting four: how many layers go to the GPU

By default the system decides which model layers run on the GPU [1].

If VRAM is limited but you still want a larger model, setting the layer count explicitly lets you use the hardware more fully.

This is split inference: some layers on GPU, some on CPU. Slower, but it runs.

Setting five: preloading the model

Every first call to Ollama has to load the model into memory, which takes anywhere from a few seconds to tens of seconds [3].

Preloading with an empty prompt keeps the model resident, so your first request is as fast as the rest.

This suits machines with plenty of spare memory. If memory is tight, keeping multiple models resident slows everything down.

Summary

Setting	Default	Set it to	Trade-off
Flash Attention	Auto when supported	Force on	Almost none
KV cache type	f16	q8_0 (needs FA on)	Minor accuracy
num_ctx	Low	Match your workload	Memory
num_gpu	Automatic	Explicit when needed	Configuration complexity
Preload	No	As needed	Reserved memory

Before you change anything

One Flash Attention behaves differently across GPUs. Some cards reportedly get slower with it on [1]. Measure before and after on your own workload.

Two KV cache compression reduces accuracy on work that needs precision, such as numerical analysis or reading legal documents.

Three The best values differ per machine. Copying someone else's settings without measuring your own gives unpredictable results.

Four Advertised speed numbers are usually measured on an idle machine. If you have other software running, expect less.

From someone who has tuned this the hard way

I run Ollama locally and the clearest lesson is that measurement has to be systematic.

Early on I changed several settings at once, saw an improvement, and had no idea which one helped. Changing one at a time with notes gave answers I could actually reuse.

The other thing I found is that Flash Attention is the least risky win for anyone on a recent NVIDIA card, because the technique was designed to manage memory directly.

My advice is to start with the two safest changes, Flash Attention on and context length matched to your work, then measure. Only reach for the settings that trade accuracy if that is still not enough.

References

[1] Khatik, K., "Optimizing Ollama Performance on Windows: Hardware, Quantization, Parallelism & More", Medium (2026), https://medium.com/@kapildevkhatik2/optimizing-ollama-performance-on-windows-hardware-quantization-parallelism-more-fac04802288e

[2] Broadcom TechDocs, "Understanding the Ollama provider" (2026), https://techdocs.broadcom.com/us/en/vmware-tanzu/platform/ai-services/10-0/ai/explanation-understanding-ollama-configuration.html

[3] Ollama, "FAQ" (accessed Sep 11, 2026), https://docs.ollama.com/faq

[4] Easton Dev, "Ollama Performance Tuning: Batching, KV Cache, and OOM" (2026), https://eastondev.com/blog/en/posts/ai/20260410-ollama-performance-optimization/