Ollama keep_alive -1 hangs every other model: a silent deadlock on a 6 GB GPU

My local LLM box is one machine with an RTX 2060. That's 6 GB of VRAM, and with the main model loaded there's about 1 GB left. Everything in the homelab that wants a local model talks to the one Ollama instance on it: the profiles of my agent platform, a user-modelling service, Home Assistant, a listing tool, and Open WebUI.

Twice now, every one of those clients has stopped getting answers. No crash, no error, nothing in the log. This is what was actually going on, including the fix I was sure about that turned out not to be one.

The first time: 26 August

Every agent turn hung. Not slow — hung. The requests for the main model sat open until each client gave up on its own schedule: 9 minutes on one, 3 minutes on another, 150 seconds on a third.

What I could see:

• nvidia-smi read 181 MiB of 6,144 used, 0% utilisation. The GPU wasn't struggling. It was idle.
• /api/ps listed exactly one model: a CPU-only build of qwen2.5:7b, with size_vram: 0 and an expires_at in the year 2318. That's what keep_alive: -1 looks like — "never unload".
• The Ollama log had nothing. No error, no model-load line, no request line.

That last one cost me the most time. Ollama's HTTP request log line is written when a request completes. A request that's waiting to be scheduled hasn't completed, so it doesn't exist in the log. No log line means in flight, not never arrived. And /api/ps only lists loaded models, so the model everyone was waiting for wasn't anywhere either.

The pinned model belonged to the user-modelling service. I'd pointed it at a CPU-only build on purpose, so it would stop fighting the agents for the card. It stopped fighting for the card and started blocking the queue instead.

The un-stick was one request, no restart:

curl -s http://<ollama-host>:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d '{"model":"<pinned-model>","keep_alive":0}'

That returned "done_reason":"unload". The queued request for the main model then loaded in 52.8 seconds and answered, and the embedding model loaded straight after it. The whole queue had been stuck behind one pin.

The fix that wasn't

The obvious suspect was how many models Ollama will keep loaded at once. I set OLLAMA_MAX_LOADED_MODELS=2 in a systemd drop-in, restarted, and ran the failure case on purpose: pin the CPU model with keep_alive: -1, then immediately ask for the main one. Before the change that hung past nine minutes. After it, HTTP 200 in 13.4 seconds, with the pinned model evicted.

I wrote it up as fixed. It wasn't.

The second time: 10 September

I went to run one of the agent profiles and it produced nothing for about 15 minutes. Same signature: expires_at in 2318, size_vram: 0. The quickest test turned out to be asking for the loaded model and then for any other one:

• the pinned model: 200 in 1.7 s
• any other model: no response after 60 s

My first note said the setting must have reverted. I checked the host. It hadn't. OLLAMA_MAX_LOADED_MODELS=2 had been in place, and active, since 26 August. That setting was never the protection.

Worth knowing too: Ollama's default for that setting isn't 1. The documented default is three models per GPU. The number of slots was never the problem.

What's actually happening

It's VRAM. On a 6 GB card with about 1 GB free, a second model can't load until the first one leaves. Ollama's scheduler doesn't refuse the request when there's no room — it waits for room. A model pinned with keep_alive: -1 never leaves, so the wait never ends.

No environment variable fixes that. None of them can make VRAM, and none of them can override a keep_alive a client explicitly asked for.

The part that made it properly bad: a hang never triggers a fallback chain. My agent profiles each have fallback providers configured, and those fire on an error. A request that just never answers isn't an error. So every profile on a local model was dead, the fallbacks sat unused, and nothing anywhere reported a fault. I found it because I happened to try to use one.

It comes back within hours

I cleared the pin at 11:39. By 14:07 a different model, plain qwen2.5:7b, was pinned to 2318. Nothing in my agent profiles or scripts referenced the first one, and none of them set keep_alive at all. So this wasn't a leftover. Something is actively doing it.

I still don't know what. I didn't pin it, and Ollama doesn't log request bodies. The clients I could see connected were the listing tool, Home Assistant, the agent host, and Open WebUI running on the box itself. Open WebUI has a per-model "Keep Alive" setting and Home Assistant's Ollama integration exposes one too, and both accept -1. Those are my two suspects. Neither is proven.

The guard's own log narrows it a little. The expiry timestamps on the pins it clears overnight almost all fall at about 18 and 48 minutes past the hour, so whatever it is runs on a half-hourly schedule.

The actual fix: a guard, not a setting

A small Python script on a cron every 15 minutes. It reads /api/ps and unloads any model whose expiry is more than 48 hours away. A real keep_alive is minutes or hours (mine defaults to 30 minutes), so anything further out was pinned, not scheduled. The core of it:

for m in (_get("/api/ps") or {}).get("models") or []:
    exp = datetime.fromisoformat(str(m.get("expires_at")).replace("Z", "+00:00"))
    if (exp - now).total_seconds() / 3600 > MAX_SANE_HOURS:   # 48
        _unload(m["name"])   # POST /api/generate {"model": ..., "keep_alive": 0}

Two decisions in there that matter:

• It only touches pins. A model with a normal expiry is left alone even if it's hogging the whole card, because that one will leave on its own.
• If Ollama is unreachable it exits cleanly and says so. A down server is a different fault with its own monitor, and this guard shouldn't page for it.

It has a --dry-run flag, and it's deterministic. No model is involved in deciding what to unload.

It's still firing. On the morning I'm writing this it unloaded qwen2.5:7b, "pinned for 292 years", at 09:45, 12:15 and 12:45. The log shows 408 unloads since it went in on 10 September, between 20 and 43 a day. Every one of those was a window where any other model would have hung.

What I'd tell anyone running Ollama on a small card

• A hang with no log line isn't a quiet server. Ollama logs a request when it finishes, so a missing line means the request is still waiting.
• /api/ps won't show you the model everyone's waiting for. It lists loaded models only. Look for expires_at years in the future and size_vram: 0 on the one that is loaded.
• An idle GPU during a hang means a scheduling problem, not a resource problem. 181 MiB used at 0% utilisation isn't a card under strain.
• Fallback chains fire on errors, not on silence. If your failover depends on the primary failing loudly, a hang gets past it. Put a timeout on the client, or watch for this directly.
• Test the fix against the real thing, then check again later. My regression test passed on the night, and the problem came back two weeks later with the setting still there.
• If one client can pin the card, assume one will. On a shared single-GPU box, clearing the pin on a schedule is more reliable than hunting down every client that might set it.

The un-stick is one curl. Knowing that you need it is the hard part, because nothing tells you.

---

🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.