The Local LLM Lab — Part 11
One controlled experiment. One number. One verdict.
A home server has a question the cloud never asks: who is eating the RAM?
In the cloud, memory is a line item on a bill. At home, it's a finite pool — 64 GB on my box — shared by every container, every service, every daemon, and the one GPU that runs the AI. When the pool runs dry, something gets killed, and the thing that gets killed is usually the thing you least expected.
I ran a memory audit on my homelab: 54 Docker containers, 93 running systemd units, one RTX 3090, 64 GB of RAM. I pulled the per-service memory history over 7 days, the current per-process state, and the kernel's non-reclaimable share. This is what the 64 GB is actually holding, and what the number says about how much RAM a homelab really needs.
The setup
- Machine: HP Z840 Workstation, 2× Intel Xeon E5-2680 v4 (28 cores, 56 threads), 64 GB RAM, one RTX 3090
- Workload: 54 Docker containers, 93 running systemd units, all up
- Telemetry: 7 days of per-service memory at 2-hour resolution, plus the current per-process state and the kernel's non-reclaimable memory
- GPU tenants (for context): Whisper ASR, a TTS model, an embedding model — see the VRAM piece for what the card holds
The 64 GB is the honest number. It's not a "big" machine by data-centre standards, but it's a real workstation, and it runs a real stack: photo storage, music, a RAG pipeline, an LLM observability stack, a CI runner, a media server, a few self-hosted apps, and the AI services on the GPU.
The number
The box is using 41.6 GB of its 64 GB — 65% — and the peak over the week was 15.9 GB of application memory, with the rest held by the kernel and the GPU's system-side buffers.
| Layer | RAM (current) | Share of 64 GB |
|---|---|---|
| Kernel (non-reclaimable) | 1,754 MB (~1.7 GB) | 2.7% |
| GPU tenants (system-side buffers + CUDA context) | ~2.0 GB | 3.1% |
| Top application (Whisper ASR webservice) | 7,870 MB (~7.7 GB) | 12.2% |
| TTS voice model (omnivoice) | 5,100 MB (~5.0 GB) | 7.9% |
| LLM observability stack (web + worker + ClickHouse) | ~2.3 GB | 3.6% |
| Photo storage server (Immich: server + postgres + ML) | ~1.0 GB | 1.6% |
| RAG pipeline (Qdrant + Ollama + Chroma) | ~0.7 GB | 1.1% |
| The other 49 containers + 93 services | ~22.7 GB | 35.3% |
| Total in use | 41,622 MB (~40.7 GB) | 65% |
The "other 49 containers + 93 services" line is the one that surprises people. It's 35% of the RAM, and it's not one thing — it's a long tail. The biggest individual entries in that tail:
| Service | RAM (current) | What it is |
|---|---|---|
| ClickHouse (Langfuse storage) | 1,082 MB | The tracing database |
| Open WebUI (chat UI) | 777 MB | The chat interface |
| Langfuse web (LLM observability) | 721 MB | The tracing UI |
| Immich server (photo storage) | 562 MB | The photo library backend |
| Immich postgres (photo metadata) | 536 MB | The photo database |
| Qdrant (vector DB) | 483 MB | The RAG index |
| Langfuse worker (trace processing) | 346 MB | The trace worker |
| Botpress (chatbot) | 292 MB | The chatbot framework |
| n8n (workflow automation) | 279 MB | The automation engine |
| Flowise (LLM builder) | 255 MB | The LLM workflow builder |
| homelab-monitor (this monitor) | 233 MB | The dashboard itself |
| jarvis2-core (my assistant) | 229 MB | The assistant core |
| jarvis-server | 213 MB | The server host |
| jarvis2-voiceline (voice pipeline) | 199 MB | The voice line |
| Immich machine-learning (face/photo ML) | 177 MB | The photo ML worker |
| Langfuse minio (trace object storage) | 164 MB | The trace storage |
| searxng (meta-search) | 111 MB | The search engine |
| UptimeKuma (health checks) | 103 MB | The uptime monitor |
| MySQL (8.0, two instances) | 34 MB + 164 MB | The databases |
| GitHub runner (CI) | 58 MB | The CI agent |
| Postgres (three instances) | 15 MB + 20 MB + 536 MB | The databases |
The pattern: no single service is the hog. The RAM is eaten by a long tail of small, resident services, each holding 10–1,000 MB, and the sum is 22.7 GB. The GPU tenants hold the most per service (7.7 GB and 5.0 GB), but they're two services. The long tail is forty-nine, and it's still a third of the box.
What the peak says
The 7-day peak of 15.9 GB of application memory is the number that matters for sizing. It happened when the photo server was running a batch of thumbnail generation at the same time the CI runner was building a project and the observability stack was ingesting a burst of traces. Three things spiking at once, on a 64 GB box, and the peak was 15.9 GB of application memory — 25% of the pool.
The headroom at peak was 48 GB. That's the insurance. The box has never come close to the ceiling, and the events log over the week shows zero OOM kills, zero memory threshold crossings — the only alerts were eight disk I/O spikes, which are storage, not memory.
The honest reading: 64 GB is comfortable for this stack. 32 GB would be tight but workable. 16 GB would not. The peak of 15.9 GB is the worst case that actually happened, and it left 48 GB on the table. If I were buying RAM for this box today, I'd buy 64 GB and stop — not because it's the minimum, but because it's the size where the peak is a non-event.
The counterargument: RAM is cheap, don't sweat it
The fair objection: RAM is the cheapest part of a server. 64 GB is a few hundred euros. Why spend an article on it?
Because the RAM question is where homelab sizing goes wrong. People size the GPU (the expensive, scarce part) and under-size the RAM (the cheap, abundant part), and then the box OOMs the one service that matters — usually the database or the photo server — at the worst moment. The GPU is the part everyone benchmarks. The RAM is the part nobody measures, and it's the part that fails.
The measurement says the failure mode is not "one big service." It's the long tail. Fifty small services, each holding 10–100 MB, add up to a third of the box. The fix is not "buy more RAM" (though that helps). The fix is knowing the tail is real, so when the box gets tight you know which of the fifty to trim, not which of the two to kill.
What I would do now
If I were building this box today:
- RAM: 64 GB. The measured peak is 15.9 GB of application memory; 64 GB is 4× peak. 32 GB would be 2× peak — workable, but the photo server and the CI runner would start fighting.
- The rule: size RAM for the peak of the long tail, not for the biggest single service. The biggest service is 7.7 GB. The tail is 22.7 GB. The tail is the size.
- The audit: a per-service memory panel, current and 7-day peak, with the kernel share separated out. The number that matters is not "the box is at 65%" — it's "the peak was 15.9 GB, and here are the ten services that were up at the peak."
The homelab monitor I use has exactly that panel. The 7-day peak per service, the current per-process state, and the kernel's non-reclaimable share, in one view. It's the difference between "the box is full" and "here's what's full, and here's what to trim."
Verdict
54 containers, 93 services, 64 GB of RAM: 41.6 GB in use (65%), 15.9 GB peak over the week, zero OOM kills. The RAM is not eaten by one big service — it's eaten by a long tail of fifty small, resident services that add up to 22.7 GB. The GPU tenants hold the most per service, but the tail holds a third of the box by itself. Size the RAM for the peak of the tail, not for the biggest single service, and the box stops being a guessing game.
The Local LLM Lab is a series of measured experiments on a home GPU stack. Every number in this article was pulled from 7 days of per-service memory telemetry on the box, not estimated. Next: the one failed service on a 93-service stack — and how the uptime checks keep the rest alive.
Top comments (0)