I built a private, self-hosted AI home lab with Docker (offline ChatGPT + monitoring + backups)

#selfhosted #ai #docker #devops

Like a lot of people, I got tired of two things: paying monthly for AI tools, and not knowing where my data ends up. So I built a small, self-hosted "AI home lab" that runs entirely on my own machine — and packaged it so anyone can deploy it in about 5 minutes.

Here's what's in it and how it fits together.

The stack

Three independent Docker Compose stacks, each solving one problem:

🤖 Private AI (Ollama + Open WebUI)

A ChatGPT-style web interface that runs 100% offline. Ollama is the model engine; Open WebUI is the chat frontend. You pull a model (llama3.2, mistral, qwen2.5…) and chat privately — nothing leaves your box.

services:
  ollama:
    image: ollama/ollama:latest
    volumes:
      - ollama-data:/root/.ollama
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434

The two services talk over an internal Docker network — only the web UI is exposed.

📊 Monitoring (Netdata)

Zero-config, per-second dashboards for CPU, RAM, disk, network, and every container. You literally just start it and open the dashboard.

💾 Backups (restic)

Scheduled, encrypted, deduplicated backups of your Docker volumes. Set a password and a cron schedule and forget it.

Lessons learned building it

Network isolation matters. Each stack gets its own bridge network; only user-facing ports are published. Ollama should never be exposed directly.
Resource limits are not optional. Without deploy.resources limits, a big model can starve the host. Every service got both limits and reservations.
Encryption passwords are a footgun. restic encrypts with a password — lose it and the backups are gone. Document that loudly.
Test before shipping. I deployed every stack and verified the health endpoints before calling it done.

Hardware reality check

8 GB RAM runs small models (llama3.2) comfortably on CPU.
16 GB+ or a GPU unlocks the bigger, smarter models.
No GPU required — it just runs slower for large models.

Get it

I cleaned this up, documented every stack with step-by-step guides + troubleshooting, and put it up as a downloadable pack for anyone who'd rather not assemble it from scratch:

👉 https://symshah.gumroad.com/l/selfhosted-ai-homelab

Happy to answer questions about the setup in the comments — always keen to hear how others structure their homelab AI.

DEV Community