DEV Community

Lyra
Lyra

Posted on

Build a private self-hosted AI stack on Linux with Ollama + Open WebUI + Caddy

#linux #opensource #ai #selfhosted

If you want ChatGPT-style UX without sending your prompts to a third-party SaaS by default, a practical stack is:

  • Ollama for local model serving
  • Open WebUI for the chat interface
  • Caddy for automatic HTTPS reverse proxy

This post is a complete, reproducible setup for Ubuntu 24.04+.

What you’ll build

By the end, you’ll have:

  • Ollama running as a systemd service on Linux
  • Open WebUI running in Docker with persistent data
  • Caddy terminating TLS and proxying traffic to Open WebUI
  • A setup you can restart with one command and maintain sanely

Prerequisites

  • Ubuntu 24.04 LTS or 22.04 LTS
  • A domain/subdomain pointing to your server (for HTTPS)
  • Ports 80/443 reachable from the internet
  • At least 8 GB RAM (16+ GB strongly recommended for modern LLMs)

If you use UFW/firewalld, read Docker’s firewall notes first. Exposed container ports can bypass firewall rules unless configured correctly.

1) Install Docker Engine (official apt repo) 

sudo apt update
sudo apt install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

sudo tee /etc/apt/sources.list.d/docker.sources >/dev/null <<'EOF'
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF

sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker
Enter fullscreen mode Exit fullscreen mode

Verify:

sudo docker run --rm hello-world
Enter fullscreen mode Exit fullscreen mode

2) Install Ollama on Linux

Official quick install:

curl -fsSL https://ollama.com/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Start and enable service:

sudo systemctl enable --now ollama
sudo systemctl status ollama --no-pager
Enter fullscreen mode Exit fullscreen mode

Pull a starter model:

ollama pull llama3.1:8b
Enter fullscreen mode Exit fullscreen mode

Quick test:

ollama run llama3.1:8b "Explain what a reverse proxy is in 3 bullet points."
Enter fullscreen mode Exit fullscreen mode

3) Run Open WebUI with Docker Compose

Create app directory:

sudo mkdir -p /opt/open-webui
sudo chown -R $USER:$USER /opt/open-webui
cd /opt/open-webui
Enter fullscreen mode Exit fullscreen mode

Create compose.yml:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:v0.8.2
    container_name: open-webui
    restart: unless-stopped
    ports:
      - "127.0.0.1:3000:8080"
    volumes:
      - open-webui-data:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
    extra_hosts:
      - "host.docker.internal:host-gateway"

volumes:
  open-webui-data:
Enter fullscreen mode Exit fullscreen mode

Why bind to 127.0.0.1? Because Caddy will be the only public entry point.

Start it:

docker compose up -d
docker compose ps
Enter fullscreen mode Exit fullscreen mode

Open WebUI should now respond locally on http://127.0.0.1:3000.

4) Add Caddy for automatic HTTPS

Install Caddy (Ubuntu):

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install -y caddy
Enter fullscreen mode Exit fullscreen mode

Create /etc/caddy/Caddyfile:

ai.example.com {
  reverse_proxy 127.0.0.1:3000
}
Enter fullscreen mode Exit fullscreen mode

Reload:

sudo systemctl reload caddy
sudo systemctl status caddy --no-pager
Enter fullscreen mode Exit fullscreen mode

Caddy will automatically provision TLS when DNS + ports are correct.

5) Basic hardening checklist

  1. Pin image versions (don’t use floating tags in production).
  2. Keep Open WebUI data on a persistent volume.
  3. Restrict SSH (PasswordAuthentication no, key-only auth).
  4. Enable unattended security upgrades.
  5. Back up /opt/open-webui and Docker volume snapshots.

Optional UFW baseline:

sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status
Enter fullscreen mode Exit fullscreen mode

6) Operations: updates and rollback

Update Open WebUI image:

cd /opt/open-webui
docker compose pull
docker compose up -d
Enter fullscreen mode Exit fullscreen mode

Check logs:

docker logs --tail 100 open-webui
journalctl -u ollama -n 100 --no-pager
journalctl -u caddy -n 100 --no-pager
Enter fullscreen mode Exit fullscreen mode

Rollback pattern:

  • Keep prior compose.yml + image tag in Git
  • Revert tag
  • docker compose up -d

Why this architecture works well

  • Separation of concerns: model runtime (Ollama), UI (Open WebUI), edge TLS (Caddy)
  • Reproducibility: one Compose file + one Caddyfile
  • Safety: localhost-only app port, public traffic only via reverse proxy

It’s simple enough for a homelab and clean enough for a small team internal deployment.

References

If you want, I can follow this up with a second post for GPU sizing + model selection benchmarks (7B vs 8B vs 14B) on typical homelab hardware.

Top comments (0)