DEV Community

abidaslam892
abidaslam892

Posted on

Run a Private ChatGPT-style Assistant with Ollama and Open WebUI

description: Step-by-step guide to self-host Ollama and Open WebUI on a Linux server,
secure it, and start chatting with a local LLM.

tags: ollama, openwebui, docker, selfhosted

You want a ChatGPT-like assistant that never leaves your network. Ollama provides the LLM inference engine, while Open WebUI gives a friendly web front-end. Both run in Docker, so you can spin them up on any modern Linux host (Ubuntu 22.04 or 24.04 LTS is used in the examples). This guide walks you through:

  • Installing Docker safely.
  • Deploying Ollama as a container and pulling a model.
  • Deploying Open WebUI and wiring it to the local Ollama endpoint.
  • Publishing the UI safely behind Nginx with free HTTPS from Let's Encrypt.
  • Common pitfalls and a quick checklist.

1. Prerequisites

  • A fresh Ubuntu 22.04 or 24.04 server (virtual or bare-metal). SSH access with a non-root user that has sudo rights.
  • At least 8 GB RAM for an 8B model such as llama3.1:8b (a 3B model like llama3.2:3b runs in about 4 GB). Bigger models need more memory.
  • A domain name pointing to the server (needed for a free HTTPS certificate), with ports 80 and 443 open in the firewall.

Security note – Run everything as a non-root user. Only give the Docker group to that user; do not run containers as root unless absolutely necessary.


2. Install Docker Engine (official method)

# Update the apt index and install prerequisites
sudo apt-get update && sudo apt-get install -y \
    ca-certificates curl gnupg lsb-release

# Add Docker’s official GPG key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg |
    sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

# Set up the stable repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" |
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine
sudo apt-get update && sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Verify the installation
sudo docker run --rm hello-world
Enter fullscreen mode Exit fullscreen mode

Add your user to the docker group so you can run commands without sudo:

sudo usermod -aG docker $USER
newgrp docker   # refresh group membership in the current session
Enter fullscreen mode Exit fullscreen mode

3. Deploy Ollama

Ollama ships an official Docker image that runs the inference server on port 11434. Create a dedicated Docker network so the two containers can talk privately. Ollama has no login of its own, so it is not published to the internet at all.

# Create a network named "ai"
docker network create ai

# Run Ollama container
docker run -d \
    --name ollama \
    --restart unless-stopped \
    --network ai \
    -v ollama-data:/root/.ollama \
    ollama/ollama:latest
Enter fullscreen mode Exit fullscreen mode

3.1 Pull a model

Enter the container and use Ollama’s CLI to pull a model. Replace llama3.1:8b with the model you prefer.

docker exec -it ollama ollama pull llama3.1:8b
Enter fullscreen mode Exit fullscreen mode

You can list downloaded models with docker exec -it ollama ollama list. The model files are stored in the persistent volume ollama-data so they survive container restarts.


4. Deploy Open WebUI

Open WebUI is a self-hosted chat front-end (Svelte and FastAPI) that talks to the Ollama HTTP API. It runs in Docker and is configured with environment variables. It has its own user accounts: the first account you create becomes the admin.

docker run -d \
    --name openwebui \
    --restart unless-stopped \
    --network ai \
    -p 127.0.0.1:8080:8080 \
    -v open-webui:/app/backend/data \
    -e OLLAMA_BASE_URL=http://ollama:11434 \
    ghcr.io/open-webui/open-webui:main
Enter fullscreen mode Exit fullscreen mode

The UI listens only on the server itself (127.0.0.1:8080), so nobody can reach it from the internet yet; Nginx publishes it safely in the next step. Because both containers share the ai network, Open WebUI reaches Ollama at http://ollama:11434 without exposing Ollama. The open-webui volume keeps your users and chats across updates.


5. Publish the UI with Nginx and free HTTPS

Nginx receives HTTPS traffic on port 443 and forwards it to Open WebUI on the server. Certbot gets a free certificate from Let's Encrypt and renews it automatically.

5.1 Install Nginx and Certbot

sudo apt-get install -y nginx certbot python3-certbot-nginx
Enter fullscreen mode Exit fullscreen mode

5.2 Configure Nginx

Create /etc/nginx/sites-available/webui.conf (replace chat.example.com with your domain):

server {
    listen 80;
    server_name chat.example.com;

    client_max_body_size 50M;   # allow document uploads

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;   # live streaming of answers
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
Enter fullscreen mode Exit fullscreen mode

Enable the site and reload Nginx:

sudo ln -s /etc/nginx/sites-available/webui.conf /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
Enter fullscreen mode Exit fullscreen mode

5.3 Get a free HTTPS certificate

sudo certbot --nginx -d chat.example.com --redirect
Enter fullscreen mode Exit fullscreen mode

Certbot adds the certificate to the Nginx config, redirects HTTP to HTTPS and sets up automatic renewal. Now open https://chat.example.com, create the first account (it becomes the admin), and in the admin settings turn off new sign-ups so only people you invite can use it.


6. Verify the Setup

  1. Open a browser and go to https://chat.example.com.
  2. Log in with the admin account you created.
  3. Type a prompt in the chat box, e.g., “Explain the difference between TCP and UDP.”
  4. The first answer can take longer while the model loads; after that, answers should appear within a few seconds on a capable server – the request travels:
    • Browser → nginx (HTTPS) → Open WebUI (127.0.0.1:8080) → Ollama (http://ollama:11434) → model inference → response back.

If you see errors, check the container logs:

docker logs openwebui   # UI errors
docker logs ollama      # model loading issues
Enter fullscreen mode Exit fullscreen mode

7. Common Pitfalls

Symptom Likely Cause Fix
Open WebUI shows no models Wrong Ollama address, or containers on different networks Check OLLAMA_BASE_URL=http://ollama:11434 and that both containers use --network ai (docker network inspect ai).
Model fails to load, “out of memory” Not enough RAM for the chosen model Choose a smaller model or add swap space.
502 Bad Gateway from Nginx Open WebUI is not running, or not on 127.0.0.1:8080 docker ps and docker logs openwebui; check the -p 127.0.0.1:8080:8080 flag.
Certbot fails Domain does not point to the server, or port 80 is closed Check the DNS record and open ports 80 and 443 in the firewall.
Strangers can create accounts New sign-ups still enabled Turn off new sign-ups in the Open WebUI admin settings.
Docker commands require sudo User not in docker group Re-login or run newgrp docker after usermod -aG docker $USER.

8. Quick Checklist

  • [ ] Ubuntu 22.04/24.04, user in the sudo group, domain pointing to the server.
  • [ ] Docker Engine installed from the official Docker repository.
  • [ ] Docker network ai created.
  • [ ] Ollama container running, model pulled, volume ollama-data persisted.
  • [ ] Ollama not published to the internet (no -p 11434).
  • [ ] Open WebUI on 127.0.0.1:8080 only, with OLLAMA_BASE_URL=http://ollama:11434.
  • [ ] Nginx site enabled and Let's Encrypt certificate issued by Certbot.
  • [ ] Admin account created and new sign-ups turned off.
  • [ ] Verify HTTPS access and successful chat response.

You now have a private, self-hosted ChatGPT-style assistant that runs entirely on your own hardware. Keep it up to date: pull the new images, then re-create the containers with the same docker run commands (your volumes keep the data) and monitor RAM usage as you experiment with larger models.


Want this done for you?

I set this up for clients, adapted to their app and accounts, tested and handed over with clear notes: see the service on Fiverr.

Top comments (0)