Keep Your Self-Hosted AI Console Off the Public Internet with Docker Compose
You wake at 3:00 AM to a billing alert: someone discovered your Open-WebUI admin panel was answering prompts from the internet. You mapped 3000:8080 in Compose, assumed the firewall would hold, and went to bed. The firewall rule was permissive. The VM had a public IP. Your internal experiment became an unauthenticated relay.
The dangerous part of self-hosted AI is not getting containers running. It is discovering that the administrative interface is reachable from somewhere you never intended.
My approach to Docker Compose is deliberately narrow: use it to describe service dependencies and network boundaries, then put human access behind Tailscale identity. Compose orchestrates containers; it does not decide which people should reach them.
The boundary comes before the application
The request path is:
Authorized tailnet user → Tailscale Serve → loopback listener → Open-WebUI → LiteLLM → model endpoint.
Redis sits behind LiteLLM as a short-lived response cache. Neither Redis nor LiteLLM publishes a host port. Open-WebUI joins the private application network, while LiteLLM additionally joins an outbound-capable network to reach upstream inference services.
The failure pattern worth designing against is an internal admin panel exposed publicly without client authentication. Adding an application password afterward does not undo the exposure. Here, Tailscale authenticates devices and enforces access before the application receives traffic; Open-WebUI still authenticates individual users.
This is an alternative to public ingress with mTLS, not an implementation of mTLS between these containers.
One deployment, three configuration files
The following shell block creates the Compose project, LiteLLM configuration, and a restrictive Tailscale policy.
Before starting, set LITELLM_MASTER_KEY, REDIS_PASSWORD, and BLOST_API_KEY in your shell. Generate the first two independently with openssl rand -hex 32.
mkdir -p private-ai
cd private-ai
cat > docker-compose.yml <<'YAML'
services:
webui:
image: ghcr.io/open-webui/open-webui:main
restart: unless-stopped
ports:
- "127.0.0.1:3000:8080"
environment:
OPENAI_API_BASE_URL: https://b-lost.com/v1
OPENAI_API_KEY: ${LITELLM_MASTER_KEY:?set LITELLM_MASTER_KEY}
ENABLE_OLLAMA_API: "false"
ENABLE_SIGNUP: "false"
volumes:
- webui-data:/app/backend/data
networks: [backend]
depends_on:
litellm:
condition: service_started
litellm:
image: ghcr.io/berriai/litellm:main-stable
restart: unless-stopped
command: ["--config", "/app/config.yaml", "--port", "4000"]
environment:
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:?set LITELLM_MASTER_KEY}
BLOST_API_KEY: ${BLOST_API_KEY:?set BLOST_API_KEY}
REDIS_HOST: redis
REDIS_PASSWORD: ${REDIS_PASSWORD:?set REDIS_PASSWORD}
volumes:
- ./litellm.yaml:/app/config.yaml:ro
networks: [backend, outbound]
depends_on:
redis:
condition: service_healthy
redis:
image: redis:7.4-alpine
restart: unless-stopped
environment:
REDIS_PASSWORD: ${REDIS_PASSWORD:?set REDIS_PASSWORD}
command:
- /bin/sh
- -ec
- exec redis-server --requirepass "$$REDIS_PASSWORD" --save "" --appendonly no
healthcheck:
test:
- CMD-SHELL
- 'REDISCLI_AUTH="$$REDIS_PASSWORD" redis-cli ping | grep -q PONG'
interval: 5s
timeout: 3s
retries: 10
networks: [backend]
networks:
backend:
internal: true
outbound: {}
volumes:
webui-data:
YAML
cat > litellm.yaml <<'YAML'
model_list:
- model_name: relay-baseline
litellm_params:
model: openai/os.environ/RELAY_MODEL
api_base: https://b-lost.com/v1
api_key: os.environ/BLOST_API_KEY
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
litellm_settings:
cache: true
cache_params:
type: redis
host: os.environ/REDIS_HOST
password: os.environ/REDIS_PASSWORD
port: 6379
ttl: 60
YAML
# Replace the model placeholder with an ID supported by your relay account.
sed -i 's|openai/os.environ/RELAY_MODEL|openai/YOUR_SUPPORTED_MODEL_ID|' litellm.yaml
cat > tailscale-policy.hujson <<'JSON'
{
"groups": {
"group:ai-operators": ["alice@example.com"]
},
"tagOwners": {
"tag:ai-console": ["group:ai-operators"]
},
"grants": [
{
"src": ["group:ai-operators"],
"dst": ["tag:ai-console"],
"ip": ["tcp:443"]
}
]
}
JSON
docker compose config --quiet
docker compose up -d
tailscale serve --bg --https=443 http://127.0.0.1:3000
Replace the model placeholder before startup. The B-Lost endpoint, https://b-lost.com/v1, occupies the upstream relay position in this topology; it is not evidence of measured latency or availability. Confirm model identifiers and API compatibility against your account before sending real prompts.
The image tags make this bootstrap convenient, not reproducible. Before retaining the deployment, resolve reviewed versions to immutable digests and record them with the configuration.
Apply the policy, not just the YAML
The generated policy file is not automatically installed. Replace the example identity, apply the policy through your tailnet administration interface, and assign tag:ai-console to the server. Tailscale Serve also requires HTTPS support enabled for the tailnet.
Treat this policy as a dedicated-tailnet example. Existing broad ACLs or grants remain additive: introducing a restrictive grant does not cancel an existing allow-all rule. Adapt the complete policy, rather than appending this fragment and assuming isolation.
Do not enable Funnel. Serve provides tailnet access; Funnel creates public access.
Open-WebUI signup starts disabled. For initial administrator creation, temporarily enable signup, recreate the WebUI container, register the first account through the authorized tailnet connection, then disable signup and recreate it again.
What this deliberately trades away
Loopback binding prevents direct remote access to the published port, but it does not isolate hostile local users. Likewise, Docker network membership is not per-process authorization. A compromised WebUI possesses the LiteLLM credential configured for it.
Redis caching also changes the privacy boundary: responses may remain in memory for sixty seconds. Benchmark upstream latency with caching disabled, or repeated prompts will measure cache performance instead of relay performance.
Keep Open-WebUI single-instance with this local data volume. SQLite lock errors under concurrent writes indicate contention, not automatically corruption; scaling replicas against the same file is not a repair strategy.
Finally, verify the boundary from two clients: an authorized operator should reach HTTPS, while an unauthorized tailnet identity should fail. Confirm port 3000 is unreachable remotely. A successful docker compose up proves container startup, not access control.
The hardest trade-off here is accepting that Tailscale becomes a single point of control failure. If your tailnet authentication breaks, coordination service is unreachable, or device keys expire during an incident, you lose administrative access to your own infrastructure. Recovery requires either console access to the host or a parallel out-of-band path you maintained independently.
How do you balance restrictive network boundaries against the operational risk of locking yourself out during the one incident where you need emergency access most? What does your fallback look like?
Top comments (0)