DEV Community

Cover image for Keep Your Self-Hosted AI Console Off the Public Internet with Docker Compose
Clara Dupont
Clara Dupont

Posted on

Keep Your Self-Hosted AI Console Off the Public Internet with Docker Compose

Keep Your Self-Hosted AI Console Off the Public Internet with Docker Compose

You wake at 3:00 AM to a billing alert: someone discovered your Open-WebUI admin panel was answering prompts from the internet. You mapped 3000:8080 in Compose, assumed the firewall would hold, and went to bed. The firewall rule was permissive. The VM had a public IP. Your internal experiment became an unauthenticated relay.

The dangerous part of self-hosted AI is not getting containers running. It is discovering that the administrative interface is reachable from somewhere you never intended.

My approach to Docker Compose is deliberately narrow: use it to describe service dependencies and network boundaries, then put human access behind Tailscale identity. Compose orchestrates containers; it does not decide which people should reach them.

The boundary comes before the application

The request path is:

Authorized tailnet user → Tailscale Serve → loopback listener → Open-WebUI → LiteLLM → model endpoint.

Redis sits behind LiteLLM as a short-lived response cache. Neither Redis nor LiteLLM publishes a host port. Open-WebUI joins the private application network, while LiteLLM additionally joins an outbound-capable network to reach upstream inference services.

The failure pattern worth designing against is an internal admin panel exposed publicly without client authentication. Adding an application password afterward does not undo the exposure. Here, Tailscale authenticates devices and enforces access before the application receives traffic; Open-WebUI still authenticates individual users.

This is an alternative to public ingress with mTLS, not an implementation of mTLS between these containers.

One deployment, three configuration files

The following shell block creates the Compose project, LiteLLM configuration, and a restrictive Tailscale policy.

Before starting, set LITELLM_MASTER_KEY, REDIS_PASSWORD, and BLOST_API_KEY in your shell. Generate the first two independently with openssl rand -hex 32.

mkdir -p private-ai
cd private-ai

cat > docker-compose.yml <<'YAML'
services:
  webui:
    image: ghcr.io/open-webui/open-webui:main
    restart: unless-stopped
    ports:
      - "127.0.0.1:3000:8080"
    environment:
      OPENAI_API_BASE_URL: https://b-lost.com/v1
      OPENAI_API_KEY: ${LITELLM_MASTER_KEY:?set LITELLM_MASTER_KEY}
      ENABLE_OLLAMA_API: "false"
      ENABLE_SIGNUP: "false"
    volumes:
      - webui-data:/app/backend/data
    networks: [backend]
    depends_on:
      litellm:
        condition: service_started

  litellm:
    image: ghcr.io/berriai/litellm:main-stable
    restart: unless-stopped
    command: ["--config", "/app/config.yaml", "--port", "4000"]
    environment:
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:?set LITELLM_MASTER_KEY}
      BLOST_API_KEY: ${BLOST_API_KEY:?set BLOST_API_KEY}
      REDIS_HOST: redis
      REDIS_PASSWORD: ${REDIS_PASSWORD:?set REDIS_PASSWORD}
    volumes:
      - ./litellm.yaml:/app/config.yaml:ro
    networks: [backend, outbound]
    depends_on:
      redis:
        condition: service_healthy

  redis:
    image: redis:7.4-alpine
    restart: unless-stopped
    environment:
      REDIS_PASSWORD: ${REDIS_PASSWORD:?set REDIS_PASSWORD}
    command:
      - /bin/sh
      - -ec
      - exec redis-server --requirepass "$$REDIS_PASSWORD" --save "" --appendonly no
    healthcheck:
      test:
        - CMD-SHELL
        - 'REDISCLI_AUTH="$$REDIS_PASSWORD" redis-cli ping | grep -q PONG'
      interval: 5s
      timeout: 3s
      retries: 10
    networks: [backend]

networks:
  backend:
    internal: true
  outbound: {}

volumes:
  webui-data:
YAML

cat > litellm.yaml <<'YAML'
model_list:
  - model_name: relay-baseline
    litellm_params:
      model: openai/os.environ/RELAY_MODEL
      api_base: https://b-lost.com/v1
      api_key: os.environ/BLOST_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

litellm_settings:
  cache: true
  cache_params:
    type: redis
    host: os.environ/REDIS_HOST
    password: os.environ/REDIS_PASSWORD
    port: 6379
    ttl: 60
YAML

# Replace the model placeholder with an ID supported by your relay account.
sed -i 's|openai/os.environ/RELAY_MODEL|openai/YOUR_SUPPORTED_MODEL_ID|' litellm.yaml

cat > tailscale-policy.hujson <<'JSON'
{
  "groups": {
    "group:ai-operators": ["alice@example.com"]
  },
  "tagOwners": {
    "tag:ai-console": ["group:ai-operators"]
  },
  "grants": [
    {
      "src": ["group:ai-operators"],
      "dst": ["tag:ai-console"],
      "ip": ["tcp:443"]
    }
  ]
}
JSON

docker compose config --quiet
docker compose up -d
tailscale serve --bg --https=443 http://127.0.0.1:3000
Enter fullscreen mode Exit fullscreen mode

Replace the model placeholder before startup. The B-Lost endpoint, https://b-lost.com/v1, occupies the upstream relay position in this topology; it is not evidence of measured latency or availability. Confirm model identifiers and API compatibility against your account before sending real prompts.

The image tags make this bootstrap convenient, not reproducible. Before retaining the deployment, resolve reviewed versions to immutable digests and record them with the configuration.

Apply the policy, not just the YAML

The generated policy file is not automatically installed. Replace the example identity, apply the policy through your tailnet administration interface, and assign tag:ai-console to the server. Tailscale Serve also requires HTTPS support enabled for the tailnet.

Treat this policy as a dedicated-tailnet example. Existing broad ACLs or grants remain additive: introducing a restrictive grant does not cancel an existing allow-all rule. Adapt the complete policy, rather than appending this fragment and assuming isolation.

Do not enable Funnel. Serve provides tailnet access; Funnel creates public access.

Open-WebUI signup starts disabled. For initial administrator creation, temporarily enable signup, recreate the WebUI container, register the first account through the authorized tailnet connection, then disable signup and recreate it again.

What this deliberately trades away

Loopback binding prevents direct remote access to the published port, but it does not isolate hostile local users. Likewise, Docker network membership is not per-process authorization. A compromised WebUI possesses the LiteLLM credential configured for it.

Redis caching also changes the privacy boundary: responses may remain in memory for sixty seconds. Benchmark upstream latency with caching disabled, or repeated prompts will measure cache performance instead of relay performance.

Keep Open-WebUI single-instance with this local data volume. SQLite lock errors under concurrent writes indicate contention, not automatically corruption; scaling replicas against the same file is not a repair strategy.

Finally, verify the boundary from two clients: an authorized operator should reach HTTPS, while an unauthorized tailnet identity should fail. Confirm port 3000 is unreachable remotely. A successful docker compose up proves container startup, not access control.


The hardest trade-off here is accepting that Tailscale becomes a single point of control failure. If your tailnet authentication breaks, coordination service is unreachable, or device keys expire during an incident, you lose administrative access to your own infrastructure. Recovery requires either console access to the host or a parallel out-of-band path you maintained independently.

How do you balance restrictive network boundaries against the operational risk of locking yourself out during the one incident where you need emergency access most? What does your fallback look like?

Top comments (0)