ODS (Osmantic Deployment System) bundles Ollama, Open WebUI, n8n, and ComfyUI into a single Docker Compose stack that turns consumer hardware into a self-hosted AI server. It has 4,906 stars and is trending because it represents a specific deployment pattern: monolithic local agent infrastructure instead of cloud-native microservices.
This is the opposite of platforms like Hoplite or InsForge. ODS puts LLM inference, chat UI, workflow orchestration, and image generation on one machine with shared GPU resources. That design exposes real plumbing questions about service isolation, resource contention, authentication boundaries, and failure recovery when agents share hardware.
Architecture: One Stack, Multiple Agent Surfaces
ODS orchestrates four primary services in a single Docker Compose file:
- Ollama: LLM inference server (llama.cpp backend)
- Open WebUI: ChatGPT-style chat interface with RAG and voice
- n8n: Workflow automation engine for agent orchestration
- ComfyUI: Image generation and diffusion workflows
Each service runs in its own container but shares the host GPU via Docker device passthrough. The installer detects NVIDIA, AMD, or Strix Halo hardware and configures the appropriate runtime (nvidia-docker or ROCm).
The repository separates the installer (root directory) from the runtime (ods/ directory). The runtime contains Docker Compose overlays, service configuration templates, a control dashboard, and CLI tooling. The installer handles zero-prereq bootstrap: GPU driver detection, model downloads, service health checks, and initial auth setup.
Service Isolation and GPU Contention
ODS does not implement GPU resource quotas or memory partitioning. All services see the same GPU device. This means:
- Ollama inference and ComfyUI image generation compete for VRAM.
- A large model load in Ollama can starve ComfyUI workflows.
- n8n workflows that call Ollama endpoints inherit the same resource pool.
The stack relies on sequential usage patterns. If you run a ComfyUI workflow while Ollama is serving a chat session, one will block or fail depending on available VRAM. There is no scheduler or queue to serialize GPU-bound tasks across services.
This is acceptable for single-user homelab setups where you control the workload. It breaks down if you expose the stack to multiple users or run concurrent agent workflows that both need GPU access.
Docker Compose Orchestration and Startup Order
The Docker Compose file uses depends_on with health checks to enforce startup order:
- Ollama starts first and waits for the inference endpoint to respond.
- Open WebUI starts after Ollama is healthy.
- n8n and ComfyUI start in parallel after their dependencies are ready.
Health checks are HTTP-based. Ollama exposes /api/tags, Open WebUI checks /health, and n8n uses /healthz. If a service fails its health check, dependent services do not start.
This works for initial deployment but does not handle runtime failures well. If Ollama crashes after the stack is running, Open WebUI does not restart automatically. You must manually restart the stack or implement external monitoring.
The installer includes a lifecycle recovery test that simulates service crashes and verifies restart behavior. This is part of the release validation gate, but the recovery mechanism is Docker's default restart policy, not a custom orchestrator.
Authentication and Secrets Management
ODS does not use a centralized identity provider. Each service manages its own authentication:
- Ollama: No built-in auth. Exposed on localhost by default.
- Open WebUI: Username/password stored in SQLite. No SSO.
- n8n: Basic auth or webhook tokens. Credentials stored in Postgres.
- ComfyUI: No auth by default. Relies on network isolation.
The installer generates random passwords for Open WebUI and n8n during bootstrap and writes them to a local secrets file. You must manually configure API keys if you want n8n workflows to call external services.
This is a security boundary problem. If you expose the stack to a network (not just localhost), you need to add a reverse proxy with unified auth. ODS does not include one. The docs recommend Tailscale or Cloudflare Tunnel for remote access, but those are external dependencies.
Secrets rotation is manual. There is no vault integration or automatic credential refresh. If you lose the secrets file, you must reset passwords via each service's admin interface.
Observability and Debugging Across Services
ODS includes a control dashboard that aggregates logs from all containers and displays GPU utilization via nvidia-smi or rocm-smi. The dashboard is a Flask app that runs in its own container and queries Docker APIs.
Logs are streamed from docker logs and displayed in a web UI. There is no structured logging or trace correlation. If an n8n workflow calls Ollama and fails, you must manually correlate timestamps across two log streams.
The dashboard shows:
- Service health (up/down)
- GPU memory usage
- Model list from Ollama
- Recent log lines from each container
It does not show:
- Request latency or throughput
- Queue depth for Ollama or ComfyUI
- Workflow execution history from n8n
- Error rates or retry counts
For deeper debugging, you SSH into the host and use docker exec or docker logs -f. The stack does not export metrics to Prometheus or send traces to Jaeger. You are working with raw container logs and manual inspection.
Zero-Prereq Bootstrap and Hardware Detection
The installer is a shell script that detects GPU hardware, installs Docker if missing, downloads models, and starts the stack. It supports:
- NVIDIA: Installs nvidia-docker2 and configures the CUDA runtime.
- AMD: Installs ROCm drivers and sets HSA environment variables.
- Strix Halo: Uses AMD ROCm with specific kernel parameters.
The installer checks for existing Docker installations and skips steps if dependencies are already present. It downloads default models (e.g., llama3.2:3b) from Ollama's registry and waits for the download to complete before starting Open WebUI.
Model downloads are not resumable. If the installer crashes mid-download, you must restart and re-download. The installer does not verify checksums or signatures for downloaded models.
Hardware detection is heuristic-based. It checks for /dev/nvidia0 or /dev/kfd and assumes the appropriate runtime. If you have multiple GPUs, the installer does not let you select which one to use. All GPUs are passed to all containers.
Deployment Trade-Offs: Monolith vs. Cloud-Native
| Dimension | ODS (Monolithic Local) | Cloud-Native Agent Platform |
|---|---|---|
| Resource Isolation | Shared GPU, no quotas | Per-service GPU allocation or queuing |
| Authentication | Per-service, manual secrets | Centralized IAM, SSO, RBAC |
| Observability | Container logs, manual correlation | Distributed tracing, metrics, APM |
| Failure Recovery | Docker restart policy | Kubernetes health checks, auto-scaling |
| Network Exposure | Localhost or manual tunnel | Load balancer, ingress, TLS termination |
| Model Management | Manual download, no versioning | Model registry, A/B testing, rollback |
| Cost | Hardware upfront, no recurring | Pay-per-use, no hardware management |
ODS optimizes for privacy and zero recurring cost. You own the hardware and the data never leaves your network. The trade-off is operational complexity: you are responsible for backups, updates, security patches, and capacity planning.
Cloud-native platforms handle those concerns but introduce vendor lock-in, egress costs, and latency for on-prem data sources. ODS is a good fit if you already have GPU hardware and want to avoid cloud dependencies.
Failure Modes and Operational Gaps
GPU memory exhaustion: If Ollama loads a 70B model and ComfyUI tries to run a diffusion workflow, one will OOM. The stack does not detect or prevent this. You must manually unload models or restart services.
Service dependency deadlock: If Open WebUI starts before Ollama finishes loading a model, the UI shows connection errors. The health check prevents this during initial startup but not during runtime model changes.
Disk space: Model downloads and ComfyUI checkpoints consume hundreds of GB. The installer does not check available disk space before downloading. If you run out of space mid-download, the stack fails silently.
Network port conflicts: The installer assumes ports 8080, 5678, and 8188 are free. If another service is using those ports, Docker Compose fails to start. The error message is generic and does not suggest remediation.
Update path: ODS does not include an in-place upgrade mechanism. To update, you pull the latest Docker images and restart the stack. This works for stateless services but risks data loss for Open WebUI (chat history) and n8n (workflow definitions) if you do not back up volumes first.
Code Example: n8n Workflow Calling Ollama
Here is how an n8n workflow calls the local Ollama instance to summarize text:
{
"nodes": [
{
"name": "HTTP Request",
"type": "n8n-nodes-base.httpRequest",
"parameters": {
"url": "http://ollama:11434/api/generate",
"method": "POST",
"bodyParameters": {
"model": "llama3.2:3b",
"prompt": "Summarize this: {{ $json.text }}",
"stream": false
}
}
}
]
}
The workflow uses the Docker Compose service name (ollama) as the hostname. This works because all containers are on the same Docker network. If you move Ollama to a different host, you must update the URL and handle authentication.
The workflow does not retry on failure or check if the model is loaded. If Ollama is restarting or the model is not available, the HTTP request times out and the workflow fails.
Technical Verdict
Use ODS when:
- You have GPU hardware (NVIDIA, AMD, or Strix Halo) and want to avoid cloud costs.
- You need privacy-first AI workflows where data never leaves your network.
- You are comfortable with Docker Compose and manual service management.
- You run single-user or low-concurrency workloads where GPU contention is rare.
Avoid ODS when:
- You need multi-tenant isolation or per-user resource quotas.
- You require centralized auth, SSO, or compliance logging.
- You want automatic scaling, failover, or distributed tracing.
- You need production-grade observability or SLA guarantees.
- You do not have GPU hardware or want to avoid hardware management.
ODS is a homelab stack, not a production platform. It exposes the plumbing of local agent deployment: shared GPU resources, per-service auth, manual secrets, and container-level observability. If you understand those constraints and can work within them, it is a fast way to get LLM inference, workflows, and image generation running on your own hardware.
Top comments (0)