DEV Community

lucanu
lucanu

Posted on Originally published at coldriver8.gumroad.com

Apple Silicon Cluster Kit: Ansible + Docker for Unified GPU Pool: A Practical Guide

#theuserwants4devtotagsfor

You have three Macs gathering dust — a Mac Studio in the office, a MacBook Pro in your bag, and an M1 Mini humming quietly in the rack. Each one sports a unified memory architecture that LLMs love, but they sit isolated. You could stitch them into a single GPU pool for vLLM or llama.cpp, but the path is littered with SSH key nightmares, Docker context switching, metal driver mismatches, and Ansible playbooks that rotted on GitHub two years ago. Three days later you are still debugging ssh_exchange_identification errors instead of serving tokens.

What Is Inside

  • Ansible inventory & playbooks that bootstrap every host — macOS version checks, Metal/MPS validation, Docker + Colima/OrbStack install, SSH hardening, and userland tuning — in one idempotent run.
  • Docker Compose stacks pre-wired for vLLM (OpenAI-compatible API), llama.cpp server, and Ollama, each exposing a unified /v1/chat/completions endpoint behind Traefik.
  • Unified GPU pool script (cluster-pool) that presents every Mac as a single logical device to your inference workload, handling model sharding and KV-cache offload automatically.
  • Runbook & troubleshooting guide covering thermal throttling, memory pressure, network MTU tuning, and rolling updates without downtime.

Who This Is For

  • ML engineers who want local 70B-class inference without cloud bills.
  • DevOps folks tired of hand-rolling Kubernetes on ARM just to run a quantized model.
  • Hobbyists with a drawer of Apple Silicon who refuse to let VRAM go to waste.

Actionable Preview 

git clone https://github.com/your-repo/apple-silicon-cluster-kit
cd apple-silicon-cluster-kit
cp inventory.example.yml inventory.yml   # add your host IPs
ansible-playbook -i inventory.yml site.yml
cluster-pool up --model meta-llama/3.1-70b-instruct-q4_k_m
curl -X POST http://cluster.local/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"pool","messages":[{"role":"user","content":"Hello"}]}'
Enter fullscreen mode Exit fullscreen mode

First tokens stream in under 30 minutes from a cold start.

Get the Kit

Stop stitching. Start serving.

EUR 16.99 → https://coldriver8.gumroad.com/l/xayfxz

AI-assisted content - reviewed by author.

Top comments (0)