We have a whole lot of web apps and services at work - most of them are not internet-facing as they are internal 'admin' type tools. At the moment we run them as fairly 'dumb' KVM virtual machines (dumb in the sense that there's no automatic fail-over and no shared storage so each vm is a disk image on the server it's running on).
I've been working on making the apps run inside of docker containers - initially just as a way of avoiding the annoying 'this needs PHP 5.3, that needs PHP 7.2, this needs python 2.7 + ...' issues. But I started getting more into the idea of using Docker Swarm or Kubernetes to give a more 'seamless' deployment process and allow for some kind of automatic fail-over of the apps.
After quite a lot of reading and experimentation - it feels like kubernetes is overkill for our needs. We basically have 10-20 apps running - and the world doesn't end if they're offline for a short while. The complexities of kubernetes seem quite high compared to Docker Swarm. On the other hand - it doesn't feel like Swarm is winning out in the container world.
Anyway - my current thinking is to go with the following setup :
- 3xServers running docker swarm (we have two nearby physical sites)
- A virtual IP managed by something like keepalived to be the primary 'in-road' to the cluster
- traefik to act as the swarm ingress
- Probably ceph as a storage layer
The end goal is that if a server running App-X goes down, then App-X is started on another server and traefik takes care of routing to the new instance. If the primary server goes down, the keepalived brings up the IP address on another box and the traefik instance there starts handling the routing to the apps.
Most of the tutorials & guides I've found are either 'running hello world in swarm!' or 'bringing up 10,000 pods inside GKE!' with very little in the 'I'd just like stuff to kinda work if the power cord is pulled out of a box by accident' kind of thing ;-)
I'd be really interested in anyone's thoughts. I'm hoping to try and document the process to fill in the 'bigger than hello-world, smaller than Walmart' gap I seem to be falling into ;-)