We have a whole lot of web apps and services at work - most of them are not internet-facing as they are internal 'admin' type tools. At the moment we run them as fairly 'dumb' KVM virtual machines (dumb in the sense that there's no automatic fail-over and no shared storage so each vm is a disk image on the server it's running on).
I've been working on making the apps run inside of docker containers - initially just as a way of avoiding the annoying 'this needs PHP 5.3, that needs PHP 7.2, this needs python 2.7 + ...' issues. But I started getting more into the idea of using Docker Swarm or Kubernetes to give a more 'seamless' deployment process and allow for some kind of automatic fail-over of the apps.
After quite a lot of reading and experimentation - it feels like kubernetes is overkill for our needs. We basically have 10-20 apps running - and the world doesn't end if they're offline for a short while. The complexities of kubernetes seem quite high compared to Docker Swarm. On the other hand - it doesn't feel like Swarm is winning out in the container world.
Anyway - my current thinking is to go with the following setup :
- 3xServers running docker swarm (we have two nearby physical sites)
- A virtual IP managed by something like keepalived to be the primary 'in-road' to the cluster
- traefik to act as the swarm ingress
- Probably ceph as a storage layer
The end goal is that if a server running App-X goes down, then App-X is started on another server and traefik takes care of routing to the new instance. If the primary server goes down, the keepalived brings up the IP address on another box and the traefik instance there starts handling the routing to the apps.
Most of the tutorials & guides I've found are either 'running hello world in swarm!' or 'bringing up 10,000 pods inside GKE!' with very little in the 'I'd just like stuff to kinda work if the power cord is pulled out of a box by accident' kind of thing ;-)
I'd be really interested in anyone's thoughts. I'm hoping to try and document the process to fill in the 'bigger than hello-world, smaller than Walmart' gap I seem to be falling into ;-)
Top comments (2)
What you've described is feasible however I would look at using multiple replicas to prevent container creation and downtime. With either solution, you have no need to failover.
Honestly, Kubernetes is complex, but it's also the future. Learning it now will only serve you later and once you have the hang of it, it's actually awesome. Swarm is great, but you're 100% correct, it's lost the war. Good luck!
Hi,
Yeah - I'd be going with replicas for the applications and using rolling deployments with them to make the fallover work more nicely - at least for the apps I care about (there are quite a few metrics/stats things that really don't matter much so I might not bother with those).
I hear what you say about kubernetes - and it's one of my worries about going with Swarm. I'd really like the simplicity of swarm with the community of kubernetes ;-) Part of my worry with k8s is the pace of development - not of the features themselves so much - but that almost every document & tutorial about it seems to be out of date the moment it's written. One tutorial will say to use X-method for ingress, the next tutorial explains that X-method is so last week and we should all now be using method-Y ;-) For my underlying architecture I quite like it not to move so fast - years of looking after Solaris and AIX boxes will do that to you I guess ;-)
Nothing's set in stone yet - so I'm keeping my options open w/regards k8s vs. swarm. But as you say - I'm painfully aware that k8s seems to have won out and I don't want to be sitting in a year re-building it all ;-)
Thanks again for your reply :-)