Kubernetes (k8s) is awesome. I am a fan. I like it so much I have a k8s-themed license plate on my car, but I cringe every time I see a blog post o...
For further actions, you may consider blocking this person and/or reporting abuse
Absolutely disagree, it's really not that hard and it provides a consistent way of doing everything.
Starting with something "simpler" then causes interuption to switch to completely different things.
The choice should more involve the skills of initial team than the size of company/budget/project.
Also starting with microsevices is much easier than splitting a monolith later.
Now I've heavily used K8s a lot I'm wishing I got into it sooner but everyone kept incorrectly saying it's over complex when it really isn't!
"Also starting with microsevices is much easier than splitting a monolith later."
It's all about picking the tradeoffs you want to deal with. Personally, I'll take "splitting up the monolith" over "re-architecting a bunch of incorrectly scoped microsercvices" any day of the week.
I agree Craig. K8s makes it so much easier for small projects because you don't have to worry about infrastructure as much. It makes CI much easier.
Well, i disagree with you, the infrastructure behind k8s is so complicated to deploy and administrate, much more complicated on-premise than on Cloud.
Where is the difference? Where is the easier? When the dev guy need to make a deployment of an app or microservice don't need to ask the infrastructure team for servers, storage and or anything... Just make a deployment on the kubernetes cluster and that's all.
But let me say you that the infrastructure is there, with higher level of complexity.
This is especially true when you over engineer K8s
Eg. Using it for the DB, persistent disks everywhere etc, all things that could be offloaded to 3rd party cloud services
👏 Don't 👏 run 👏 stateful 👏 services 👏 in 👏 k8s. 👏
Well, deploying stateless microservices will not dissapear all infrastructure required to run k8s.
The discussion is about k8s does not require complex infrastructure...
👏 Don't 👏 run 👏 stateful 👏 services 👏 in 👏 k8s. 👏
I'm new to them, but I feel like I'm about to do that exact thing. What is your solution to it?
Run them outside of k8s in traditional VMs or services like AWS RDS.
That applies to standard tools like databases, not specific ones for your use-case specially if you are building lot of workers and doing cpu heavy computation tasks. i.e not every app is CRUD like.
I disagree with this, but not because kubernetes isn't hard.
I built a K8s system for a startup and it was definitely challenging, but once it was done we had minimal down time, when stuff did go down it self healed, and (most importantly) I was able to teach our juniors very quickly how it worked, what they needed to care about and what was going on in the background that they didn't need to worry about.
A lot of this was simplified by automating away deploys etc.
Using K8s was a small investment that paid off very quickly, and if I were to go to a startup again, I would definitely use it again.
Agreed. K8s - for all its amazingness - can introduce unnecessary complexity early on. Assuming the eventual need, I'd go for something like docker-compose|swarm early, which provides a clean migration path if the time comes - but easy-peasy till then.
Docker-compose yes (for small scale stuff). Docker-swarm definitely not. We ran into some show-stopper bugs with it and docker weren't interested in fixing them. We're using k8s now. Complex, yes, but rock solid.
You got a 🦄 for
Very nice.
I appreciate your perspective. From the developer side k8s seems really great but from the sysadmin side it scares me. There seems to be a great deal written about how to use k8s but I have been unable to find much about the administration side. Any good places to look for that information? Most things I find tells you how to spin up the cluster and then moves on to just deploying things.
When I can't find documentation for something, I usually take that as a sign that that whatever it is, I shouldn't be doing it. I'd recommend avoiding running your own clusters entirely and use hosted-k8s. It's good to have the theoretical knowledge of how the control plane works and such - all things you can learn from github.com/kelseyhightower/kuberne... - but the circumstances where running your own clusters makes sense are very niche.
For day-to-day admin, the k8s docs are probably your best resource. Past that, it's a lot of delving into Github repos looking for references to things in random Readme files and source code. I have occasionally found some helpful things on kubedex.com/
Thanks for these pointers. My conclusion at this point is the same. However the benefits look really nice on the developer side so I’d really like to understand the sys admin side also. My current employer likes to keep internal tools on internal networks. Maybe in the future that will change.
The startup I work for uses k8s AND we use it for stateful services (neo4j and ElasticSearch).
To provide a bit of detail, we're on GCP, and use GKE. We use helm to roll out new deployments as part of our CI (via circleCI). All of our data is in google cloud storage, so we could recover in the event of a persistent disk failure (it would be awfully inconvenient though). Our system is a bit different than your standard database where you're reading and writing all the time. We're usually read-only, but we periodically recompute new databases, and atomically switch our services to point to those. This is quite easy with k8s.
There is certainly a learning curve when using k8s, and I wouldn't recommend it for everyone. However, once you've learned the basics I find it makes it a lot easier to experiment and try out new things.
A great read!
Question, do you think Docker-Compose suffers from similar disadvantages? is this a problem with container orchestration in general?
Also, I bet K8s might have left you with some scars, any battle stories for us?
💻
I’ve only used docker-compose for local development. It works fine for that and is an order of magnitude simpler than k8s. I don’t really think of it as being container-orchestration and more as “portable dev environment” but I may be ignorant of its capabilities.
Any type of orchestration adds complexity. I’d avoid it in general unless there was a clear, painful need.
No real k8s battle stories, but this was an interesting problem: dev.to/liquid_chickens/kubernetes-...
100% agreed with the author. This image says it all:
We're kinda comparing different levels of abstractions here. K8 has its complexities but so does managing VMware vSphere, IBM Websphere or Pivotal CloudFoundry for apps of any complexity -- those are better things to compare in my opinion.
The disruption has always been that vanilla K8 has lifted some traditionally silo'd infrastructure concerns to some intersection of Dev and Ops.
Yes, the old way of "here's a box, IP address and SSH key, install your app" works to a point, but someone has always had to manage the patching, firewall rules, storage & backup, monitoring, load balancing, etc. None of those concerns go away with public cloud, and enough people are still on-prem where silo'd Ops groups are normal. So, Ops groups hack away with Chef and Puppet to make lives easier and automate, but they have to do it without standard APIs or even sane CLI tooling to manage the infra (yes, it's awful).
K8 is nothing more than a set of open-source, vendor agnostic APIs and constructs from which to build platforms. Most start ups do not need to build bespoke platforms. Platforms that are built atop OSS tooling and standards will make a lot sense for a lot folks.
THANK YOU! Way to often a tech is touted as
this will fix all your problems; when in reality tech is a series of trades-offs. Your article explains those trades nicely.Do you recommend starting with deploying docker containers in another way, or skipping docker entirely.
To me Docker seems too valuable from a development perspective to pass over, even initially.
🤷 If it's solving problems for you and provides more value than it costs, go for it.
I have seen a number of folks using Docker just "because" though, and that's obviously not the best idea. Abstraction for the sake of abstraction...
Hi Chris, where do you run your production k8s? Have you tried the Hosted-k8s (EKS, AKS, GKE)? Was wondering which is best. Thanks.
Trade offs with all. Google has the most fleshed out k8s support since it was born there. I dislike a lot of the Googlishness of GCP, but that’s just personal preference. Currently run in EKS and like it well enough. I’m one of the few people who really like Cloudformation and think it is far more valuable than most people give it credit for, so having clusters and such defined with CF is appealing. Haven’t used AKS, but I straight up loath the Azure console, so I’m the wrong person to ask.
Thanks Chris, totally understand what you mean by "Googlishness of GCP" :D
On my previous job, we were running k8s on premise and it was very difficult since we had to manage the cluster itself and services running on top of it. I agree with the complexity and my point of comparison was docker swarm. We used terraform + ansible to provision the VMs, k8s, and containers. You have to really want it if you wish to use k8s.
Hope you can share your experiences with k8s in your next article so we can learn from it as well.