Hey everyone, this is my first post 👋
Today, I'm bringing you a quick run-down of how I created my Homelab Kubernetes cluster (again) from scratch, utilising Proxmox Virtual Machines, Kairos (OS), Terraform, and deployed K3s on top of this.
PS: no code as of yet, but I'm cleaning it up to hopefully share with the community!
The Problem
I needed a test bed for dumping all my wacky tools, horrible spaghetti code, and other OSS nonsense. Yes, I could use kind (https://kind.sigs.k8s.io/) on my own computer, but that would defeat some of the purpose of the things I'm doing: testing production-like environments, testing multi-node cluster things, testing for longer-term viability etc. Kind could probably do all these things, but where's the fun in that? ;)
The Plan
My goal was to always take my sweet time, I jumped around a lot between different ideas, and often changed my mind on what tools to use. I started with K8s, bare metal, but realised that wouldn't be much different to what I have on my laptop today... I also thought about using TalOS, but ran into some issues with customising things, and found the lack of SSH tough (it's not a bad thing, I am just not very used to that). I spent a solid 20 hours across about 2 months, on-and-off, trying to get SUSE Harvester running... More on that below. I wanted to learn as much as I could on this journey and create a short documentation to explain what I've done, and the journey so far.
I strongly believe one of the most important tools in a developer's toolkit is knowledge; whether you write down your learnings, journal things, create blog posts, tweet, keep it in your memory bank, any of these things will help you to generate and store knowledge, making you more of an independant thinker. In the AI-era, I've found this extremely helpful, as I can both talk to AI and humans, bridging the divide between people and machines.
The Solution
I made a flow-chart which explains some of the core components and the process for creating the Kubernetes cluster (ultimate goal), and all the little pit-stops we make along the way. Just like an Olympic race, it's important that all the preparation is undertaken before sprinting down the track. Just like a heat race, I've tried to write tests where I can, or at the very least, identify gaps where tests would be beneficial in identifying problems early on.
Background - Why I chose these tools
Feel free to skip this, it explains how I came across these tools and why I use them.
- Homelab: what better way to tinker with things than having a giant, poorly ventilated Dell Tower server in a cupboard?! Cloud would have been a nice option, but I want to keep costs low, and be as efficient as possible. In my opinion, it's easier to learn if you tinker yourself too.
- Proxmox: I played with a few options, including SUSE Harvester, but I had issues orchestrating Virtual Machine creation. In the end, I opted for something a bit more bare-bones in Proxmox. Open-source was an absolute must for Homelab activities.
- Kairos: A customizable, immutable, and optimised operating system designed for edge (https://kairos.io/). I found the concept very cool, and it had a few more bells and whistles when compared to TalOS which were great for a Homelab. TalOS had a much better security posture, but was a bit more difficult to customise, so I opted for Kairos.
- Terraform: the ubiquitous infrastructure-as-code language and software combo that gives a nice balance of customisability, flexibility and abstraction of resources. I created virtual machine "templates" (terraform modules) that contain the base, and then layered my Kairos configuration, cloud_init, and other customisation on top of that. Whilst terraform's state management, drift detection, and statefulness are a bit lacklustre, there's bliss in being unaware of these facts - and this makes terraform perfect for my use-case (Homelab, simple, relatively easy to understand). Shoutout to "bpg" for the incredible terraform-provider-proxmox that does the heavy lifting! https://github.com/bpg/terraform-provider-proxmox 🎉
- K3s: a more simple kubernetes distribution, great for me testing various things, new CNIs, new CRDs, new applications and OSS tools/apps. I didn't need the hassle of configuring a full k8s cluster, so I opted for K3s. It's also bundled into Kairos rather neatly, which was a big plus.
SUSE Harvester
Super cool, and I think SUSE have made nice, simple, easy-to-use UIs. I tried to get the Harvester > Rancher > Longhorn stack going, but had some issues with Rancher, that were holding me back a bit. I managed to get the basics, but had some issues when using the in-built provider in Harvester that auto-magically spins up VMs (never figured it out, NIC issue, or maybe something to do with virtualisation?). It wasn't playing nice with my hardware, and I didn't have the knowledge or stamina to try to resolve it. I did try... I considered posting in their official Harvester Slack, but opted to pick Proxmox for the better home-enthusiast support. One day I'll revisit Harvester when it's more mature, I'll probably install it on another disk or something.
Top comments (0)