DEV Community: Tyler Auerbeck

VSCode Starter Pack: Getting Started with Python For Data Science

Tyler Auerbeck — Wed, 30 Oct 2024 20:42:08 +0000

Occasionally, I find myself teaching some courses on a number of topics. Most recently, I’ve been teaching a course around getting started with Python and Data Science concepts.

Wait. Didn’t you say that already? We get it. You teach classes.

Yeah, I know. I said that before. But then my brain got real hyped about ways of teaching and took me in a different direction. But we’re back! And now we’re gonna talk about what I actually meant to write the first time

And what’s that?

Well, setting up a nice starter environment for a Python Data Science class of course! I’ve taught classes like these in a few different places and in a few different ways and it’s always bothered me a little bit that it’s done in a way and generally with tooling that I’ve just never seen or heard of before.

Why Teach What We Don’t Use?

Listen, I get it. We’ve got a lot to cover. We just want to make things easy. We will just give you a ready baked environment to get started in and we’re off the the races. Which is probably okay. They can focus on the topics at hand. But that’s good until either the next course OR even worse, their first job. How are they going to know how to get started on their next project without actually having to set it up themselves?!

I don’t know about you, but speaking for myself, a developers tooling is fine tuned to the developer themselves. We may show someone how to get started, but that environment eventually becomes a reflection of that persons brain and their way of thinking. With enough time and practice, it’s an extension of their brain itself.

So that’s why I believe in introducing students of all experience levels to tools that they’re actually going to see. Let them get used to and familiar with them now. Make sure they know how to navigate with them, make sure they know how to discover and navigate the community and documentation and most importantly, make sure they know how to make it their own.

Easy Bake Oven, But For Code And Data

Now this isn’t to say it’s best to just toss everyone into the deep end (although sometimes it is; an exception for every rule). My thinking here though is give them the baseplate. Give folks the basic ingredients and let the kids cook, as they say.

So for this particular example, what we’ll get started with is VSCode. I’m not going to be here to argue about what’s best, the value of some other IDE or why I’m not forcing anyone into starting out in VIM. What I’m here to do is to show that you can (and should) get started with a well known IDE and get it set up in a way that shows you how to actually accomplish the goal you want to complete.

Our Goal:

Have a basic environment where we can begin completing Python based data science exercises, focusing initially on primarily on Pandas.

So let’s get started

Installing VSCode

This first bit is about as plain as it gets. We need the tool. So let’s go get the tool. You can pick up the installer VSCode for your platform here. This is a standard installer, so just launch it and follow the standard instructions to get it installed. Once you’ve got it installed, you can launch VSCode and you’ll have a regular, degular install that looks like this:

A very empty, freshly installed VSCode installation

My install will reflect a MacOS setup, but for the most part after the install everything should be the same regardless of the platform you’re running on.

The Plugins

That being said, a normal install of VSCode is kind of like buying a thing, and then just leaving it in the box. You know it can do so many things! But you’re just leaving all of that untapped potential sitting there, not serving you. A waste. So let’s make sure we don’t make that mistake.

Plugins can be the key to unlocking a ton of velocity in VSCode. And the nice (but sometimes very sharp) bit of this is that it mostly comes at the click of the button. There are times when installing a plugin can reduce a ton of overhead in your developer workflow. That being said, because of the ease of installation, it can also lead you down a deep dark hole of plugin FOMO, which inevitably results in the bloat of your IDE. So buyer, beware. Start with what you need and build from there!

In our case, let’s remind ourselves of our goal:

Python Environment
Data Science Starter Kit

So to begin we’ll find ourselves the following plugins:

Python
Pylance
Python Debugger

This will give us the Python environment that we’re desiring. A majority of the other tooling that we’re interested will be handled via Python itself, but one particular set of plugins will be helpful to us as we start some of the DS work. Specifically, we’ll want the following Jupyter plugins:

Jupyter
Jupyter Keymap
Jupyter Slideshow
Jupyter Cell Tags
Jupyter Notebook Renderers

To grab all of these we will want to navigate to the plugins tab of our VSCode window and we can search for them by name:

Tracking down our Python plugins

On the left hand side of your window, you’ll see the little building blocks icon which is where you’ll find your plugins. From there, you can see a search bar where you can type in the name of all of the above plugins that I’ve mentioned. You’ll see the install button and you can go ahead and click on them to begin the installation. Once it’s finished, you may (or may not) be prompted to restart VScode. If you see no restart prompt, you’re good to go!

We Have The Technology

Alright, we’ve got the tools and presumably we’ve got the time. So what better time to get started than now! However, this is really just the beginning. Before we can write any code, we still need the packages that will unlock all of our data science needs. To do this, we’ll use a few different tools.

Virtual Environments

The first, and maybe most important, is configuring a virtual environment. Why do we want a virtual environment. To save yourselves the pain of forever fumbling between dependencies and potentially causing yourself a ton of grief by interfering with your local system packages. I’ve experienced no greater pain than having one project that works, going to work on another and then coming back to my previously perfect project and finding out, nothing works and the house is on fire because suddenly none of my packages know how to work together any longer.

This is exactly what a virtual environment is for. While this of course can be done directly from the command line, since we’re talking about VSCode we’re going to focus just on how to do this through the UI.

The first thing we need to do is get ourselves some space to begin working. We’ll use VSCode to create a new directory to begin creating our python files in. After you’ve launched VSCode, you can just click the Open prompt and it will pull up your file explorer.

Creating a space to work in VSCode

From here, you can go ahead and click the New Folder option (or otherwise navigate to somewhere you would like to create a new folder at) and create a directory of your choosing. For our purposes, we’ll name ours starter-pack . Once the directory has been created, go ahead and click on it and then click Open .

Creating a directory in VSCode

From here, you’ll see your folder appear in the file explorer in VSCode (that’s the icon that looks like multiple pages folded over each other). The directory is empty for now as we’ve just created it, but let’s go ahead and solve that problem.

Our empty directory in the VSCode file explorer

We’ll go ahead and click on the New File icon, which is the first icon next to the name of our directory. You should see your cursor start flashing and you’ll be able to type a filename of your choosing. For our purposes, let’s start with main.py . We’ve go ourselves a directory to work in and we’ve stubbed out an example file. WE’RE ALMOST THERE!

At this point, we’re ready to get our virtual environment up and running. This is where our plugins will begin to come in handy. Once a plugin is installed it can come bundled with any number of pre-set actions. In the case of our Python plugin, it provides a set of actions to allow you to set up a virtual environment. The way that we gain access to these actions is through the Command Pallete . You can find this under the View menu, alongside the keyboard shortcut you can use to automatically pop it up. However you decide to access it, once it pops up, we can begin typing Python: Create Environment and you’ll begin to see the action that we want to launch. Once we see that, go ahead and click on it to begin.

Creating a Python virtual environment

Once launched, you’ll have two options. You can create a virtual environment with venv OR you can create an environment with conda . What you’ll find is this will really come down to your preference. For our purposes, I’ll stick with venv because it’s what I know. Go ahead and click venv and you’ll then be prompted to choose a Python interpreter. Depending on the number of versions installed on your system, you may have a number to choose from. It’s generally safest to go with the latest to ensure you’re developing with an up to date version of Python. However, if there’s a specific version of Python that you need to target, feel free to proceed with that one as well.

Selecting venv as our preferred virtual environment

After we’ve got that sorted, you should see a small loading bar at the bottom right hand of your editor. But more importantly, we should see a .venv folder pop into existence in our file explorer. We don’t have to worry too much about the specifics of what’s in this folder — but just know that VSCode has activated this virtual environment for you and Python is now using the contents of this directory. It includes the binaries that you’ll be referencing (python,pip, etc.) and any of the packages that you will go on to install.

One additional step (although unnecessary, but I highly recommend) is updating your settings to automatically activate your virtual environment when you launch a terminal. This can be done by navigating to Settings . Once this screen pops up, you can begin typing python.terminal in and a number of settings should rise to the top. The ones that we’re interested in is Python > Terminal: Activate Env In Current Terminal and Python > Terminal: Activate Environment . In my experience, without these checked you can sometimes find yourself attempting to do something and then realize that your environment actually hasn’t been activated. With these checked, it should take care of this for you. You may need to restart your editor after making these changes.

Configuring venv activation by default in the Settings menu

Note: If you ever find yourself having to manually activate your environment, it’s not a big deal. You can just run source .venv/bin/activate and you should be in good shape.

Packages? What Packages?

Ah, yes. Those pesky packages that we actually need to begin doing our work. What we have at the moment is just a clean environment to begin doing our work in and the promise of not blowing up our laptops. We’re not even at the point of having the tools we need for the job yet! But now we can begin to solve that. The first time we go to set this up, we’ll be doing some additional leg work to get this all set up. To do this, we’ll need to drop into a terminal.

Note: It’s going to be okay. I promise. If you’re not familiar with the terminal, it can’t hurt you. We’re going to just start with some basic pip commands to get what we need. We can make sure you two become friends another time.

We’ll first start by launching our terminal. Again, you can find this under the menu Terminal and choose New Terminal . You should also see the shortcut to launch this a bit quicker in the future. You also could use the Command Palette like we did earlier. You’ll find there are usually many ways to do the same thing and you’ll slowly find the way that you’re most comfortable working. Do what works for you!

With our terminal now open, let’s install some packages. As a reminder, the second part of our goal here is to build out a data science toolkit that will allow us to quickly jump into a number of DS exercises. So the packages we’ll jumpstart our project(s) with will be used for that purpose.

With that said, the packages we’ll be interested in are:

pandas
seaborn
numpy
requests
matplotlib

Note: This should not be considered an exhaustive list. Again, starter pack. This will give us just enough to get started with some initial exercises. What you’ll find as you begin to do more and more is that you’ll find that either a) some of this doesn’t fit your needs or b) it does and actually you want to add more to it. That’s great! Keep what works. Toss the rest! This is most importantly an exercise in building out a toolbox that could work for you. What you fill it with will be completely up to you!

With that rant complete, let’s go ahead and get these packages installed. We’ve got our terminal open, we’ve got our list of packages. Now all we need to do is feed them to pip and we’ll get these installed. We can do this all at the same time with:

pip install pandas seaborn numpy requests matplotlib

You’ll then see a bunch of text pass you by telling you that all of these packages have been installed. You’ll notice we didn’t pass in any particular version. At this point, we’ve done that on purpose. We want the latest. However, there will be a time when you want to make sure that you continue to build to certain tooling.

To make sure we’ve got a list of what exactly it is we pulled down, we can actually use pip to generate ourselves an inventory so that future us can take this same code, in a new environment, and end right back up in the same place as we are today. To do this, we can run:

pip freeze > requirements.txt

If we take a small sample out of that file, we’ll see something like:

requests==2.32.3
seaborn==0.13.2

We can see that it tells us the exact version of the package that has been installed. We could then use therequirements.txt in any other environment (or even the same environment if we trash and recreate our virtual environment) to recreate everything without running those individual pip install commands. We could instead run the following (with our generated file) and get the same result:

pip install -r requirements.txt

And just like that, we now know how to setup a space to start developing in and cobble together the tools we need to get started. We also have the ability now to recreate that same environment over and over again should we like to. Almost like it would be cool to use it as a template for any future projects or exercises we might want to get into.

Ah, a template. This was the plan all along.

GitHub Template

And this brings us to the nice, bright pink bow we can slap on all of this. The worst part of any project tends to be getting started. You’ll sit there and think to yourself:

I do this. Every time. Why can’t I just click a button and deal with all of this scaffolding?!

Well, as it turns out, you can. You could keep things simple and just do the ol’ copy paste from this directory into a new directory each time. But.. that’s kind of gross and only works from your local machine. What if instead we made this a template that could be accessed from anywhere that we could grab it from GitHub?

I’ll save the lecture on the importance of source control for another time. But that aside, being able to get a fresh copy of your template with a few taps of the keyboard can save a ton of time. All we need to do is create a repository in GitHub and commit our main.py and requirements.txt files. We’ll save ourselves the pain of the terminal for the moment and look at this via the UI.

Assuming that we already have a GitHub account and are logged in, we can visit https://github.com/new to create a new repository. For our purposes, we’ll call this example-starter-pack , but you can call it whatever you would like. Click Add a README file as well and then click Create Repository .

Creating a new repository in GitHub

Once we’ve got our empty repository, it should drop you on a screen that looks like this:

A GitHub repository with an empty README

Now all we need to do is add the two files we mentioned above main.py and requirements.txt . So now all we need to do is click the Add file button that we see above and choose either Add file or Upload files . Since main.py is just going to be an empty file, we can just click Add file , give it the title of main.py and click Commit changes . Since we’re setting this up as our own template repo, it’s okay to merge directly to main . We can talk about the peril of committing to main another day ❤.

At this point, we’ll have something that looks like the following:

Our GitHub repository with our empty main.py file

So now our final piece will be adding the requirements.txt file. Since this is just a handful of lines, let’s go ahead and copy those from VSCode to our clipboard (a little copy pasta never hurt anybody). Once we’ve copied those contents, let’s go ahead and add another file just like we did for main.py . This time we’ll be configuring the name of the file AND pasting the contents into that file.

Adding our inventory of python packages to our repository

With that completed, we’ll commit this file just like we did the last one and we’ll end up with all the contents that we need. You might be asking:

But what about our .venv folder ?!

While that’s important on the machine and during the time you’re working in your virtual environment, once you’re done with it you don’t need it at all! You especially don’t want/need to be committing that into source control as you’ll just be lugging a number of binaries and other packages along that you don’t actually need.

But now that we’ve got our repository in the shape that we want, there’s one last step in making it a fully fledged template directory. As it stands, you could just keep cloning this repository, making changes, and then committing it to some other repo. But again, we have better tools to make your developer workflow fly!

What we’ll want to do is click on Settings at the top of our GitHub screen

Finding the Settings tab in GitHub

Once we’ve clicked through this menu, we should a button that says Template repository right under our repository name. Go ahead and click that. Once you click it, it will immediately make this repository a template repository!

If we navigate back to the root of our repository, you’ll notice a new button that says Use this template . If you go to click on that button, you’ll see that you have the option to Create a new repository which will return you to the repo creation screen that we saw before. This will look almost identical to how we created ours manually previously, except that it nows shows that we’re using our template. It’ll ask us where we want to put it and then we can choose to create a repository as normal from there.

Just like that! No more copy and pasting all over the place. Anytime we want to jump into a new project, we can bust out our newly built toolkit and get right to business.

The End

If you’ve made it this far, you’re probably saying to yourself.

That was a lot of work. And a whole lot of words.

And I completely agree. But like with most things, a little investment up front will save you a whole lot of time in the long run. At this point, we now know how to get our VSCode environment up and running, ensure that we keep our work confined to just the location we want it to and then set up a toolkit which we can then use repeatedly anytime we want to dive into a new project.

Are there better ways? Almost definitely. Are there better tools? I’m sure you’ll continue to refine that as you dig deeper into more complex projects. But for folks just getting started or joining a new team, I’ve found this to be a nice way to remove complexity to allow them to focus on the task(s) at hand and provide them a jumping off point to begin filling out their own toolboxes. My approach has changed over time and I’m sure will continue to change over time and I hope that if you take anything away from this, it’s that yours should too! As I mentioned earlier: keep what works, toss everything else. And then keep doing that, repeatedly.

More Apprentices, Less Parrots: The Importance Of Teaching How To Think vs How To Work

Tyler Auerbeck — Wed, 23 Oct 2024 20:42:39 +0000

Occasionally, I find myself teaching some courses on a number of topics. Most recently, I’ve been teaching a course around getting started with Python and Data Science concepts.

I personally love teaching these introductory style courses. Working with folks who are just getting started in the field can be invigorating and helps me realize a ton of the things I take for granted as someone who has been doing this for longer than I care to think about now.

But, for some reason, in a number of tutorials and places I’ve been to, we seem to underestimate the audience that we’re working with. We show them how to learn in tools that I’ve never once come across in the real world. We focus on showing them how to do things perfectly instead of considering the problem and considering approaches to take to solve the problem. All of this likely in the absence of how to surface documentation, assess the value of an example or learning how to engage with the community they will find themselves working in.

This, I believe, is detrimental to learners in a number of ways. The most obvious being that they’re going to go need to learn something new once they find some of their first opportunities in the real world. While learning how to work within a specific tool is a key skill, being able to learn about how to learn to use a tool or decide whether that is the tool for the problem is equally as important.

This last piece is something that I believe should be focused on by the teacher/writer just as much as it should be for the learner. While we all may have our best laid plans about what we’re going to show our students, inevitably there will be a question about how to do something that we either don’t know or haven’t planned for. THAT’S PERFECT. Show them how to do the thing. Even better if you don’t know, walk through how you go about figuring out how to do the thing. One of two things will happen.

You will either:

figure out how to do the thing; PERFECT , student learns how to do the thing
not figure out how to do the thing; EQUALLY AS PERFECT ; student learns how to investigate how to do a thing; the places to search, the places to ask questions; how to deal with temporary defeat while trying to accomplish something

These are all equally beneficial to a new learner. These are things that they will inevitably need to know how to deal with. I think failure is an equally good teacher, especially early in the career, because it starts to build the idea that just because something didn’t work, doesn’t necessarily mean it was bad or a waste of time. The amount of time I’ve had to spend coaching junior engineers out of feeling like an imposter or being nervous to admit that they can’t do something is astounding. I’ve found that once you’ve done enough to coach them out of that shell, it can almost be like working with a different person.

SO WHY ARE WE NOT TEACHING THIS FROM THE START!?

Well, in writing, having to pull all of this together can be difficult. Our attention span these days is not great. By the time we get through the setup, we’ve probably lost 90% of readers. But, if we just hand wave to get to the exciting bit, we keep folks captured and ensure they read what we actually wanted them to. Likewise with teaching, you sometimes have to choose your battles. Is it more important to hit the notes that you want? Do you need to figure out you cover all of the specific content within a given timeline? Sure. But that burden is on us. What we need to remember is that we have a responsibility to the folks who have come to us to learn and to show them the skills they’ll need to be successful moving forward, whether they’re viewed as soft skills or not.

What’s The Fix?

Unfortunately, there’s no silver bullet or one way of writing/teaching that is going to solve this problem. But what I would encourage you to do is think out loud when writing or teaching. There is nothing someone early in their career is going to benefit from more than seeing how someone who has been doing this for some time thinks about a problem. They’ll be able to borrow from and build on ways that you wrestle with a problem. They’ll start learning where to go and who to talk to when you don’t have answers. Don’t be afraid to fail spectacularly! Talk through what you did, how it apparently didn’t work and more importantly what you do next to go find the right answer! Find folks to bring in and talk about their horror stories. Everybody can learn something from burning rubble. It’s how we learn not to set the next dumpster on fire!

In summary: build things, watch them blow up, tell everybody what you learned and how you got there. There’s still nothing wrong with unloading a half-baked idea onto the internet or ensuring that you hit specific topics that you need to throughout a course. But building the next generation of critical thinkers and problem solvers is going to be crucial. Otherwise we may find ourselves with barely functional software and nobody to figure out how to make it better.

Plight Of The VolumeClaimTemplate: How To Update Your PVC Once It’s Been Created By A Managed…

Tyler Auerbeck — Wed, 16 Oct 2024 16:01:37 +0000

Plight Of The VolumeClaimTemplate: How To Update Your PVC Once It’s Been Created By A Managed Resource

Live photo of me, each time I find a StatefulSet and the associated VolumeClaimTemplate not matching the actual PersistentVolumeClaim

Recently, I’ve been reminded of an old enemy of mine. The VolumeClaimTemplate. Now you might say to yourself, it’s YAML. It can’t hurt you. Well, that may be true. But it certainly was enough to stress eat a whole box of girl scout cookies whiled I pleaded with the universe that I wasn’t going to accidentally unravel the database that was in charge of something fairly important: making all the money.

Gather ‘round. Let me tell you a story.

There once was a tiny lil’ StatefulSet. And it ran a fairly important database. For you see, people did the things, those things were kept track of , and then this particular database allowed us to collect dollar bills for those things. Without this database, we were not able to keep track of the things and thus, no dollar bills for us.

So, whenever maintenance needed to happen, you wanted it to be real, real fast. So that you know, it didn’t impact the tracking of the things, etc., etc.

Enter: The On-call Engineers

Well, lucky for us, things were going well. Many a thing was happening. Dollars bills were flowing all around. This was great! As you can imagine, everyone was walking around high-fiving, the money was flowing. Life was good.

Except for the cranky on-call engineers. You see.. things were going a little .. too well. The things were being collected and stored, but at a very high rate. So we (yes, I am one of those cranky on-call engineers), would get woken up. Oh, you’re running out of storage? AGAIN!? Well, after more nights than we all probably cared to admit, it was time to do something about this.

Luckily for us, a nice little project called pvc-autoresizeralready existed. I love it when a project already exists! We didn’t even really have to do anything, just deploy the resizer, apply some annotations, and success. The PVC’s would get resized per the configuration we supply.

Oh.

We have to.. apply annotations.

To the PVC.

That is managed by a StatefulSet.

Hello everyone. Welcome. Let me introduce you to the villain of our story.

And why is that? Well that is because once an object like a StatefulSet creates a VolumeClaimTemplate, it doesn’t really let you touch it via the StatefulSet again.

The promise of the VolumeClaimTemplate is great. It allows you to describe the storage you want as part of your initial workload without actually having to manage the PVC itself. This is generally fine if you have a very well defined deployment and understand its storage needs almost completely. However, should you ever find yourself in a spot where you need to modify this at all, you’ll quickly run into a problem. You see, none of the fields under the VolumeClaimTemplate are fields that you are able to modify once a StatefulSet has been created. Need to resize your storage? Not happening. Just need to slap some labels or annotations on this thing? Don’t even think about it.

# statefulsets.apps "web" was not valid:
# * spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden
#

Now, this isn’t to say that there isn’t a way to get the deed done. To accomplish the things you need to do, you can directly patch the managed PVC.

kubectl patch pvc very-fake-pvc \ 
  -p '{"metadata":{"annotations": { "hello": "world"}}}'

Anything you need to add, just patch it directly into the PVC and you’re in great shape. But doesn’t that kind of defeat the purpose? If anything, it almost feels like this is worse. THIS ISN’T THE DECLARATIVE FUTURE THAT I WAS PROMISED!

Following this patch method, I now have a StatefulSet that describes my storage one way, but the reality is that my storage now doesn’t reflect the thing that I originally defined. What happens if I need to go apply this workload to another cluster? What happens if some sort of disaster happens to our cluster and we need to redeploy this workload. Under fire, I certainly don’t want to have to remember:

“oh yes, that patch command”.

When you begin to add any GitOps tooling on top of this, it becomes immediately worse. Because at this point, you’ll forever have some object that is out of sync. You could try to sync it. But then you’ll be cursed with some red nastygram telling you what I mentioned above. These fields can’t be updated. GET LOST!

Out of sync objects and red lettering as far as the eye can see

Ride The Keyboard Lightning Out Of This Nightmare

Now, there is still a way out of this mess should you ever find yourself in this situation. And let me be clear. It’s still super duper gross. You would prefer not to find yourself here. Yet, here is where we are.

The first step is what we mentioned above. Ultimately, your goal and the value you’re looking to find here is updating your PVC. So you’ll need to grab your patch command and go at it alone. The StatefulSet is not going to be here to help you. So whatever you need to patch into your PVC, you can do via this command (or if you’re feeling lazy, feel free to hop right on in viakubectl edit .

DISCLAIMER: Busting out kubectl in production environments is ideally not the first tool you’re leaning into. Humans typing on keyboards are error prone in the best of situations, so make sure you know what you’re doing and double check your work.

At this point, you’ll have accomplished exactly half of the job. The still more problematic bit is that you have a StatefulSet that is describing your PVC one way and reality reflecting another. As the error message that we’ve seen before clearly states: can’t touch this.

But, we have ways of making these particular Kubernetes objects sing. It again requires leaning into kubectl ( read disclaimer above ), but it will get the job done. This step will require us to orphan our running pods from our StatefulSet. What this does is allows the workload itself to continue running (re: NO DOWNTIME!), while allowing us to reapply the object that actually manages the workload. And when we are reapplying that object, we can make sure to update our VolumeClaimTemplate to reflect the new reality of the PVC. To do this, you’ll run the following command:

 kubectl delete statefulset <statefulset-name> --cascade=orphan

Now, if you run kubectl get statefulset , you’ll notice that the StatefulSet itself is gone. However, checking to see that your pods are still running with kubectl get pods will show you that there’s nothing to fear, your application will still be humming along just as expected.

Now, let’s be clear about one thing. While we do want to update the StatefulSet YAML to reflect the reality of the situation, nothing is actually happening to the PVC after we update our StatefulSet YAML. That’s because it’s seeing that the storage already exists so it’s not going to touch it. If the PVC was missing of course, it would create it according to the definition that we’ve provided. The process we’re going through however is purely to save future on-call engineers from having to untangle the mystery of why two definitions of a PVC don’t align.

If you’re living in GitOps world, it also makes all the red go away and everything will believe it’s synced up again.

Looking To A Better Future

I share this story because I’ve needed to walk quite a few folks through situations like this over the past few years. There are long running issues that have been open in the Kubernetes repo asking for updates for these exact issues! But, here we are in 2024, and good things may (hopefully) be coming our way soon. There have been a few KEP’s looking to address this in the past, but this one in particular seems to have some legs.

Hopefully, this workaround will let those of us in this situation continue to run those dollar making workloads that need some additional manual intervention. But If nothing else, hopefully this saves you a few extra minutes and some unnecessary heartburn along the way while we all keep our fingers crossed for a fix to deliver us from the pain.

Publishing to Dev.To From Medium

Tyler Auerbeck — Tue, 08 Oct 2024 16:02:10 +0000

A Foot In Both Worlds; Meeting The Community Where They’re At

As I’ve recently restarted slinging words onto the internet, there were questions I needed to answer before I got back to business. After deciding to stop wasting my time agonizing over a personal site, I needed to grapple with exactly where I wanted to toss these words to. In the past, I’ve gravitated towards Dev.To because that’s where all my friends were. However, I also liked the idea of Medium because I knew folks who wrote things here and the option of potentially monetizing things if I wanted to go that route in the future.

So, like any reasonable technologist, I decided I didn’t want to think about this anymore and I was going to do both. I’d use Medium as a home base and I’d also shuttle those same writings back over to Dev.

Medium As Home Base. Why?

Now why have I decided to write in Medium instead of to Medium? At this point, I’m still figuring out the how of my writing process, but at this point I’ve been bouncing between Notion and the Medium app itself. At this point, the UX of the Medium app is nice enough and allows me to just tap out some words as I’m thinking about things and then easily come back to polish them up at my computer later. So the ease of use at this point is what has drawn me to stay here instead of just publishing out to Medium as a second location.

How.To Dev.To

So now that I figured out where I’m initially publishing come content, I needed to figure out how I was going to get it to a second location at DEV. I initially thought I would need to dust off a very old GitHub Action to get this done. While I’m always up for breathing some new life into my old projects, I also didn’t want to go find out all the ways that the DEV API has probably changed since I last touched it. So I was pleasantly surprised to find out that they were going to make this much easier on me.

Trust, Dusty RSS

Enter the well established, very reliable RSS protocol. RSS, which stands for Really Simple Syndication, is a protocol that allows websites to publish and update content in a common format for downstream consumers to ingest.

I was pleasantly surprised to find out that both of these sites offered RSS in the direction that I needed them. Medium on the sending side offers an RSS feed of articles that you publish that others can subscribe to. One thing on the Medium side that I want to call out is just how well documented and configurable it is. You can get an RSS feed in almost any shape that you that your heart desires (your whole feed, just specific tags, custom domain, etc.). It’s amazing.

For my example, it was easy. I wanted it all.

So I was able to get this by utilizing an address in the shape of https://medium.com/feed/@ . With my RSS feed in hand, I was then able to head on over to DEV to get things configured on that end.

Configuring RSS Replication To DEV

As I already had a DEV profile to start, I was able to get right to business. If you don’t already have one, you can head on over and go through a standard sign up process just as you would with any other site. Once you’ve got your account in hand, you can get logged in and you’ll be able to click on your account iconin the upper right hand corner of the screen. Once you click on that icon, you’ll see a dropdown and you can click on Settings.

Account Dropdown Screen

Once you click on Settings , you’ll be taken to your account info page. You’ll see an Extensions option in the navbar on the left side of the screen. Click on extensions and you’ll be greeted with a number options. The one we’re interested in is labeled as Publishing to DEV Community from RSS . It’ll look something like this:

There’s a few lines of information here (and you’ll notice they specifically call out how to do this from Medium, which is convenient). But what we’re most interested in is dropping the RSS feed link we’ve already pulled together. From here, there are two additional options that you might be interested in. The most important of these is Mark the RSS source as canonical URL by default .

What does that mean? What is a canonical url? Well if you do a quick search, you’ll have some folks who think it is super critical. Others will say it’s SEO nonsense that no one actually follows. But to keep things simple, according to our good friends over at Google, the canonical URL is the most representative link to a piece of work on the internet. In other words, it’s the original or source of the content. So in this case, we’re marking our source article on Medium as the original source so that search engines drive traffic to the original source, while still making it available to the community we still want to be part of over on DEV.

Once we’ve got all of that plugged in, we can go ahead and click Save Feed Settings and everything will begin to get sucked into Dev. And we’re off to the races, right? RIGHT?!

A Little Tidying Up Never Hurts

Well, turns out we’re not quite finished yet. Setting up our RSS ingestion only brings things over to DEV as a Draft. You’ll see all of these articles in your dashboard, but you’ll still want to do a quick read through of those imported articles to make sure things like images and formatting look appropriate. While DEV does it’s best to take the content as it is from the source, there will inevitably be a few things that will need cleaned up.

What you should see for all articles that are pulled over, is something like this:

To get things published, is not exactly self-explanatory. I was expecting to be able to click Edit and then see a big shiny red button that said Publish . However, what you’ll be greeted with instead is only a button that says Save Changes . While you may think to yourself that’s all you’ll need to do, if you click that button it’ll actually just save your changes and stay in Draft mode.

For those a bit more well versed in Markdown this might make a little more sense to you though. At the top of every imported post, you’ll see something that looks like

---
title: Making Your Docker Network Reachable In MacOS
published: false
date: 2024-09-21 21:45:18 UTC
tags: mac,development,localdevelopment,docker
canonical_url: https://medium.com/@tylerauerbeck/making-your-docker-network-reachable-in-osx-e68f998f8249
---

This is called frontmatterand is actually really cool to be able to see! It tells you the canonical_url that we talked about, as long as the title, date, tags, etc. that get pulled over from Medium. But the important one is published . Until you set that to true , your post will remain in Draft and only those with the direct link will be able to come see it.

That’s not what we want! So once you’re ready, flip that flag to true and click Save Changes . Once you do, your post will be live on both sites!

Is This Worth It?

To be honest, I have absolutely no idea. I saw that I could make two things that I like using work together, so I pressed some buttons. In the long run, I’ll be interested to compare the stats between the two sites. I figure at the least, this will help me figure out where I should direct some of my attention. Maybe there will be a time where I decide to flip between the two. On the other hand, maybe I’ll find that they’re complimentary and both have their benefits. For now, outside of the initial effort to get this set up, I don’t think it’s going to cost me much of my time to maintain, so I’ll be interested to see what it is I do learn from this. I’ll be sure to follow up with anything I do learn or any fun ways that I find to use this in the future.

Hacktoberfest Is Coming

Tyler Auerbeck — Tue, 01 Oct 2024 20:20:57 +0000

And we should all be excited (and cool, everybody be cool…)

It’s October 1. And all through the repositories.. a cold shiver has (maybe) gone down the spines of maintainers everywhere. There’s a storm on the horizon. The hordes are coming. Hacktoberfest has begun.

And to be honest, that’s a real bummer to me. If I look back to my first(ish?) pull request, when and where did it come from?

Hacktoberfest.

So for me, this event will always hold a special place in my heart. Why? Because it’s one of those things that helped me figure out “oh, that silly little fix is important”. Prior to this, I had always just been a consumer of open source projects. I’d type away into GitHub issues and beg for help. Why? Because I was still learning how to navigate within the ecosystem. Crafting a PR was intimidating. Was my change important enough? Was it even correct? Probably not, I’m sure someone else will get to it.

But with the nudge of Hacktoberfest, it suddenly didn’t seem so scary. Was it the gamification? Was it the swag? I’m sure a bit of both had caught my interest, but suddenly I was a lot less self-conscious (and a lot more focused on filling the meter). But, looking back, the benefit here was that it encouraged me to open up pull requests. And looking back, the real value wasn’t even any of the above. It was learning how to go through the PR process. It was learning about working with upstream maintainers. And maybe even more personally, it was figuring out that not all feedback needed to hurt.

So to me, this is always how I’ll remember Hacktoberfest. As a month that has the potential to change folks lives. From that first PR, I managed to open quite a few many more. I’ve gotten new jobs off the back of that PR. I’ve met friends living all across the world. I can attribute a ton of forward momentum in my life directly to that moment. All I needed was that first push.

Up Next: An Avalanche

And of course, as with every coin, there is another side to consider. While I’m very sure I am not alone in the story I’ve told above, there are of course some negatives that have come along with it. While this annual event has certainly been great at driving contributions (apparently, people love(d) t-shirts), it also has a tendency to overwhelm already weary maintainers on small and large projects alike.

These projects, who in most cases would love the help, once a year find themselves facing an avalanche of issues and pull requests. And in some cases, it’s not exactly the additive contribution that they are looking for. In some cases, it’s just changing the order of words in a sentence or adding some unnecessary punctuation. All just to increase their contribution count.

This is not the way.

To be fair, as years have gone by, Hacktoberfest organizers have come up with better ways to help discourage this type of behavior. Instead of it being a free-for-all, things now are a bit more opt-in. In addition to this, maintainers have a few more tools at their disposal to mark this type of undesirable behavior. While I’m not sure exactly how many folks do this level of spelunking, if thorough enough it would certainly be possible to link someone back to their undesirable behavior during Hacktoberfest. So beware, your reputation is likely to follow you around depending on how you decide to act during this most code-filled of months, for better or for worse.

On the flip side though, as a maintainer I’ve tried to look at Hacktoberfest through a slightly different lens. Instead of seeing it as a burden to be overcome, I see it as a potential set of signals. While there are always going to be folks just looking to pump their numbers, I think the majority of folks are just looking for the right on-ramp. If I’m getting bad PRs or Issues, maybe those signals are telling me that some work is needed to make that on-ramp more approachable. So what if Hacktoberfest was instead an exercise in sanding down sharp edges. Work on making that CONTRIBUTING.md a little easier to interpret. Write up something that looks like My First PR. What I’ve come to find is that if we make the desired behavior the easy behavior, you tend to get where you want a lot faster.

Go Forth, Together

So my advice to everyone? Lean in. Everyone be cool, but the only way to proceed is together. The only way we continue to grow this ecosystem that we all love is by encouraging others to get involved and showing them the ropes.

So go forth. Be excellent to each other and enjoy your Hacktobering.

MetalLB and KinD: Loads Balanced Locally

Tyler Auerbeck — Tue, 24 Sep 2024 16:01:40 +0000

When You Need LoadBalancer Services On The Go, MetalLB and KinD Are There For You

There are some blogs that no matter how many times you do something, you always come back to. Whether it’s because it’s complicated OR the fact that they’re just good and you decide that means you don’t need to commit them to memory — they just end up as part of your toolbox.

I recently had to revisit one of these oldie, but goodies and it struck me; tech blogs tend to age more like a bad cheese than a fine wine. Things are constantly moving. And while concepts will hold up, configs are generally a different story.

In my case, I wanted to spin MetalLB up on KinD to support some testing that I was doing. The task at hand was simple. All I needed was the following:

A Kubernetes Cluster
A CNI
The ability to create some LoadBalancer services

So back to my dear old friend I went. But this time, things didn’t just work. I, as someone who frequently uses MetalLB, should have expected this. In versions since 0.13.2 there has been a switch from using a ConfigMap for configuration towards using a number of Custom Resource Definitions (CRDs). And yet, there I was.

Stumped.

So, in order to bring my future self some peace when I need to refer back to how do to this again, I figured it would be helpful (at least to myself, if nothing else) to write this down.

So let’s dive in.

Step 1: Kubernetes Cluster

The first thing I needed to do to get started was spin myself up a Kubernetes cluster. Since this was just testing out some bad ideas, I didn’t need anything robust, so spinning up KinD was more than plenty. For the purposes of our example, we’ll spin up a two node cluster by crafting the following config file (config.yaml) and applying it.

---
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker

With our configuration ready, we can then get things fired up by running:

╰─❯ kind create cluster --config.yaml --name metallb-kind

In this example, the above will create a cluster named metallb-kind , but please be aware that you can give your cluster whatever name you would like. Or you could just accept the default. Whatever you choose, just make sure if you’re running through this that you’re pointing yourself at the appropriate cluster and not firing this into a cluster that actually matters.

Once your cluster is up and running, you should have something that looks like the following:

╰─❯ kubectl get nodes
NAME STATUS ROLES AGE VERSION
metallb-kind-control-plane Ready control-plane 4m57s v1.31.0
metallb-kind-worker Ready <none> 4m41s v1.31.0

Step 1.5: Reach For Your Kubernetes Nodes

Usually, just being able to access your kubernetes cluster is enough. However, with the testing that we want to do, we need to actually make sure that we can reach our Kubernetes nodes by address. Specifically, this is a test to make sure that our Docker network is reachable. Otherwise, trying to make any other additional addresses available to us would be pointless.

To begin this test, we’ll first grab the addresses associated with our KinD nodes. You can get these by running the following and grabbing the results under the INTERNAL-IP column:

╰─❯ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
metallb-kind-control-plane Ready control-plane 10h v1.31.0 172.18.0.4 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-15-generic containerd://1.7.18
metallb-kind-worker Ready <none> 10h v1.31.0 172.18.0.5 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-15-generic containerd://1.7.18

Once you’ve got your addresses, a simple ping check will suffice to verify your connectivity.

╰─❯ ping 172.18.0.4
PING 172.18.0.4 (172.18.0.4) 56(84) bytes of data.
64 bytes from 172.18.0.4: icmp_seq=1 ttl=64 time=0.065 ms
64 bytes from 172.18.0.4: icmp_seq=2 ttl=64 time=0.062 ms
64 bytes from 172.18.0.4: icmp_seq=3 ttl=64 time=0.042 ms

If you’re seeing some responses, you’re good to go! Otherwise you may need to troubleshoot what may be in the way of your connectivity. This should work without much issue on Linux based operating systems, but in the likely chance that you’re using a machine with a nice fruit stamped on the lid, there may be some additional steps you need to take. If you’re searching for some assistance, taking a look at this article may help.

Step 2: Sourcing Some Network Blocks

At this point, we’ve got a cluster and we’ve verified that we can access the nodes. But now we need to source a pool of addresses that we can use to advertise our services with via MetalLB. Since we’re relying on the Docker networking that KinD is using, we’ll need to grab some unused address space from there. By default, the Docker network kind is used, so we can use tools like jq to poke at this a bit (unless you feel particularly inclined to eyeball this directly). The value we’re looking for will be buried under IPAM.Config in the subnet value. There may be more than one entry here, so what you’ll want to grab is the IPv4 subnet. If you don’t have any complex configurations in your docker network, then running the following will yield us what we’re looking for:

╰─❯ docker inspect kind | jq .[].IPAM.Config
[
  {
    "Subnet": "fc00:f853:ccd:e793::/64"
  },
  {
    "Subnet": "172.18.0.0/16",
    "Gateway": "172.18.0.1"
  }
]

Step 3: Bringing MetalLB To Life

At this point, we’ve got the values that we’ll need to get a basic MetalLB deployment up and running. To keep things simple, we’ll deploy the manifests provided by the MetalLB GitHub repo:

╰─❯ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml

If you prefer, you can also deploy this via Helm with the following:

╰─❯ helm repo add metallb https://metallb.github.io/metallb
╰─❯ helm install metallb metallb/metallb

Whatever path you choose, what you should end up with is a functioning MetalLB deployment of a single controller and two speaker pods based on the cluster that we’ve created.

╰─❯ kubectl get pods -n metallb-system
NAME READY STATUS RESTARTS AGE
controller-8694df9d9b-fhkp2 1/1 Running 0 1d
speaker-5xc9r 1/1 Running 0 1d
speaker-as78d 1/1 Running 0 1d

Step 3.5 — Configuring MetalLB

Now that we’ve got a functional instance of MetalLB, we can make sure that it’s configured correctly to use the network ranges that we dug up earlier. As I mentioned, in previous versions this was managed via a ConfigMap. However, in recent versions there are now a number of CRDs that are available to us. The resources that we’re interested in for this example are the ipaddresspools and l2advertisements . If you’re interested in the other CRDs that are available, you can check out this listing via doc.crds.dev .

metallb/metallb

To begin, we’ll need to create one of each of the above. The first will be an empty l2advertisement and the other will be an ipaddresspool containing a small portion of the subnet that we gathered earlier.

---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: demo-pool
  namespace: metallb-system
spec:
  addresses:
  - 172.18.255.1-172.18.255.25  
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: demo-advertisement
  namespace: metallb-system

This is probably larger than whatever we’ll need to run locally, but we may as well give ourselves some room to grow while we’re here. Once we have the above in place, MetalLB will be ready for any service that applies the appropriate annotations requesting an address.

Step 4 — Creating A LoadBalancer Service

Now that MetalLB is up and running, we can begin to create services that will utilize it. For a quick test, I’ll rely on an old favorite: http-echo . To test this out, you can create the following deployment and service to get started.

---
apiVersion: v1
kind: Namespace
metadata:
  name: echo
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo
  template:
    metadata:
      labels:
        app: echo
    spec:
      containers:
        - name: echo
          image: hashicorp/http-echo
          args:
            - -listen=:8080
            - -text="hello there"
          ports:
            - name: http
              containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: echo
spec:
  ports:
    - name: http
      port: 80
      targetPort: http
      protocol: TCP
  selector:
    app: echo

With the above applied, you should be able to see a running echo pod in the echo namespace. You’ll also see that you’ve got yourself a regular ol’ service there as well.

╰─❯ kubectl get svc -n echo
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
echo ClusterIP 10.96.173.217 <none> 80/TCP 2m40s

That’s great to get us started. But we want a LoadBalancer service to make use of that sweet MetalLB magic! So let’s make a few modifications. There are two things we’ll need to add/update with our service defintion.

---
apiVersion: v1
kind: Service
metadata:
  annotations:
    metallb.universe.tf/address-pool: demo-pool
  name: echo
spec:
  ports:
    - name: http
      port: 80
      targetPort: http
      protocol: TCP
  selector:
    app: echo
  type: LoadBalancer

The two areas that we’re interested in live under the annotations and type fields.

The first, fairly self-explanatory change is to the type field. We want to specify that we want this to be of type LoadBalancer so that we can get an external IP assigned to our service.

Next, we’ll want to add a MetalLB specific annotation to our service so that the MetalLB controller knows that it needs to take action on it. MetalLB annotations can allow you to take a number of actions such as targeting a specific pool, a specific address and much more. In our case, we’re just going to target the pool that we created earlier (because in our case — we don’t care what specific address gets used). The pool we created was named demo-pool , so we’ll add the annotation metallb.universe.tf/address-pool: demo-poolto our service. After applying our new annotation, our service should look more like the following with a shiny new address that we can reach it at.

╰─❯ kubectl get svc -n echo
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
echo LoadBalancer 10.96.190.254 172.18.255.200 80:30216/TCP 2m21s

At this point, we can see that the service was updated to be of type LoadBalancer and that an address has been assigned from our pool under EXTERNAL-IP . From here, a quick curl should suffice to prove that this is all functioning as expected.

╰─❯ curl http://172.18.255.200
"hello there"

SUCCESS! We’ve got MetalLB running and configured, we’ve got an app running and we’ve advertised an address that we can hit it directly on. We’ve got the whole world in front of us! I mean really, we do. This is a pretty basic example and there’s a ton of more exciting things that we can do now that we’ve confirmed our basic configurations are in place.

But, for the purposes of this blog, we’ve accomplished what we’re looking for. We’ve got a KinD cluster and MetalLB up and working together. Personally, this is a huge benefit to me as while KinD is more than capable of poking some holes and making things available via certain ports — it’s always been much more beneficial to me to run something that looks more like my production environment. So being able to test out and play with the tools that I actually use has drastically reduced the number of lumps that I need to take during my deployments and vastly sped up my development loops.

Hopefully this provides you some similar benefit for any local development you’ve got planned!

Making Your Docker Network Reachable In MacOS

Tyler Auerbeck — Sat, 21 Sep 2024 21:45:18 +0000

Paging Docker Network. This is localhost. Do you copy?

Hello? Docker Network? Are you there?

In a past life, all of my development was done from my (or a series of) Fedora machines. Those were the days.

But these days. I just have to be honest. I’ve sold my soul to Apple. I plug things in and they just work. I know! I’ve grown soft in my old age. We’re all still just sitting here waiting (very) patiently for the year of Linux on the desktop and in the meantime, I turn my headphones on and they just connect! No wizardry required.

But I digress. I say all of this to admit there are some things that I miss in this exchange. One of the most important being the native(ish) way that things could just run on my linux machines before.

Before I’m flooded with an unnecessary amount of

Remote Development Is The Only Way

Trust me. I know. I love it. Great stuff. Big believer.

With that being said, I’m sure I’m not alone in the fact that there are times when that remote environment just isn’t available to me. Also, sometimes I’m just feeling lazy and want to run a stupid experiment and it would be really cool to do that fast and locally. It’s in these situations where I find myself working on my MacBook and trying to do something as simple as ping a container running on my machine and get this very sad response:

PING 172.18.0.2 (172.18.0.2): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3

First off. How dare you. Second, what happened to my sweet, sweet pings.

Well, this is where I’m once again reminded of this pain point. Docker isn’t actually running on my machine. It’s running in a machine, inside of my machine.

Whether it’s Docker Desktop or Colima, what’s actually happening is that the docker daemon is running inside of a virtual machine on my host. And that means that the docker network isn’t actually being made available on my localhost. If I want to access that network directly, I’ve got to SSH into that VM.

Gross.

While I’ve mucked around with a number of workarounds in the past, the one that I’ve come across most recently is actually one I like the most. The docker-mac-net-connect is a really cool project that utilizes Wireguard to make the docker network from inside your VM available to your localhost.

Now before talking a bit about how great this is. Some notes. It does currently only work with Docker Desktop. However, there is an open PR enabling this to work with Colima. So this _should_ work nicely for a number of setups here in the near future. But for now, we’ll assume that you’re treading down the Docker Desktop path. Otherwise, your mileage may vary.

Alright, now back to the cool stuff. As we saw before, without this running we saw heartbreak. Dropping pings as far as the eye can see. However, after getting this setup, what we can expect: All your hopes and dreams. Slinging pings to your docker network like it’s actually running on your machine. But how do we get there? Well, let’s dive in.

The first thing we need to do? Installation. Since this is just a Go application, you can go ahead and just pull it down and build from source if you’d like. However, we’ll take the nice and easy approach here. This project publishes everything to Homebrew. So if you’ve got it installed on your machine, installation is easy as:

# Install via Homebrew
$ brew install chipmk/tap/docker-mac-net-connect

# Run the service and register it to launch at boot
$ sudo brew services start chipmk/tap/docker-mac-net-connect

With these two commands, we’ve got everything installed and ready to go. With the service up and running, we can run our ping test again.

PING 172.18.0.2 (172.18.0.2): 56 data bytes
64 bytes from 172.18.0.2: icmp_seq=0 ttl=107 time=17.485 ms
64 bytes from 172.18.0.2: icmp_seq=1 ttl=107 time=20.892 ms
64 bytes from 172.18.0.2: icmp_seq=2 ttl=107 time=21.390 ms

That’s it. It works. End of story. Thanks for coming out.

Seriously though. That’s all you need to do to make your docker network available directly from your machine. HOW IS IT THIS EASY?! Why have we suffered with this problem this long if that’s all we had to do. What is actually going on behind the scenes to make this happen?

Well luckily, this project has a very nice diagram to help us visualize this:

docker-mac-net-connect/assets/connection-diagram.png at main · chipmk/docker-mac-net-connect

Summarizing from this projects README, this works by creating a tunnel between your local machine and the Docker Desktop virtual machine via Wireguard. The docker-mac-net-connect binary acts as the local wireguard server and on initial creation, creates a temporary container that configures the VM appropriately. From there it monitors the docker networks on the VM and adds the subnets to the OSX routing table via the utuninterface.

So now, armed with a nice utility and a TLDR; level understanding of what’s going on in your machine: GO FORTH. ACCESS YOUR CONTAINERS LIKE YOU’RE LIVING YOUR BEST LINUX-Y LIFE.

Using Open Policy Agent and Conftest to Validate Your Openshift 4 IPI Configuration

Tyler Auerbeck — Fri, 25 Jun 2021 18:54:48 +0000

When deploying OpenShift clusters -- there's nothing that's more frustrating than kicking off a deployment and realizing half way into the process that your deployment failed just because of a bad config caused by a typo, an invalid region or any other small, preventable error. I've recently run into this myself when I was attempting to deploy a cluster into an unsupported region in AWS and realized I've finally had enough. There's got to be a way to prevent this self-caused insanity.

What's The Problem?

Let's first take a look at the problem we're trying to solve.
In OpenShift, if you want to pass any customizations to the installer, you can do this through a file called install-config.yaml. This can be used to tune general cluster settings, but is especially useful when dealing with cloud-providers when you need to specify things like regions, availability zones, etc. In this specific instance, I was working with AWS -- so this example will be centered around that use case (although is applicable to anywhere you want to run your installer).

The particular issue that popped up was that I was attempting to deploy to a cluster and hadn't given any thought to where I was deploying this (the wonders of cloud infrastructure!) as I just wanted a cluster to play around with. However, because of some of the requirements that OpenShift has, this was an issue. In a given release of OpenShift, there are approved regions that can support a deployment do to various requirements. As this specific region didn't meet all of the requirements and wasn't on the list that the documentation lists as an approved region; things did not go well.

Note: The list of approved regions for any given release can be found here.

So there I was, in a state of despair because I had sat there and wasted my precious time only for the cluster build to fail.

It was at this point that I decided enough was enough. I've got to stop doing this to myself. Surely there's a way to save future me some pain and suffering.

Luckily, as mentioned above, this documentation is available and easy enough to find. But I wanted things to be a bit more actionable. I want something that allows me to fight back against the YAML! So I got to thinking. What's are some of my favorite tools to abuse YAML with?

Open Policy Agent and Rego

That's right! Time to write some Rego. policies so that these types of things are checked when I start pushing them into my version control of choice. In this way, not only can I run these checks while I'm doing any sort of development, but I can also rig this up within my CICD processes so that anytime these things change I can make sure they'll still be functional.

If you're not familiar with Open Policy Agent (OPA) and Rego, I highly suggest you take a look here. At a high level, OPA is a general purpose policy engine that allows you to store your policy as code in a language called Rego. With these tools, I can describe policies that look at the value contained in specific parts of my YAML to ensure it is contained in an approved list and show an appropriate message if things go wrong. So let's get started!

Install Config

The first thing we'll need is the problem child. Our install-config.yaml is the source of our issues here, so we'll want to make sure we have one to test against. For our purposes, we'll just steal the one that is provided as an example on the OpenShift docs site (which you can find here. This will look something like this. We'll make one change to the example provided by the OpenShift team by changing the region from us-west-2 to eu-north-1 for the purposes of our example.

apiVersion: v1
baseDomain: example.com 
controlPlane: 
  hyperthreading: Enabled   
  name: master
  platform:
    aws:
      zones:
      - us-north-1a
      - us-north-1b
      rootVolume:
        iops: 4000
        size: 500
        type: io1
      type: m5.xlarge 
  replicas: 3
compute: 
- hyperthreading: Enabled 
  name: worker
  platform:
    aws:
      rootVolume:
        iops: 2000
        size: 500
        type: io1 
      type: c5.4xlarge
      zones:
      - us-west-2c
  replicas: 3
metadata:
  name: test-cluster 
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  machineCIDR: 10.0.0.0/16
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16
platform:
  aws:
    region: eu-north-1
    userTags:
      adminContact: jdoe
      costCenter: 7536
pullSecret: '{"auths": ...}' 
sshKey: ssh-ed25519 AAAA...

Now that we have this content, we can save it to a file called install-config.yaml, we now have the base of what we're going to test with. So next, let's write a policy that will make sure that our region is supported. We'll write an initial policy for OpenShift 4.1 (where eu-north-1 was not supported) and then write a seperate policy for OpenShift 4.5 (where eu-north-1 is supported).

Make Things Actionable

Understanding Rego Policies

So let's get started with our policy. The first thing to do is understand what exactly makes up a policy. To do that, let's take a look at a basic example.

<action>[msg] {
  myvar = 1
  input.my.yaml.struct != myvar
  msg := sprintf("%s is broken. This is my message",[input.my.yaml.struct])
}

So what does all of that actually mean? Well the first thing you'll want to decide is your action. Do you simply just want to alert people to things? Or do you want to throw a hard error when something isn't as expected? This is where you decide the level of intervention you want to display to your user and what you'll see here most frequently is warn or deny. When you see warn, you'll see the message associated with your policy, but it won't return an error code. On the flip side of this, if we use the deny action, this will sound the alarms and return an error code that alerts you to an issue with your policy (as well as still showing the message that you've defined). This is important to think about upfront as to how you want to handle this and what your strategy may be rolling forward when handling things like deprecations where you may want to move a policy from just throwing a warning to actually throwing an error.

Example:

<action>[msg] {

What you'll see next inside of the brackets is the actual content of your policy. There are a number of ways to store values inside of variables, so we won't cover all of them. If you're interested in what is available, please refer to the documentation here. The important takeaway here is that you can set a certain set of values that you think may be approved or disapproved and then test the value of your YAML against that. In our example instance above, we're just storing the value 1 inside of myvar.

Example:

  myvar = 1

The interesting part comes next. Now that we now what value we want to compare against, we need to decide what our "rule" is. Do we want to make sure that a certain part of our configuration matches something. Do we want to make sure it isn't set to a specific value? What happens here is the evaluation over whether your policy should be triggered or not. So in our example, we just want to make sure that a config in our YAML isn't equal to our value in myvar.

Example:

input.my.yaml.struct != myvar

The final part of our policy is the "thing" that should be thrown should a policy get triggered. In our case, we're just showing a message telling everyone what is happening to let everyone know that things are not as they should be. We can do all kinds of formatting and things of that nature here, or we could just pass in a simple string. The important piece to take away is that this is what lets the user know what is going on, so make it useful!

Example:

  msg := sprintf("%s is broken. This is my message",[input.my.yaml.struct])

Building A Policy For OpenShift install-config.yaml

So now that we understand what we're seeing when looking at a policy, let's build one for our use case. To remind ourselves what that is -- remember, we want to make sure that we are only providing appropriate values for an AWS region. If we don't have an appropriate value, we want to see all of the alarms go off so that we don't waste our time waiting for a failed deployment. So since we want things to break quickly, that leads us to building a deny policy. To start, that means we'll have something that looks like this:

deny[msg] {

}

By itself, this isn't useful. Why? Because we haven't told it how to do anything. So let's move to the next step. Now that we know we want to deny things, what is it that we want to deny? We need to make sure that a value that is provided is within a certain set of values that we know are approved. In this case, that means we need a bunch of strings that align with our approved AWS regions. We're going to start off with building a policy for an older version of OpenShift (4.1) because a) it gives us more broken options to play with and b) because we want to make sure we have a set of policies that allow us to inspect changes when we're doing things like upgrades (i.e. moving from 4.1 to 4.2 to 4.X).

So after looking at the approved regions for OpenShift 4.1, we end up with this list:

ap-northeast-1
ap-northeast-2
ap-south-1
ap-southeast-1
ap-southeast-2
ca-central-1
eu-central-1
eu-west-1
eu-west-2
eu-west-3
sa-east-1
us-east-1
us-east-2
us-west-1
us-west-2

So now that we know what values we're working with, we can add them to our policy. Since we're working with a group of strings, we can just look to shove them into an array so that we can compare what is contained in our config against them. This will end up looking like this:

deny[msg] {
  regions := ["ap-northeast-1", "ap-northeast-2", "ap-south-1", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "sa-east-1", "us-east-1","us-east-2","us-west-1","us-west-2"] 
}

With that, we know what we want to check. But now we have to have some part of the config that we check against. The install-config.yaml file contains the AWS region under the .platform.aws.region part of the config. So what we need to do is inspect input.platform.aws.region. What this does is it says "for whatever file we're inspecting (input), let's grab the value at .platform.aws.region and compare it to something (the rest of our rule)". So how do we check it against an array of values? We can use the notation of != regions[_]. What this says is we don't want something to be equal to any item that is contained within the regions variable. This ends up giving us the below policy:

deny[msg] {
  regions := ["ap-northeast-1", "ap-northeast-2", "ap-south-1", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "sa-east-1", "us-east-1","us-east-2","us-west-1","us-west-2"]
  input.platform.aws.region != regions[_]
}

Great! We're almost there! Now we just need to provide a useful message to let folks know what is going on. To do this, we just need to set the variable that we're using for our message (msg) to what we want to display to our users (i.e. ourselves). To do this, we can do something like the below:

  msg := sprintf("%s is not a supported region. Please reference the associated list for supported regions.", [input.platform.aws.region])

What this does is allow us to be a bit dynamic about the message that we're presenting to users. Instead of just saying that they're using an unsupported region, we can use sprintf to provide a formatted message and just pass in what has been provided by our config file by using input.platform.aws.region. In that manner, we let them know both what is going on as well as the problematic configuration value. Pulling all this together, we end up with a final policy that looks like this.

package main

deny[msg] {
  regions := ["ap-northeast-1", "ap-northeast-2", "ap-south-1", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "sa-east-1", "us-east-1","us-east-2","us-west-1","us-west-2"]
  input.platform.aws.region != regions[_]
  msg := sprintf("%s is not a supported region. Please reference the associated list for supported regions.", [input.platform.aws.region])
}

And with that, we almost have something that is actionable. We have our configuration file and we have a policy that we can run it against. But now you might be asking yourself "How do we run this"?

Open Policy Agent and ConfTest

While Rego is the policy language we use to assemble our policies, we still need something to run those policies with. If you have a cluster and you want to actively evaluate policies, you can end up running an instance of Open Policy Agent and it's associated tooling. However in our case, we just want to check things at runtime (or just on some recurring basis such as when changes get checked in or a pull request is submitted). In the latter instance, we are able to use another tool from the Open Policy Agent project called ConfTest. What ConfTest allows us to do is to specify a file or directory of files that we want to inspect along with the set of policies we want to inspect them with. It then takes all of that and dumps out the associated outputs from those policies and tell us the results (i.e. the messages, how many policies were checked and the results of those policies). This tool is much better suited for our use case, so this is what we will proceed with. To grab the latest version of ConfTest, you can grab the latest release from here.

Note: Just make sure you grab the appropriate release for the OS that you are working with.

Pulling It All Together

So now we have it all: our configuration file, our policy and the tool to bind them together. So let's take a look at how we do that. Given that we have our configuration file stored in install-config.yaml, our policy stored in a file called ocp-4.1.yaml and conftest available on our path, we can just run the following:

conftest test -p ocp-4.1.yaml install-config.yaml

Given the above command executes successfully, we should then see our output. Since we've purposefully given it a file that goes against our policy, we should expect an error and see our deny policy.

FAIL - install-config.yaml - main - eu-north-1 is not a supported region. Please reference the associated list for supported regions.

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions

And just like that, we're now able to test our configurations against our policies. Before kicking off our cluster builds, we can run these against a policy for a specific version and we'll know that we'll be in good shape. Right now we're only putting policies in place for the regions, but the sky is the limit for what you would like to enforce. Does your organization only want certain machine types to be used? Do you want to make sure that each cluster contains a certain set of tags? Write as many policies as you would like! Then just make sure to have a process in place that checks the results before you kick off a build to save yourself all of that wasted time we've seen in the past!

Deploying Bitnami's Postgres Helm Chart on OpenShift

Tyler Auerbeck — Thu, 17 Jun 2021 03:43:49 +0000

If you've deployed a Helm chart that depends on Postgres lately, you've likely found yourself coming across Bitnami's chart as a dependency (or even just being used to deploy Postgres itself!). This has been very useful for others to standardize on and allows us all to spin up a Postgres instance quickly. However, I've found myself on the wrong end of this chart a handful of times in the past as I am generally looking to run things on top of OpenShift. Because of the security defaults present on OpenShift, some of the default settings of this chart cause the deployment to fail and you end up finding yourself looking at an error that looks like this:

Events:
  Type     Reason        Age               From                    Message
  ----     ------        ----              ----                    -------
  Warning  FailedCreate  1s (x11 over 7s)  statefulset-controller  create Pod pg-postgresql-0 in StatefulSet pg-postgresql failed error: pods "pg-postgresql-0" is forbidden: unable to validate against any security context constraint: [provider restricted: .spec.securityContext.fsGroup: Invalid value: []int64{1001}: 1001 is not an allowed group spec.containers[0].securityContext.runAsUser: Invalid value: 1001: must be in the ranges: [1000630000, 1000639999]]

To hopefully save you some of the same grief that I've went through, I figured I would write up how to get around these issues moving forward.

What Does This Error Mean

Before we go about fixing this issue, it's helpful to understand what this error is actually telling us. The key part to pay attention to is unable to validate against any security context constraint. What this is telling us is essentially: ⚠️🔥⚠️🔥. By default, OpenShift has a number of security measures in place. One of them defines the range that a UID must be within in a given project. When running an application in OpenShift, it will attempt to assign a random UID within this range to the application running within the pod. What a number of Helm charts do is attempt to set the securityContext for you. This tends to cause some issues in OpenShift as these securityContexts dont align with the rules that are in place, which cause the application to not be able to start up.

In some well designed charts (such as those in the Bitnami collection), there are parameters that allow you to to configure this appropriately, which is what we will look at next!

How To Resolve It

By looking at the values.yaml file provided by the Postgres chart, Bitnami actually gives us a few hints as to what we need to do.

  ## Init container Security Context
  ## Note: the chown of the data folder is done to securityContext.runAsUser
  ## and not the below volumePermissions.securityContext.runAsUser
  ## When runAsUser is set to special value "auto", init container will try to chwon the
  ## data folder to autodetermined user&group, using commands: `id -u`:`id -G | cut -d" " -f2`
  ## "auto" is especially useful for OpenShift which has scc with dynamic userids (and 0 is not allowed).
  ## You may want to use this volumePermissions.securityContext.runAsUser="auto" in combination with
  ## pod securityContext.enabled=false and shmVolume.chmod.enabled=false

What this tells us is that we want to have something that looks like the following included in our own values.yaml:

volumePermissions:
  enabled: false
  securityContext:
    runAsUser: "auto"

securityContext:
  enabled: false

shmVolume:
  chmod:
    enabled: false

So let's give this a shot. We'll first want to configure our environment so that we can deploy the Postgres chart. To do this, we just want to let our Helm client know where to find the chart:

helm repo add bitnami https://charts.bitnami.com/bitnami

Once this is done, we can create our own my-values.yaml with the above values. This will allow us to override what is provided by the Bitnami chart defaults. After we have this saved to our file, we can run things into our cluster with the following:

helm install our-release -f my-values.yaml

Afterwards you should see mention of a successful installation and some instructions on how to get logged in, interact with the cluster, etc. What we want to check though is the status of the StatefulSet. This will look something like:

╰─ oc get sts
NAME            READY   AGE
pg-postgresql   0/1     5m

At this point, it looks things still aren't up and running. By running the following we'll also confirm that we see a similar error:


Events:
  Type     Reason        Age               From                    Message
  ----     ------        ----              ----                    -------
  Warning  FailedCreate  1s (x10 over 4s)  statefulset-controller  create Pod pg-postgresql-0 in StatefulSet pg-postgresql failed error: pods "pg-postgresql-0" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.runAsUser: Invalid value: 1001: must be in the ranges: [1000630000, 1000639999]]

If we take a look at the original comment in the default values.yaml, we seem to have disabled all of the appropriate securityContext options to allow OpenShift to start configuring this itself. However, what we've discovered is a common problem with comments. They're static 💀. The longer they stick around, the more likely to become out of date they are. A quick search for securityContext in the values file shows us the following:

containerSecurityContext:
  enabled: true
  runAsUser: 1001

This is also confirmed if you search the ArtifactHub page for this chart and search OpenShift.

For OpenShift, one may either define the runAsUser and fsGroup accordingly, or try this more dynamic option: volumePermissions.securityContext.runAsUser="auto",securityContext.enabled=false,containerSecurityContext.enabled=false,shmVolume.chmod.enabled=false

So to clean this up, all we need to do is add the above to our my-values.yaml file:

volumePermissions:
  enabled: false
  securityContext:
    runAsUser: "auto"

securityContext:
  enabled: false

shmVolume:
  chmod:
    enabled: false

containerSecurityContext:
  enabled: false

Once this is in place, we can just clean up after our previous installation and then reinstall our chart to make sure things are in good shape:

╰─ helm uninstall our-release
╰─ helm install our-release -f my-values.yaml

Afterwards, we should see similar messages that we saw during our previous install. But now when we go to look at the status of our StatefulSet, we should be in a much better place.

╰─ oc describe sts
<snip>
Events:
  Type    Reason            Age   From                    Message
  ----    ------            ----  ----                    -------
  Normal  SuccessfulCreate  9s    statefulset-controller  create Pod pg-postgresql-0 in StatefulSet pg-postgresql successful

We should now be able to see the pod being created (assuming that we don't hit any odd dockerhub limitations):

╰─ oc get pods
NAME              READY   STATUS    RESTARTS   AGE
pg-postgresql-0   1/1     Running   0          87s

And just like that, we have a functional Postgres instance that we're able to interact with. Hopefully this was helpful to you and will help you in the future with other charts that act in a similar manner! If you run into any issues or see any issue with what is noted above, feel free to reach out!

Update:
About halfway through this article, I noticed that this was really rooted in a documentation error and only hits folks like myself who don't venture away from the terminal too much. If you look at the README for this chart (which I believe also feeds the ArtifactHub landing page), you'll see a note that says that this additional securityContext settings needs set to false. I've updated the article to call this out in the appropriate places.

Apply Labels to Molecule Platform Containers

Tyler Auerbeck — Fri, 11 Jun 2021 23:37:13 +0000

I've recently found myself in need to apply labels to containers that are spun up as part of my Molecule tests. I was unable to quickly dig up any documentation around this use case, so I thought it might be useful to write this up so others don't find themselves in the same position.

For those who aren't familiar with Molecule, it is an Ansible testing framework that allows you to specify any number of scenarios to test your Ansible roles/collections/etc. as well as the environment that they will run against. The main configuration of a Molecule test is done in the molecule.yml file for a given scenario and tends to look something like this.

---
dependency:
  name: galaxy
driver:
  name: docker
platforms:
  - name: app
    image: dummy-app:v1.0.0
    command: "run.sh"
    pre_build_image: true
provisioner:
  name: ansible
verifier:
  name: ansible
scenario:
  name: default
  test_sequence:
    - lint
    - syntax
    - create
    - converge
    - destroy

Defining Your Environment

As you can see from the above, the section for setting up your environment is contained under platforms. In my case, I was spinning up a number of containers and the app itself uses container labels to do some service-discovery. Without them, my tests fail because the app itself isn't able to find what it is looking for.

Adding Labels

To remedy this, I started making some educated guesses due to the lack of documentation (or lack of my ability to find the documentation). Ultimately I ended up with something like this:

platforms:
  - name: app
    image: dummy-app:v1.0.0
    command: "run.sh"
    pre_build_image: true
    labels:
      - hello: world

After running my default scenario with molecule test, I was able to see that my container spun up with the appropriate labels! As you can see, all Molecule expects is a list of key-value pairs under the labels key. While this is a watered down example (hello: world), if you find yourself using more complex keys (special characters, etc.), I've found that you tend to be more successful by wrapping your key-values in quotes.

Hopefully this saves some others some time and as always, feel free to reach out with any questions (or if you have a better way of handling this yourself!).

Terratest unable to find executable

Tyler Auerbeck — Tue, 12 Jan 2021 21:43:57 +0000

TLDR; Don't forgot to add .exe to your executable on Windows (even when using a *nix emulator)

Lately I've been thinking a ton about writing tests for everything. Gone are the days that I just need to make sure my software works. I want to see the same exact effort put into ensuring that my infrastructure is in just as good of shape. This led me to start investigating a tool called terratest. This is quite a useful package to use if you're comfortable with Go as it provides a ton of helper functions to interact with multiple clouds, Kubernetes and Terraform. I'll be writing more on these efforts in the coming weeks, but wanted to write up an issue that I bumped into that has a simple solution, but I didn't see captured elsewhere.

The problem

Note: For "reasons", I've been doing some development in a Windows environment. Not a huge issue for me, but seems to be the root of the issue for this particular problem. Inside of this environment, I've been using the git bash shell

I began by just walking through a quick example with terratest. I just wanted to apply a hello-world application during a test just so that I could see something working. This looked something like the following in a file called main_test.go:

package test

import (
    "testing"

    "github.com/gruntwork-io/terratest/modules/k8s"
)

func TestKubernetesHelloWorldExample(t *testing.T) {

    kubeResourcePath := "../examples/kubernetes-hello-world-example/hello-world-deployment.yml"

    options := k8s.NewKubectlOptions("", "", "hello-world")

    defer k8s.KubectlDelete(t, options, kubeResourcePath)

    k8s.KubectlApply(t, options, kubeResourcePath)
}

What I Expected

This felt like it should be pretty straight forward -- just grab my configuration and apply it. However, the reason I wrote this is not because it was straight forward.

Instead, I received an error:

error while running command: exec: "kubectl": executable file not found in %PATH%

Troubleshooting

I learned long ago to not just expect things to work on the first run, so this wasn't totally a shock to me. I figured maybe I just hadn't included my binary (kubectl) in my path as a lot of what terratest does is just pass some parameters down to the binary that you're using. I generally toss any extras that I've pulled down to my machine into /usr/local/bin. So I took a quick look just to make sure things were in good order.

> echo $PATH

<A lot of unecessary Windows paths>:/home/me/go/bin::/usr/local/bin:/home/me/bin:C:/bin

Hmm. Odd. Maybe I typo-ed the names as I was moving them around? Maybe they're not executable?

> ls -ld /usr/local/bin

-rwxr-xr-x. 2 tyler tyler 74528312 Nov 18 08:36 /usr/local/bin/kubectl

Also. No. At this point, I ran out of ideas. So I had to take a step back, grabbed something to drink and came back.

I need answers

What could be going on? I see the binary on the machine. Can I execute it?

Can I execute it. As soon as I said these words out loud, I knew the mistake that I made. Can I execute it. I won't say it's always the case, but in general whenever I forgot to add the file extension to a windows executable (.exe) it does not tend to end well for me. And as was my experience, it wasn't helping my case here. So I went about renaming the file to kubectl.exe and gave things another go.

> go test main_test.go

PASS
ok      command-line-arguments  1.809s

And there it was: magic. I received the expected output from my test and it passed. Now some of this is likely caused by my choice of emulator (git bash) and I'd probably be better served in the future by using something like WSL2, but I also realize that other folks are likely in similar scenarios. So lesson learned, don't forget .exe!

Reverting your default editor to Vim in Fedora 33

Tyler Auerbeck — Tue, 05 Jan 2021 21:36:29 +0000

TLDR; Starting with Fedora 33 the default editor is nano. If you're expecting vi(m) then there's a simple package you can install to get back into good shape.

I recently made the switch over to Fedora 33 and was a bit shocked when I began setting up my machine. I had gone to run visudo and the unexpected happened:

I feel like I had already stumbled across this change at some point in the past. But there it was right in front of my face. NANO! Where was my precious vim?

How do we fix this injustice inconvenience?

All jokes aside, this wasn't a huge deal. But it was definitely something that was going to throw a wrench into my usual workflow. So I set off to figure out what the best way to get things back in working order was going to be. Luckily for anybody looking to make this change as well, it's already been wrapped up into a package and you just need to run a single command to get back into working order.

dnf install -y vim-default-editor

This was great because although you can likely just set some environment variables, there are also some other defaults and configurations that may not rely on the EDITOR environment variable that you don't want to manually go hunting for.

One important thing to call out here though is after you install this package, make sure to reboot your machine (or maybe at least restart your shell ?). One problem that I ran into was that although I had installed this package and most things were functioning as expected, the EDITOR variable wasn't immediately changed. So when I went to run things like kubectl edit <something>, it still popped me into nano. This likely isn't the only tool that will do something like that, so hopefully this helps prevent some strife for someone else in the future.