DEV Community: Jochen Lillich

A homelab is a skill booster

Jochen Lillich — Sat, 09 Aug 2025 11:50:04 +0000

In her article “What’s the biggest unsolved problem within Site Reliability Engineering?”, Paige Cruz identifies a critical challenge facing our industry: how do we pass the hard-won knowledge of veteran system administrators down to a new generation that has never worked outside cloud infrastructure?

Cruz makes a compelling case that this knowledge transfer gap represents SRE’s biggest unsolved problem. However, I take issue with one of her conclusions. She argues that “It’s also not acceptable or sustainable or accessible in my opinion to continue to expect people set up home-labs and experiment themselves. We don’t ask veterinarians to practice surgery in their free time at home and I don’t think we should expect that from future SRE.”

I respectfully disagree. A home lab represents the best investment a future SRE can make to secure their career growth.

The home lab advantage

The veterinarian comparison doesn’t hold up under scrutiny. Veterinarians need live animals, surgical equipment, and sterile environments – resources that are neither practical nor ethical to maintain at home. Particle physicists need billion-dollar accelerators. But what does a sysadmin or SRE need to experiment with infrastructure? A few decommissioned Lenovo ThinkCentre boxes and a network switch.

The barrier to entry is remarkably low, and the learning potential is enormous.

Scientific research consistently shows that skill development requires more than passive consumption of learning materials. Growing expertise demands practical exploration – running into problems, analyzing failures, and working through solutions. A home lab provides exactly this environment, free from the constraints and pressures of production systems.

You don’t need a data centre in your basement. One or a few small networked PCs will serve you well. These machines consume minimal power and don’t need to run around the clock. Fire them up when you want to experiment with a new service mesh, test disaster recovery procedures, or break something just to see what happens.

For those who can’t or don’t want to maintain physical hardware, alternatives exist. “Pubmox” – the shared Proxmox Virtual Environment I maintain for members of “The Server Room” – offers free access to virtual infrastructure for hands-on learning.

The learning laboratory

A home lab creates space for the kind of curiosity-driven exploration that builds deep technical intuition. When you’re troubleshooting a networking issue at 2 AM in your home lab, you’re not worried about affecting customers or violating change management policies. You can take risks, make mistakes, and learn from the consequences.

This environment encourages the experimental mindset that separates great SREs from those who merely follow runbooks. You learn to ask “what if” questions and then actually test your hypotheses.

Knowledge transfer requires community

While I advocate strongly for home labs as skill boosters, they’re not the complete answer to Cruz’s original question about knowledge preservation. The solution to transferring veteran sysadmin wisdom doesn’t lie in solitary exploration – it requires collaborative knowledge transfer.

Home labs excel at building technical skills, but they can’t replace the mentorship and context that comes from working alongside experienced practitioners. The real magic happens when hands-on experimentation combines with guided learning from those who’ve solved these problems before.

This is why I’m launching “Basic Linux System Administration” – an instructor-led course backed by “The Server Room” community. Participants get free access to Pubmox, allowing them to practice what they learn while benefiting from my decades of system administration experience.

It’s the best of both worlds: the freedom to experiment and break things, combined with the guidance to understand why systems fail and how to make them more reliable.

A home lab won’t solve SRE’s knowledge transfer problem on its own. But for individual practitioners looking to deepen their skills and advance their careers, it remains an invaluable investment. The combination of practical experimentation and collaborative learning creates the foundation for the next generation of systems experts our industry desperately needs.

The post A homelab is a skill booster appeared first on The Monospace Mentor.

80-hour weeks don’t make a great engineer. They’ll break a great engineer.

Jochen Lillich — Tue, 16 Apr 2024 07:00:00 +0000

I’m not exaggerating when I say I was shocked by a question about working hours that I got during a recent office hour live stream. “So I heard from a developer that he would code and learn for like 80 hours a week when he started in order to be come an amazing engineer. I can barely do 8 hours a day and I want to be an amazing engineer. Advice on that?”

Here’s the TL;DR: Working 80 hours a week is neither healthy nor effective. I’d go as far as to saying it’s impossible. Sitting in front of a screen for 80 hours? Yeah, maybe, if that’s your thing. But doing actual work for 80 hours? You might very well be fooling yourself, but you’re not fooling me.

Photo by Muhammad Raufan Yusup on Unsplash

More negative side effects than benefits

Now, before we get into how many hours of work per week you really should invest into your growth, let’s first consider the side effects of spending 80 hours per week working. That is 16 hours every weekday or more than 11 hours every single day, working on your engineering career. Even if we count necessities like eating and pooping as part of work time, it’ll still leave only between 8 and 12 hours per day for the non-work parts of your life. Parts like, you know, hobbies, social life and sleep.

And it’s scientifically proven that for most people, any amount of sleep below 7 hours a night is not enough. The body needs time to recover physically and mentally. In other words, if you sacrifice sleep for other things for more than a very short time, you’re going to ruin your health. Let’s see how not only your career but your whole life flourishes while you’re dealing with the consequences of burnout (or falling asleep driving your car).

Way to sabotage yourself

Total work time also has severely diminishing returns. In other words, putting in more and more time will yield less and less additional results. Effective work requires effective rest. Why do you think Ford and his fellow industrialists themselves introduced the 40-hour work week? And that was just about physical work. Every athlete knows that muscle growth doesn’t benefit from endless training. There needs to be downtime during which the body can recuperate and grow to meet demand.

For our brain, it’s similar, but it’s even more limited in how long it can work at high performance. For knowledge workers, an 8-hour day is like a 16-hour day for the industrial laborer. It’s just not possible to keep your focus for that long, not only because people keep interrupting you, but also for physiological reasons. The brain needs time and rest to build new tissue such as the myelin that grows around neurons.

It’s not about the working hours at all

There’s also the aspect of how much you can even achieve on your own. There’s that story of someone who shipwrecked on an island with just a chess board, and ten years later returned as a grand master. That’s bullshit, because you need other people to beat the Dunning/Kruger effect, i.e. you don’t know what you don’t know. The only way to find the solutions that your brain couldn’t come up with by itself is collaboration. Grand masters learn from other grand masters, amazing engineers learn from other amazing engineers. That’s why I’m so excited about The Server Room, the community of DevOps practitioners I’m going to launch soon.

The good news is that success is not about the total time you spend working at all. Rather, it’s about how much focus time you can get in. In his book “Deep Work”, Cal Newport explains how he manages to thrive as a busy author and academic. “Three to four hours a day, five days a week, of uninterrupted and carefully directed concentration, it turns out, can produce a lot of valuable output.” Just 15 to 20 hours of focused work per week will do the trick. And you’ll still have plenty of time left that you can spend schmoozing and snoozing.

Please don’t blindly believe those hustle culture tech bros who tout how they built their success on relentless work. Even if there’s a kernel of truth to their claims, what they don’t tell you is that they also had the means to mitigate the negative consequences of their self-abuse. Instead, I recommend that you read the book “Time Off: A Practical Guide to Building Your Rest Ethic and Finding Success Without the Stress“. It’s full of good advice like this one:

But as entrepreneurs, managers, and individuals, it’s a dangerous and vicious cycle if we keep assuming that we are productive and effective only because we put in a crazy amount of hard work and long hours, grinding it out. All too often, we get stuck in mediocrity because we are afraid, or maybe just too lazy and comfortable, to question the status quo and the rules set by societal norms.

John Fitch, Max Frenzel, and Mariya Suzuki, “Time Off”

This article was originally published in my newsletter, “News From the Server Room”. To get my column “Mentor Monologue” fresh when I publish it, subscribe here.

The post 80-hour weeks don’t make a great engineer. They’ll break a great engineer. appeared first on The Monospace Mentor.

Startup lessons: 5 mistakes I made when I chose my brand name

Jochen Lillich — Mon, 06 Nov 2023 00:00:00 +0000

Choosing “Opsitive” as the brand name for my teaching and mentoring business was a mistake. I know that now, and in this article, I’ll explain what made me realize that there’s much more to branding than just being clever. I’ll outline the tests that helped my identify the weaknesses of my current brand and verify that my new choice, “Monospace Mentor”, is going to be much stronger.

A few weeks ago, I discovered “Creator Science”, a podcast for content creators I can already highly recommend. One of the latest episodes peaked my interest in particular: “How to invent a GREAT brand name.”. For years, I’ve been dissatisfied with my hosting company’s branding, so I was eager to learn more about this topic. And learn I did!

On the podcast, branding expert Alexandra Watkins outlined two acronyms she invented to gauge the quality of a brand name; one checks the attributes to aim at, the other points out pitfalls to avoid. These acronyms are “SMILE” and “SCRATCH”, and you can find their detailed explanations over on Alexandra’s website, “Eat my words”.

Failing the SCRATCH test

Applying these checks to our hosting product confirmed my gut feeling – its name bombed spectacularly. We’ll address this issue in the marketing strategy development process that we’ve just launched recently. But then I did the same test with “Opsitive”, and it dropped on me that it was weak sauce, too!

First of all, the term “Opsitive”, while being a play on the words “Ops” and “Positive”, does look like a typo and thus fails the first “S” in the SCRATCH test, “Spelling challenged”. The fact that I kept getting emails meant for an online shop for women’s healthcare should have alerted me early on.

Its wordplay character also makes the name suffer from the “Curse of knowledge”, symbolized by the second “C” in SCRATCH. If you’re not familiar with the fact that “Ops” is short for “IT Operations”, you’ll not get the joke.

Taking a less negative perspective, the name is unfortunately also lacking all of the qualities you should search for in a brand name according to the SMILE guidelines:

It’s not suggestive. It doesn’t say much at all, really.
As a made-up play on words, it’s hard to memorize. Was it “Opsitive”, or “Opositive” like the blood type, or maybe just “Opsitiv”?
It doesn’t conjure any image because it’s abstract at best and nonsensical at worst.
It doesn’t have much legs in that it can be extended to other topics. In fact, its explicit emphasis on “Ops” already puts software development out of scope, an area I definitely want to cover!
For the same reasons for which it fails to spur imagination, it’s also not emotional.

There’s another important shortcoming of the old name. It doesn’t put me in the centre, even though that’s the core of my business model: It’s not about the topics I teach, it’s about how I teach them, about my unique way to guide my mentees to their success.

So I needed a new name.

Finding a new name

Coming up with a new name was hard. At first, it seemed that all the good names were already taken. My main problem was finding a name that could accommodate both sides of DevOps, operations and development, but wasn’t at the same time terribly generic. I used several thesaurus websites to research variations on the themes of teaching and mentorship. After many experiments and letting ideas germinate in my brain for days, inspiration finally hit me. There is one thing that unites all system administrators and software developers: they use terminal emulators, text editors and IDEs. And in those, they use monospace fonts. Nerds love their fonts. There are dozens of websites showcasing the huge spectrum of programming fonts, and some people like me even pay good money for a neat commercial one.

And since I just can’t shake my silliness completely, I went for an alliteration as my new brand name. From here on out, I’ll be the “Monospace Mentor”, and my web presence is now at www.monospacementor.com!

Let’s do the SMILE check again.

“Monospace Mentor” is suggestive of personal teaching, and it conjures the image of a programmer’s computer screen.
It is easy to memorize, helped by the alliteration.
It does have legs for expansion in the area of DevOps education.
Is it emotional? I’m not sure, but it’s definitely whimsical!

Did I eliminate the problems of the old name?

It doesn’t look like a typo.
It’s not confusingly similar to the brand of anyone in the teaching space I know of.
It doesn’t restrict later expansion.
It’s not annoying.
It’s not too tame or bland.
No curse of knowledge issue.
And it’s easy and unambiguous to pronounce.

Much better!

Lots of work ahead

I’ll admit that I’m a bit self-conscious about making this kind of a sharp turn. Curiously, we don’t have these inhibitions when we’re coding. Hey, naming and cache invalidation, amirite? 😄 And after all, learning in public is at the core of what I do.

Since I never had any marketing training, I’m grateful that people like Alexandra Watkins share their expertise so others like me can learn from it. The SMILE and SCRATCH tests were tremendously helpful in finding a name that doesn’t make me anxious it’ll lose me potential business simply for the reason that it doesn’t really work as a brand.

Now I’ve got my work cut out for myself. I have to create new brand assets and update the many places where my teaching business is already present.

I’m happy with the new name. Are you, too? Please let me know what you think on my new Mastodon account or on the Opsitive – wait, no – Monospace Mentor Discord!

Un-breaking breaking changes using feature flags

Jochen Lillich — Tue, 24 Oct 2023 23:00:00 +0000

When we got a feature request that we really wanted to implement because it would be a great improvement to our product, we found ourselves in a bind. The problem was that the change would most likely break the workflows of other customers. In this article, I describe how I managed to roll out the change without triggering a flood of angry emails.

Recently, a customer of our managed hosting platform for Drupal and WordPress platform asked us to tighten the security of their website deployment. We realized that the necessary changes would not only help this customer but also improve website safety for all our customers. We love this kind of universally useful feature request because for operational efficiency reasons, we try to avoid making changes for individual customers only.

However, it was easy to see that the restrictions we had to implement for this feature request came with a high risk of breaking existing workflows. People were accustomed to the current level of restrictions, and when you’re above a certain number of customers, you can be sure that some of them will have set up processes that rely on this exact status quo.

A breaking change requires copious advance notice. But the customer who had raised the issue was more or less blocked until we shipped a solution. So how do you make a change in your production environment, but limit its effects to exactly those customers whom it will benefit, not disrupt?

The solution to this Catch 22 came to me late at night, right before falling asleep. (Let me know if you have the same kind of brain.) It’s called “feature flags”.

Feature flags

Feature flags are used to enable new features for a limited number of users only. Common reasons to limit the use of a feature to a smaller circle, at least initially, are that you’d like to prevent bursts in resource usage, or that you want to quickly discover issues that only show up in production without the risk that they’ll impact all your customers at once.

Technically, a feature flag allows you to add code paths additional or alternative to existing ones, and switch those new code paths on or off based on specific conditions. Because the state of a feature flag is usually based on information outside of your code, you can enable or disable a feature at any time without having to deploy updates.

Usually, state of a feature flag is derived from other data, for example a database field or an environment variable. If you have an early access program, the feature condition could be customer.eap_member == true.

In a gradual rollout scenario, you might want to make the new feature available to only a fraction of your customer base for starters. You could pick those customers in a targeted or random way, write their customer ID’s to a database table with a feature identifier, and define the feature state like this:

feature_enabled?(customer: current_user.customer_id, feature: 17)`

It’s easy to build a simple feature flag implementation yourself. In more complex scenarios, I recommend you check out mature solutions like the Ruby gems Rollout and Flipper.

Feature flags for breaking changes

In my security change situation, I realized that I was dealing with just another feature rollout, only with the tiniest of target groups. I basically had to create an “early security improvement program” with the customer who requested the change as its only member. This would enable us to roll out the change immediately for them, and after a generous notice period, for all our customers.

The resulting logic in our infrastructure code turned out very simple:

set_up_website_environment

if feature_active?(:full_isolation)
  set_more_restrictive_permissions
else
  set_the_usual_permissions
end

By the time of writing, the new code has long been deployed to production. Funnily enough, it’s still completely dormant because our customer just could not find a good time for its activation yet. We might end up doing the full rollout process right away. Thanks to the feature flag being already in place, we’ll apply the change first to customers in our Early Access Program, and a bit later to everyone else — including the customer who initiated the whole project.

Summary

Feature flags allow you to have your cake, and eat it, too. You can roll out new code, but limit its activation dynamically in a data-driven manner.

Next time you have to introduce a code change that might not make everyone equally happy, consider using a feature flag to control its blast radius.

Use the Rails console to do a manual Mastodon link verification

Jochen Lillich — Wed, 16 Nov 2022 16:31:00 +0000

When someone on Mastodon asked for help with a Ruby question, my friend and business partner Markus kindly referred them to me, knowing that I’m always happy to help other Ruby developers. Their question was about executing Mastodon code from a custom script. In this post, I describe how I solved this problem using the Rails console and a custom subclass.

The problem the person wanted to solve was testing the logic Mastodon uses for verifying the ownership of a user’s websites.

For details on how Mastodon verifies websites, see the section “Link verification” in the Mastodon user profile documentation. In short, you have to add a link from your website back to your Mastodon account, using either a <a rel="me" ...> link in the page body or a <link rel="me" ...> reference in its header.

Ideally, this test would use the original Mastodon codebase to make sure it worked across application updates.

Not the solution: standalone script

On my live stream, I first attempted to write a standalone script. My thought was to instantiate the VerifyLinkService class fromapp/services/verify_link_service.rb. I was hoping I’d get away with requiring a few dependencies from the Mastodon application, and maybe replacing a few classes with mocks. But I quickly ended up including ActionSupport and the kitchen sink, which made this approach unviable.

The solution: Rails console

In order to use the original verification code, I had to run it in the context of the Mastodon application. The easiest way to do this is using the Rails console. However, spinning up a Rails console requires a working Rails application–database and all. Before I describe how to solve this, let’s first build the code we’re going to run inside the Rails console to test the verification.

Building the test

Maybe I would have been able to use the original VerifyLinkService class with a few tweaks, but after closer inspection, I was actually only interested in its methods perform_request! and link_back_present?. One fetches the website in question, the other checks if it contains a valid link back to Mastodon.

Unfortunately, both methods use instance variables set in the constructor using Mastodon user data. And, to make matters worse, they’re (rightfully) private methods, which meant that nobody is allowed to call them. And just when I told my viewers, “Well, nobody but the class itself, of course”, inspiration hit me: Not only a class is allowed to call its private methods, but also its child classes! All I had to do is create a subclass of VerifyLinkService, provide our test details in its constructor, and call the two methods required to execute the verification. This is what I ended up with:

class ManualVerifyLinkService < VerifyLinkService
  def initialize
    @link_back = "https://mastodon.social/@geewiz"
    @url = "https://www.geewiz.dev"
  end

  def check
    perform_request!
    if link_back_present?
      puts "Verification successful."
    else
      puts "Verification failed."
    end
  end
end

Spinning up Mastodon

This left the problem of getting Mastodon running. The fastest way to run a Rails application with its auxiliary services and not have to do lots of manual installation is generally with Docker containers, and Mastodon is no exception. Conveniently, the Mastodon code repository already comes with adocker-compose.yml definition that contains everything we need. I found a How-To on the web that describes the launch process well. All I had to do was follow it until, and including, the step where the whole setup is started by thedocker-compose up -d command.

In this state, I was able to launch a Rails console inside the already running web application container:

docker-compose exec -it web bundle exec rails console

Running the test

Into this console, I pasted the definition of my ManualVerifyLinkService class from above.

And finally, I was able to execute the verification by instantiating an object and calling its check method:

v=ManualVerifyLinkService.new
v.check

Since my website is already verified by Mastodon, the result was positive.

This was a fun experiment, and as is often the case when I help someone, I’ve learned something myself in the process.

How I do remote administration using ngrok

Jochen Lillich — Tue, 12 Apr 2022 00:00:00 +0000

This month, a new intern from Spain started at my company. While we used to use Macbooks for our interns, he was the first to get a Linux laptop. Nothing fits the DevOps use case better, after all.

I set up the machine with Manjaro Linux in my office, but I had to solve the problem of later installing and upgrading software after the laptop was handed over. It’ll always access the internet from a private network behind a router, and therefore will not be visible for me from the outside. And I certainly don’t want to commute to Dublin, where we book a coworking space for our interns, every time maintenance is required.

In the end, I chose a minimalist approach by using ngrok. This nifty command line tool, paired with the online service at ngrok.com, allows developers to create network tunnels with an endpoint on the public internet. Its main purpose is for web developers to share access to an application running on their personal machine. In other words, it allows people with whom you share the connection details to connect their web browser to your development laptop, even if it’s inside a closed WiFi network. However, ngrok extends this functionality beyond the HTTP protocol and port 80.

When our intern asked me to install Discord and VS Code earlier today, I asked him to run a simple command and tell me the tunnel endpoint address. The command was:

ngrok tcp 22

It sets up the network tunnel as well as a local website on which it lists the details of each connection. Our intern then gave me the listed address tcp://0.tcp.ngrok.io:14463, from which I constructed an SSH command:

ssh -p 14463 geewiz@0.tcp.ngrok.io

Within a second, I was logged into his laptop. A few minutes of installing new packages and upgrading existing ones later, I was done and asked him to cut the connection. It doesn’t get much simpler than that, does it?

YT channel recommendation for neovim and zsh users

Jochen Lillich — Mon, 10 Jan 2022 13:32:00 +0000

I don’t think I ever copied more vim or zsh configuration from anyone than from ChrisAtMachine. Looks like I found a soulmate. 😊 His YouTube channel is well worth watching!

dotenv and direnv for better developer QoL

Jochen Lillich — Wed, 23 Jun 2021 11:46:12 +0000

A RubyTapas tutorial taught me the difference between the dotenv and direnv tools. While they have significant overlap, they complement each other quite nicely. dotenv simplifies both development and production ops by importing environments variables either from a .env file or the application hosting platform. direnv on the other hand augments the dev environment even further; not only can it add dev-only variables, but also modify shell settings like extending $PATH on a per-project basis.

Verify your assumptions

Jochen Lillich — Fri, 18 Jun 2021 14:42:24 +0000

That’s what I keep preaching to my team but still fail at myself every so often.

This week, I wasted a whole day implementing a fix for a bug that wasn’t there. My task was to add one line to a configuration file in a Chef cookbook. When I ran its test suite, it surprisingly failed and it was clear that my simple change couldn’t be the cause. I assumed it was due to recent changes in software packages installed by the cookbook. So I started adapting our old code to these changes, and one change led to another, and another, until I finally realised that over the course of multiple hours, my one-line bugfix had turned into a full code overhaul. How did I get here?

I decided to start from scratch. git reset --hard HEAD. I quickly fixed my config file and ran the tests. When they failed again, closer inspection revealed that it was actually just the same test failing twice. The cause was a bug that had found its way into the main branch without breaking the CI pipeline (finding the reason for this will be interesting). It had been there all along! After another one-line fix, my change was ready for review. Round 2 had taken me less than half an hour.

The lesson: Don’t rush into battle blindly. Look closely. Identify assumptions and verify them. It might just be windmills!

Weeknote W5 2021

Jochen Lillich — Sat, 30 Jan 2021 18:32:00 +0000

To tile or not to tile

Looking at Twitch streams and Youtube videos around Linux, it seemed to me that everyone was into tiling window managers. This week, I decided to finally try this approach of filling and subdividing the whole screen area. On macOS, I installed yabai, and on Linux (more on that below), I chose the popular i3.

After a few days of the tiling life, my conclusion is that I don’t see much benefit in forcing my application windows to fill up all the available screen real estate. Maybe it’s because I already have a habit of tiling my windows manually – when it makes sense. However, often it just doesn’t make sense. Some windows really don’t like to be squeezed into a tiny space, and even on a 13” screen, a full-size terminal window isn’t of much use to me. I’ll keep my windows floating. If I really want to tile them, there’s always BetterTouchTool.

Linux on Apple hardware

Another item of curiosity this week (is this some kind of lockdown syndrome?) was if 2021 could be the year of Linux on the Desktop for me. While I still enjoy the stability and user-friendliness of the macOS ecosystem, some of its constraints do annoy me from time to time. I have an unused Macbook Pro 13” from 2015, and it’s now running Ubuntu 20.04. My go-to applications like Brave, VS Code and Obsidian worked fine out of the box. But when it came to more specialised use cases, I quickly realised how much macOS spoils me with its choice of well-polished applications.

Maybe I will pursue a “best of both worlds” approach, where I do my management work on the Mac and my DevOps work on Linux. That is, if the hardware issues that drove me into Apple’s arms more than a decade ago don’t spoil it for me again. For example, there’s a 50:50 chance that after powering up the laptop, Bluetooth doesn’t work, preventing me from using a proper mouse and keyboard. Sigh.

Clean Chef code: Depend on public cookbook interfaces

Jochen Lillich — Wed, 22 Jul 2020 20:48:00 +0000

For about a year, we’ve been cleaning up the Chef Infra code for freistilbox to make updating dependencies, Chef versions and even operating systems easier. It’s a lot of work because our early code is functional but not pretty. There have been many instances of “we didn’t know better”, and that’s what refactoring is for. But I also came to realise that we were missing a critical fact: Common software engineering principles and practices apply to infrastructure code like they do to any other type of code.

Or as early Chef developer Joshua Timberman puts it:

“Hey, ya’ll remember when devops really just meant you knew how to write all your bash in ruby instead?”

Ouch. Making this connection earlier would have saved us weeks of work. That’s why I’m going to share my findings in a series of posts.

In this post, I’m going to advocate for treating a Chef cookbook as a unit of software that provides explicit interfaces instead of tempting its users to depend on implementation details.

Chef Infra cookbooks use node attributes as variable parameters for system configuration. In software engineering terms, node attributes are global variables. They’re implementation details of a cookbook. In consequence, using a cookbook’s node attributes for any purpose other than defining setup parameters creates a dependency on implementation details which can change at any time.

For example, it’s common practice to use a search on node attributes for service discovery. As node attributes are global variables, any cookbook can do this:

service_nodes = search(:node, "webservice_id:myservice")

From this code, we can tell that web service nodes are identified by a node attribute named webservice_id; nodes sharing the same webservice_id value belong to the same web service.

The problem with using this information outside the cookbook which provides it is that this particular implementation can change at any time. This kind of tight coupling is a liability. For example, a second attribute webservice_status might get introduced, reducing the node set by adding AND webservice_status:active to the query. Since this change in semantics is not necessarily a breaking change, there’s no simple way like semantic versioning to inform everyone who depends on this unofficial interface.

How about we provide an public API instead? Our web service cookbook could for example provide a class we can use for service discovery:

webservice = Company::Cookbook::Webservice::Discovery.new("myservice")
service_nodes = webservice.nodes

This is easy to implement as a cookbook library. By using namespaces, we make sure that method names don’t conflict. In my practice, I tend to use the camel-cased cookbook name under the namespace Cookbook and the company namespace.

With service discovery encapsulated and hidden behind a public interface, we could even reimplement the cookbook’s service discovery using a different technology like Consul without breaking any code outside our cookbook.

But even if other cookbooks depending on our implementation isn’t a concern, implementing auxiliary logic in a central library instead of scattering it across recipe files makes it much easier to maintain. I’m going to talk about “Plain Old Ruby versus Chef DSL” in a separate post.

DIRECT - The seven core topics of remote team communication

Jochen Lillich — Fri, 22 May 2020 11:59:00 +0000

Pick any guide on remote work (oh, there are so many…) and it’ll tell you that communication is essential when you work in a distributed team. The problem is that it’s not obvious where exactly the centre of balance is between sharing too little and sharing too much. That’s why I’ve created a framework that uses a simple acronym to remind you of good opportunities to keep your distributed team in the loop.

You can watch it here or on YouTube. I’m already working on my next videos, so don’t forget to subscribe to my YouTube channel!