DEV Community: Bryan Woolsey

A Criticism of Scrum

Bryan Woolsey — Sat, 24 Feb 2024 23:09:24 +0000

Okay, look... scrum is just terrible. It's frustrating, mind-numbing, and absolutely kills productivity.

Don't get me wrong, on paper scrum seems amazing collaboration framework. And maybe in an ideal world, it is amazing. But down on planet earth, I have yet to be apart of a team or company that actually saw a net-benefit from the scrum framework.

This topic is something that I've been thinking about off-and-on for much of the past decade as agile frameworks have become more popular. And I debated posting this article knowing full well that scrum or scrum-like collaboration frameworks are used in MANY (probably most) software engineering teams nowadays. Surely it's all in my head, right? Maybe I'm just doing it wrong? Maybe I'm thinking about it wrong? Why would scrum be so popular if it didn't actually work?

So to come clean, this really is not so much a criticism of the theory or formula of scrum, but instead more of a cry for help on how it's been implemented in practice, at least in my experience. I can't possibly be the only one struggling with this, can I?

I guess we'll see. Let's dig in.

Sprint Towards Mediocrity

Everyone (I assume) who has followed the scrum framework will be familiar with sprints. But in case you are not, sprints are designed to be a fixed duration of time to complete a bunch of tasks, usually 2 weeks long. The goal (I think) is to timebox tasks to a specific period of time to incentivize teams to keep tasks small and achievable. The hope is at the end of a sprint you end up with some sort of deliverable that can actually be demonstrated to others--something to show for your work.

Prior to a sprint, most scrum teams will have a series of sprint planning and backlog refinement meetings where the team discusses the list of tasks (typically called "User Stories") that need to be completed during sprints. During that discussion, the team is asked to apply an arbitrary number value to that task, usually called "Story Points". Most teams, for whatever reason, use numbers from the Fibonacci sequence: 0, 1, 2, 3, 5, 8, 13, 21... with larger numbers applied to user stories that are more difficult, require more effort, or have a lot of uncertainty attached. Once user stories are assigned a numeric value, the team makes a determination of how many user stories can be completed within a sprint by looking at their average velocity from previous sprints (how many story points worth of work were completed previously), and their upcoming velocity (how many story points they expect to be able to complete). Cool.

Except here is the kicker: This whole story points business is nonsense. It's like the TV show Whose Line Is It Anyway?: The rules are made up and the points don't matter. It is extremely rare to complete the same task twice (and if you are, you should probably automate it). So really the question being asked is this:

"Hey can you assign an arbitrary point value to a task that you've never done before so we know roughly how long it will take and difficult it will be?"

No, not really.

So what do engineers do in the face of such a question? Well, in my experience most will answer it in one of two ways:

Make up some absurdly high number to ensure they are confident they can complete it within a sprint. This usually triggers the typical 10-15 minute sidebar conversation where everyone on the team reiterates all the things we don't yet know. Seems like a good use of time.
Give it a "normal" number so the Project Manager and/or Scrum Master just accepts it and moves on to something else without discussion.

Regardless, once the user stories are pointed and assigned out to engineers, the story points attached to those stories tend to be static. It is extremely rare for a story to be re-pointed after it has been completed to reflect the true complexity. The tasks that were unexpectedly simple were over-estimated, while the tasks that ended up being unexpectedly difficult get taken out of the sprint and assigned to the new sprint. The end result? The team's overall velocity is incredibly inaccurate and not a reflection of the actual amount of work being done. Not only does this hurt the team's performance when reporting up to the people that actually care about that stuff, it also makes planning future sprints a fool's errand. You're assigning arbitrary units of work to tasks you know very little about, all while trying to keep it the overall workload under an imaginary capacity number that is estimated from previous velocity numbers that don't actually reflect the team's true output. Brilliant, just brilliant.

But that's not even the worst of it. The biggest issue is the combination of sprints and story points breeds mediocrity within a team. No one likes to carry incomplete user stories between sprints. Some teams (or, more accurately, their managers) consider it a failure if user stories do not get completed within a sprint. This does nothing but incentivize scrum teams to over-point their user stories, and/or under-estimate their capacity. In an ideal world, if an engineer finishes with their user stories before the end of a sprint, they should simply pull the next user story off the top of the backlog. In practice, there is a much larger incentive to work on a side project, or even just idly sit there twiddling their thumbs while waiting for the end of the sprint. After all... you wouldn't want to pump up your velocity numbers. That could cause you to over-commit in the future and burn yourself with a user story that ends up being unexpectedly difficult. Ugh.

I'll freely admit I don't know a better way to estimate work load and work capacity. But what I do know is that, despite what all of the agile coaches will tell you, story points are really a way to measure how long a task will take. And if engineers were to answer that question honestly they would probably answer it in one of three ways:

"It will take about 15 minutes"
"It'll take longer than it should"
"I have no clue how long it will take"

Facts. (As I see them, anyway.)

Backlog Shoveling

To your left is a large pile of unfinished "to-do" items, typically known as the backlog. In most scrum teams, this is the domain of the product owner or the project manager who is responsible for acting as the liaison between the engineering teams and the business stakeholders. Their job, roughly speaking, is to maximize the value the team delivers each sprint by ensuring the team is always working on the items that are most important to the stakeholders. They are effectively the person who determines the priority of the to-do list. At most companies, these individuals are typically less technical in nature but much more business-savvy. Which, if you've never seen a group of engineers attempt to communicate with business people, is definitely not a bad thing.

The rub with this setup is engineers can't always be working on the items that provide direct benefit for the stakeholders. There are many necessary evils in IT that engineers must devote their time and energy to in order to keep the engine churning. It is the omni-present threat of crippling technical debt. Software currency is perhaps the easy example to point at when it comes to technical debt, but it comes in many forms. And it is extremely difficult for engineers to adequately communicate exactly how important it is to address technical debt. The conversations usually goes something like this:

Engineers: "Hey this piece of software is end-of-life, and we really need to have time this sprint to update it.

Product Owner: "Okay, what happens if we don't update it?"

Egrs: "Well... nothing right now. But it'll be a problem someday."*

PO: "When exactly is 'someday'?"

Egrs: "Hard to say, but it'll be a problem, and we need to address it."

PO: "Will it be a problem in the next sprint or two?"

Egrs: "Probably not, but someday it'll bite us."

This conversation happens every. single. time. there is backlog refinement meeting on all of the user stories that don't clearly provide direct business value. The product owner, who has no idea when "someday" will occur, will naturally lean towards deprioritizing the technical debt user stories in favor of stories that provide more immediate value to the stakeholders because it is their job to do so. At first the decision seems correct and all is fine & dandy... until one day it isn't. There will be a day when the volcano of technical debt explodes, your team is caught in the lava flow, and you're looking at a solid 6 months of slogging through that molted lava of debt. When it happens, and it will happen, there will be a predictable chorus of I-told-you-so's as everyone takes a step back to wonder why they are running production Ubuntu 14.04 servers in the year 2024 (true story).

Now obviously this situation is a problem that many organizations encounter regardless of whether they are operating under the scrum framework or not. The temptation to ignore technical debt is real, and it is not a uniquely scrum problem. However, working under a scrum framework means you are continually refining the backlog, which in-turn makes it easier to encounter these issues. The two or three sprints you initially reserved for tackling technical debt tend to disappear as the associated user stories get refined to the bottom of the pile and completely forgotten about, never to see the light of day until the volcano explodes.

It is not the fault of the product owner, scrum master, or the engineers that these issue pops up. It's ultimately a shared responsibility between all parties to ensure these sorts of issues do not happen. But for whatever reason it is a shared responsibility that seems to be exceedingly easy to ignore while operating under scrum, especially when your backlog reaches the size of "mountain".

Stand-ups... or stand-downs?

Part of the scrum formula calls for daily scrums, which most people in my experience refer to as "stand-ups" regardless of whether you are standing or not. The scrum is facilitated by a scrum master to discuss the sprint's progress or discuss blockers that are preventing progress. In theory, this meeting is supposed to last less than 15 minutes, although in my personal experience it typically lasts around 30-40 minutes to give us ample time to once again talk about all the things we don't yet know. I'm guessing the goal of this meeting is to keep everyone engaged with each other's work, ensure progress is being made, and increase team synergy. But in practice it's just a means for the manager to complete his/her daily check-list so they can update superiors on the work being done. Most engineers don't actually listen to what their team members are doing, they just silently plug away at their tasks (or make themselves coffee) with their microphone muted until their name is called by the scrum master.

Admittedly a "true" stand-up, in which everyone is actually standing in the same room, would largely resolve this problem. But in a world where hybrid and remote work is commonplace, we need something better than these mindless daily scrums. And if no alternative can be found, just let us place an update in the Teams/Slack channel so we can go about our day.

Not Another Agile Ceremony

The biggest issue I have with scrum (and agile in general) is the sheer amount of meetings. Sprint plannings that can take hours, daily scrums that are rarely 15 minutes or less, post-sprint retrospectives that take hours and often turn into complaining festivals. And worst of all, the increment planning meetings—intended to allow organizations to plan out several months in advance—can easily chew up several days and suck away your will to do anything remotely productive for an entire week. These meetings got so bad at a previous employer that my boss at the time coined the term "NAAC", short for "Not Another Agile Ceremony". "Oh, sorry, can't talk right now I have to go to another NAAC."

It would be one thing if these meetings were productive. But when you can feel scrum isn't working... when you know your velocity numbers are nonsense, when you start throwing out random meaningless story points so the meeting ends quicker, when you discuss the same unresolved issues in the retrospectives sprint after sprint after sprint, it just becomes an utterly painful exercise. Judging by the level of interaction (or lack thereof), I would guess a good chunk of engineers out there have stopped expecting to gain value from these NAACs, and instead opt to silently sit in the meeting while doing something more productive, whether that be working on unfinished stories from previous sprints, or even finishing up their laundry.

But don't worry! Bring in the agile coaches to tell us that we're doing it wrong (thanks captain obvious), and let's go to more NAACs to talk about how we're actually supposed to do scrum. Maybe, just maybe, when we've spent more time talking about how to organize the work than it actually takes to do the work, all will be fixed! The punishments will continue until morale improves!!

(Save me.)

But can it be saved?

Can scrum work? Honestly, I don't know.

Unless there is some grand conspiracy to reduce productivity industry wide, I imagine that scrum is truly working for a lot of people, and I've just been unlucky with the 5-6 teams that I've been apart over the past decade. Or maybe I'm just expecting too much from scrum. Or maybe we should be using SAFe instead. I honestly have no idea.

The fundamental issue, I suspect, is switching from a more traditional waterfall-like workflow to one of the many agile workflows requires a fundamental change in how the business is structured and operates. On top of that, it only can truly work if the culture of the company itself changes. And honestly, that's hard to do. These companies are big ships with a lot of built-up inertia. It requires a tremendous amount of leadership to change the direction of that ship. It's just straight-up a hard thing to do, and very few leaders are capable of making it happen. Most leaders seem to prefer the "agile-in-name-only" approach rather than truly putting in the work to fundamentally change the company.

But regardless of the exact reason, my scrum master announced last week that the team will temporarily be changing to a Kanban workflow after an internal company reorganization, and I was more excited than I care to admit. Not that Kanban is some magical framework--many of the issues I've discussed with Scrum also apply to Kanban. But at least there is no longer a pressure to tightly squeeze everything into 2-week units of work. No more of taking square-peg-shaped units of work and shoving it into a round-hole-shaped sprints. And for that alone, I'm thankful.

Your own private cloud? A brief look at Antsle

Bryan Woolsey — Sat, 24 Oct 2020 01:49:12 +0000

If your browsing habits are anything like mine, you likely have seen online advertising from a company called Antsle. But just in case you haven't, allow me to introduce you:

Antsle provides products to enable users to run their own "private cloud". Why exactly does that mean? Well... if you cut down to the meat of it, they sell hypervisor appliances running Antsle's web-based management software called antMan.

I've been keeping my eye on the company and their offerings for a while now, wondering if I could effectively replace my Digital Ocean, GCP, and/or AWS accounts with a cheaper Antsle-based solution for my personal use. Up until now the cost has kept me away, but recently they released their newest appliance called Antsle Nano, a significantly cheaper solution based on the Raspberry Pi 4. So I pulled the trigger and spent the last two months or so playing around with my Antsle Nano to see what it's capable of.

The Hardware

Antsle sells three series of appliances on their website: Antsle one, Antsle two, and the new Antsle Nano.

Antsle one is the evolution of Antsle's original product line. They are effectively silent, fan-less servers in a small box that could easily fit on your desk/shelf. The hardware platform itself is based around the Intel Atom or Intel Xeon-D processor (depending on your selection), and is similar to the hardware found in many NAS devices sold by companies such as Synology. Prices range from $1000 up to $6500+ USD. While I haven't personally used an Antsle one device, the hardware choices available make sense for anyone looking to have a tiny, yet reasonably powerful and silent device.

Antsle two is Antsle's rack mount offering, and is clearly marketed towards business users rather than home users. Much to my surprise (and delight), Antsle two devices run on AMD's EPYC platform rather than the more widely seen Intel Xeon platforms. Given my focus to find a low-end home lab solution, buying an Antsle two device was never really a serious thought of mine. But for those curious, prices run from $3,500 to $50,000+ depending on the configuration.

And finally... there is the Antsle Nano. As I mentioned earlier, the Antsle Nano is based on the Raspberry Pi 4. The prototypes shipped out to early adopters were based off the 4GB variant of the RPi 4, but it seems Antsle has decided to use the 8GB variant for the final version. Business users likely will want to stay far away from the Antsle Nano—but for home lab users such as myself... it is a very compelling device. Antsle's version of the Raspberry Pi 4 ships with a beefy heat sink pre-installed, plus the typical USB-C wall-wart power supply and a 128GB MicroSD card preloaded with the antMan software. $150 is the cost of the device itself, but the software is unfortunately an added cost... more or that below.

The Software

Ansle's control software is called antMan. antMan is a browser-based solution (with an accompanying API) that allows you as the user to control the underlying hypervisor. You can configure your networks and virtual machines—known as "antlets" in the Antsle world—from the antMan dashboard. The antMan user interface is relatively straight forward and well laid-out. After the first 5-10 minutes, I had no problems finding my way around to do what I needed to do.

Currently, there are five different pricing models for antMan, summarized below in my own cheeky manner. Feel free to review Antsle's pricing page for all the details.

Community Edition - A free version of antMan limited to three small (think AWS free-tier) antlets. Can’t do much else.
Home Lab - Comes bundled with any Antsle one purchase. For reasons I do not fully understand, the Home Lab license does not come bundled with the Antsle Nano. Allows you to do most of the usual hypervisor tasks you’d expect, with some additional networking options.
Essential - $29/mo. The Home Lab capabilities with some added backup features, internal VLANs, and the ability to submit 3 support tickets a year. This license level is the minimum level available for the Antsle Nano.
Grow - $149/mo. Comes with even more backup functionality, network trunking, more support tickets, and the ability to cluster multiple Antsle units together.
Scale - $599/mo. Even more backup/migration functionality. Unlimited support tickets.

The operating system behind the curtain

antMan itself runs on top of Antsle's edgeLinux operation system. edgeLinux is essentially a customized version of CentOS 7 that comes packed with all of the usual open source hypervisor technologies that you'd expect to run LXC containers and KVM virtual machines. Docker also comes pre-installed, although no effort is made to expose Docker's functionality in the antMan user interface.

antHill

antHill is Antsle's cloud-based management portal. Antsle will tell your that it is used to ease the initial setup and management of your Antsle unit(s). But if you read between the lines a little bit... antHill seems to the mechanism Antsle uses to manage your antMan license.

Honestly, there isn't much to say about antHill beyond that. I used it once to activate my license and haven't used it since. Although I will note that it was a rather annoying to have to reconfigure my home network to allow my Antsle to talk to antHill long enough to activate my license.

So is it any good?

Good question. For now, my answer is "kinda/sorta... but not really".

If you glance at Antsle's advertising, you might expect that you'll be purchasing a fully featured private cloud-in-a-box. I know when I first read about Antsle, I was envisioning a simple platform capable of creating S3-compatible object storage, a ready-to-go container/Kubernetes platform, an easy method to spin up common open source databases (MySQL, PostgreSQL, etc.), and perhaps even a functions-as-a-sevice mechanism similar to AWS Lambda or GCP Functions.

What you get is small, compact Type 1 hypervisor with a web management portal. Based on my (admittedly limited) personal experience, the hardware itself seems solid, but not something that you couldn't purchase elsewhere from another source. And the software certainly leaves a lot to be desired. Unfortunately for Antlse, the main functionality that antMan provides can already be achieved for free from other hypervisors such as ESXi, Xen, etc.

Honestly... I think there is a place in the market for Anstle’s products, but I’m struggling to clearly articulate where that place is. Solo, startup, and freelance developers looking for a public cloud replacement will likely be frustrated by antMan's limited feature set. Home lab users are likely to be better served using something like Proxmox. And business users likely have the budget to simply use a public cloud provider.

Perhaps the biggest downside, at least for me, is that their most attractive device for people looking to build a home lab—the Antsle Nano—does not come bundled with their Home Lab license and instead requires subscription-based license. The discounted launch pricing on the antMan license is appreciated, but just knowing that I’ll have to pay another $348 in a year to keep my device functional is frustrating.

I do think there is potential for Antsle to improve. If Antsle was able to replicate more of the public cloud services in antMan, I think it could be a very compelling product overall, even with the subscription-based licensing. But as of right now, I find it hard to recommend. That said, if you are just looking for a small, quiet, easy-to-setup device to run some virtual machines... look no further. Antsle's products will do just that.

What do you think? Is Antsle worth it? Or would you stick with your public cloud account? Perhaps you are using another solution at home that I should look at? Let me know!

p.s. Given my current antMan license is valid for another 10 months, let me know if you’d like more in-depth walkthrough.

Dealing with long strings in YAML

Bryan Woolsey — Fri, 23 Oct 2020 00:38:06 +0000

We have a step in our Puppet CI/CD Pipeline that lints YAML data using adrienverge/yamllint. One of the rules limits the number of characters on a single line. Normally not a huge deal... except when dealing with long URIs or other long strings.

The problem is, I have to change URI Hiera data so infrequently that I always forget exactly what the syntax is. Admittedly, this post is really for me to look back on and reference, but why not share?

And while we're on the topic:

YAML Cookbook for Ruby is an awesome resource for YAML syntax in general.
Puppet parses YAML using the following function in Ruby:

  YAML.safe_load(File.read(path), [Symbol], [], true)

Combine with Interactive Ruby Shell or ruby -e for a quick and easy way to test any YAML file you create.

Plain Style

"Flow" Styles

No character escaping, or characters matching /[#:]/.

---
key: This is a very
     long string.

{ "key" => "This is a very long string." }

Single-quoted style

No special characters, no escaping. Literal ' must be doubled-up ('').

---
key: 'This isn''t
     a very short string.'

{ "key" => "This isn't a very short string." }

Double-quoted style

Characters matching /[\\"]/ must be escaped by a \ character. Common escape sequences may be used. Line concatenation with a trailing \.

VERY useful for long URIs.

---
key: "http://this.is.my\
     .very.long.string"

{ "key" => "http://this.is.my.very.long.string" }

Block Notation

Literal Style

---
key: |
  This is a very
  long string.

{ "key" => "This is a very\nlong string.\n" }

Folded Style

---
key: >
  This is a very
  long string.

{ "key" => "This is a very long string.\n" }

Block chomping indicator

You may notice that both strings have newlines attached to the end. Want those gone? Use a different chomping indicator:

|, >: "clip". Keeps the newline.
|+, >+: "keep". Keeps the newline, and also keeps tailing blank lines.
|-, >-: "strip". Removes the newline.

---
key0: >
  Do. Or do not.
  There is no try.

key1: >+
  I find your lack of
  faith disturbing.

key2: >-
  The Force will be
  with you. Always.

{
  "key0" => "Do. Or do not. There is no try.\n",
  "key1" => "I find your lack of faith disturbing.\n\n",
  "key2" => "The Force will be with you. Always."
}

Source: YAML v1.2 Specification

Avoiding Resource Conflicts in Puppet

Bryan Woolsey — Fri, 21 Aug 2020 03:51:10 +0000

One of the most common questions I get from coworkers new to Puppet is "how can I avoid resource conflicts?".

The typical resource conflict I run across is usually caused by multiple modules attempting to manage a common folder or package dependency. (C:\temp in particular seems to be a big hit for some reason.) If multiple modules manage the same resource, Puppet will throw a duplicate resource declaration error.

In an ideal world of roles and profiles and perfectly coded modules, resource conflicts should never occur. But, given that we don't live in an ideal world, there are a few helpful functions to avoid the issue.

Before we move on, It is important to note that the functions listed below are still impacted by evaluation order (explained in greater detail later). These functions are not magic "fix all" functions, and should be avoided if at all possible.

`defined`

The defined function is built into Puppet and will return true if a given resource is defined within the catalog. The function may be used within a conditional to check if a particular resource exists (yet).

unless defined(File['/tmp/shared']) {
  # do stuff ...
  file { '/tmp/shared':
    ensure => 'directory',
  }
}

`defined_with_params`

This function (and every function hereafter) is provided by the puppetlabs/stdlib Forge module. As the name suggests, this function checks for a resource with specific attributes.

if defined_with_params(User['johndoe'], {'ensure' => 'present'}) {
  # do stuff ...
} else {
  user { 'johndoe':
    ensure => absent,
  }
}

Check the puppetlabs/stdlib README for more information.

`ensure_resource`

This function is pure gold. Given a resource type, title, and attribute hash, this function will create a resource in the catalog only if it does not already exist. If the resource does exist, but has different attributes, a duplicate resource definition error is thrown.

ensure_resource('file', '/tmp/shared', {'ensure' => 'directory'})

You can also use pass in arrays to create multiple resources with a single function call. Check out the README for more info.

`ensure_resources`

Essentially a souped-up version of the ensure_resource function that allows you to pass in a hash of resources. In my opinion, this function quickly makes your code unreadable, and I tend to avoid it. However, it definitely has its place, particularly when describing resources within Hiera.

ensure_resources('cron', {'backupjob' => { 'command' => '/usr/bin/backup', 'user' => 'root', 'hour' => 0 }, 'copyjob' => { 'command' => '/usr/sbin/copystuff', 'minute' => 30 } }, { 'ensure' => 'present' })

`ensure_packages`

Similar to the previous function, but specifically for package resources. I use it for dependencies when a node's package manager cannot sort it out on its own (namely Windows).

$my_pkgs = {
  'libcurl3' => { 'ensure' => 'present' },
  'zabbix' => { 'ensure' => '3.4.0' }
}

ensure_packages($my_pkgs)

Impact of evaluation order

How and when resources & functions are evaluated during catalog compilation may impact the result of these functions. Puppet does not always evaluate classes and resources in the order in which they are defined within a manifest. This evaluation-order independence has the potential to change the outcome of these functions and may make it seem like they are... well... broken.

Exactly how catalog compilation works gets into the nitty gritty of Puppet, and I won't pretend to completely understand how it works. All I can do is attempt to explain the issue as I understand it. (Feel free to yell at me if I get anything wrong.)

# Evaluated first
file { '/tmp/shared':
  ensure => 'directory',
}

# Evaluated second
unless defined(File['/tmp/shared']) {
  file { '/tmp/shared':
    ensure => 'directory',
  }
}
# This catalog will compile.

In the example above, the catalog will compile successfully, as the file resource is evaluated before the ensure_resource function. However, if the evaluation order is flipped, the catalog will fail to compile, as the ensure_resource function already added the File['/tmp/shared'] resource to the catalog before the file resource declaration is evaluated.

# Evaluated first
unless defined(File['/tmp/shared']) {
  file { '/tmp/shared':
    ensure => 'directory',
  }
}

# Evaluated second
file { '/tmp/shared':
  ensure => 'directory',
}

# This catalog will fail

Evaluation order can quickly become an issue when the same resources are added to the catalog in two completely different, unrelated classes. This issue can largely be avoided by using these handy functions in all classes that define the resource. That said, use some restriant to ensure these functions aren't used everywhere. If you have widespread use of these functions, chances are you aren't structuring your code base correctly.

The better way to access Puppet facts

Bryan Woolsey — Fri, 21 Aug 2020 03:51:01 +0000

I spend a lot of time working with facts in Puppet. I've probably spent more time figuring out the best way to manage custom external facts unique to our environment (node lifecycle status, datacenter, etc.) this past year than anything else in the Puppet universe. So I was rather surprised to find out there are better ways to access node facts within manfiests than using the standard $facts hash.

Most of the information that follows came from a recent office-hours conversation on the Puppet Community Slack. If you haven't already, I highly suggest you check out the Puppet Community Slack channel. It is a great place to bounce ideas off of Puppet engineers and other members of the community.

The fact variables

Classic top-scope variables

Node facts have always been accessible in Puppet using top scope variables. For example, the fact kernel is automatically available within a manifests in the variable $kernel.

if $kernel == 'Linux' {
  # do stuff...
}

The downside of using variables is that it isn't immediately obvious you are using a node fact in your manifest. May not be an issue for common well-known facts, but for custom facts it'll cause your manifest to be difficult to read. Many developers will explicitly call a top-scope variable (ie. $::kernel) as a hint, but it still doesn't completely resolve the issue.

The `$facts` hash

In Puppet 3.5 (I think), PuppetLabs added the $facts hash. Node facts are merged together into a single hash, with may be accessed using the fact name as the key.

unless $facts['kernel'] == 'Linux' {
  # do non-Linux stuff...
}

This is how I've always accessed facts within my manifests, and so far I haven't had any trouble. However, while this approach works for most facts, it does not work well for structured facts.

Take a node that has the following custom structured fact:

{
  "cmdb_data": {
    "datacenter": "virginia"
  }
}

If I wanted to retrieve the node's datacenter within my manifest, I could simply use the variable $facts['cmdb_data']['datacenter']. On nodes that do not have the custom structured fact (a newly provisioned node, for example), I had assumed the variable would simply have the value undef. Turns out this is not the case, in reality the catalog will blow up and fail to compile.

The functions that help

Luckily there are a few functions available to help you access structured facts.

`dig`

Similar to Ruby's built-in dig function, the Puppet variant allows you to "dig" into complex data structures. Refer to Puppet's function documentation for more information.

Example fact:

{
  "node_meta": {
    "owner": {
      "name": "johndoe"
    },
    "datacenter": "virginia"
  }
}

$facts.dig('node_meta', 'owner', 'name') # returns: 'johndoe'
$facts.dig('node_meta', 'enclosure', 'rack') # returns: undef

$facts['node_meta']['enclosure']['rack'] # fails with catalog compilation error

`fact`

A relatively new entry to the puppetlabs/stdlib forge module, the fact function behaves similarly to the dig function, but is explicitly used for facts. The function also supports dot notation, making it relatively compact and easy to read.

$server_owner = fact('node_meta.owner.name')

`get`

get is a new function added in Puppet 6. It provides generally the same functionality as dig and fact, but has the ability to return a default value if the value isn't present in the data structure. Like the fact function, this function uses dot notation for navigation values.

As an aside, for those who are huge fans of stdlib's pick function, the get function has the potential to replace pick in many situations.

$facts.get('node_meta.owner.name', 'no_owner_information')
$server_owner = get($facts, 'node_meta.owner.name', 'no_owner_information')

The get function also has an optional lambda for more advanced error handing should the value not exist. More information is available in Puppet's function
documentation.

TL;DR

You should probably be using the dig, fact, or get functions to retrieve fact values rather than top-scope variables or the $facts hash.

If you are on Puppet 6, the get function is a no-brainer in my opinion. If you can't make the jump to Puppet 6, my vote would be the fact function available in puppetlabs/stdlib.
dig is my least favorite of the three (not a fan of the array notation), but it will still get the job done.