<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Tony Green</title>
    <description>The latest articles on DEV Community by Tony Green (@albatrossflavour).</description>
    <link>https://dev.to/albatrossflavour</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1055806%2F402f4836-b395-4931-8b37-ffc6e9f0e73c.jpeg</url>
      <title>DEV Community: Tony Green</title>
      <link>https://dev.to/albatrossflavour</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/albatrossflavour"/>
    <language>en</language>
    <item>
      <title>Handling Dirty Frag and Copy Fail with Puppet</title>
      <dc:creator>Tony Green</dc:creator>
      <pubDate>Wed, 13 May 2026 21:00:55 +0000</pubDate>
      <link>https://dev.to/puppet/handling-dirty-frag-and-copy-fail-with-puppet-6ff</link>
      <guid>https://dev.to/puppet/handling-dirty-frag-and-copy-fail-with-puppet-6ff</guid>
      <description>&lt;h2&gt;
  
  
  Do you know which of your Linux servers are vulnerable right now?
&lt;/h2&gt;

&lt;p&gt;Two critical Linux kernel vulnerabilities are being actively exploited. Dirty Frag targets the IP fragment reassembly modules that almost every Linux server has loaded by default. Copy Fail targets the AF_ALG cryptographic subsystem's AEAD interface. Both can allow an attacker to gain escalated local privileges, and both have interim mitigations you can deploy right now, before the kernel patches land.&lt;/p&gt;

&lt;p&gt;The first question every team needs to answer is not "how do we fix it?" but "how many of our systems are actually exposed?" Not tomorrow, after someone runs a scanner. Not next week, when the security team finishes their audit. Right now.&lt;/p&gt;

&lt;p&gt;If you are running Puppet, the answer is already within reach. You just need a fact.&lt;/p&gt;

&lt;h3&gt;
  
  
  Add these to your Puppetfile. That is it.
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight ruby"&gt;&lt;code&gt;&lt;span class="n"&gt;mod&lt;/span&gt; &lt;span class="s1"&gt;'albatrossflavour-dirty_frag'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'1.0.1'&lt;/span&gt;
&lt;span class="n"&gt;mod&lt;/span&gt; &lt;span class="s1"&gt;'albatrossflavour-copy_fail'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'1.0.0'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each of these open-source modules, that I have published to the Forge, ship a custom structured fact that reports vulnerability exposure on every node. No class to include. No Hiera changes. No manifest edits. Next Puppet run, the data is there.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://github.com/albatrossflavour/dirty_frag" rel="noopener noreferrer"&gt;dirty_frag&lt;/a&gt; reports exposure to Dirty Frag (&lt;a href="https://nvd.nist.gov/vuln/detail/CVE-2026-43284" rel="noopener noreferrer"&gt;CVE-2026-43284&lt;/a&gt; and &lt;a href="https://nvd.nist.gov/vuln/detail/CVE-2026-43500" rel="noopener noreferrer"&gt;CVE-2026-43500&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/albatrossflavour/copy_fail" rel="noopener noreferrer"&gt;copy_fail&lt;/a&gt; reports exposure to Copy Fail (&lt;a href="https://nvd.nist.gov/vuln/detail/CVE-2026-31431" rel="noopener noreferrer"&gt;CVE-2026-31431&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  See where you stand
&lt;/h3&gt;

&lt;p&gt;Once deployed, every node reports its exposure state as structured data. Here is what the output looks like.&lt;/p&gt;

&lt;h4&gt;
  
  
  dirty_frag
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"esp4"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"loaded"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"blocked"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"available"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"esp6"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"loaded"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"blocked"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"available"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"rxrpc"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"loaded"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"blocked"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"available"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"vulnerable"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"reboot_required"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h4&gt;
  
  
  copy_fail
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"algif_aead"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"builtin"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"loaded"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"active"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"blocked"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"available"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"initcall_blacklisted"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"vulnerable"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"mitigated"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"reboot_required"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Both facts surface the same two summary keys that give you the answers you actually care about. &lt;code&gt;vulnerable&lt;/code&gt; is &lt;code&gt;true&lt;/code&gt; if the exploitable module is active. &lt;code&gt;reboot_required&lt;/code&gt; is &lt;code&gt;true&lt;/code&gt; if you have applied the mitigation but the module is still sitting in kernel memory.&lt;/p&gt;

&lt;p&gt;That &lt;code&gt;reboot_required&lt;/code&gt; flag is the one that catches people. You apply the mitigation, see the block in place, and assume you are safe. But the module is still loaded and exploitable until the machine restarts. The facts make this gap visible.&lt;/p&gt;

&lt;h3&gt;
  
  
  Query your entire fleet in one line
&lt;/h3&gt;

&lt;p&gt;The facts land in PuppetDB like any other structured fact. No extra tooling, no scheduled scans, no agents phoning home to a separate platform. It is just data, maintained automatically every Puppet run.&lt;/p&gt;

&lt;p&gt;Show me every node vulnerable to Dirty Frag:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;puppet query &lt;span class="s1"&gt;'facts[certname, value] { name = "dirty_frag" and value.vulnerable = true }'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Show me every node vulnerable to Copy Fail:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;puppet query &lt;span class="s1"&gt;'facts[certname, value] { name = "copy_fail" and value.vulnerable = true }'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Show me nodes where the Dirty Frag block is applied but a reboot is still needed:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;puppet query &lt;span class="s1"&gt;'facts[certname, value] { name = "dirty_frag" and value.reboot_required = true }'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Show me nodes where &lt;code&gt;algif_aead&lt;/code&gt; is built into the kernel (the harder mitigation path):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;puppet query &lt;span class="s1"&gt;'facts[certname, value] { name = "copy_fail" and value.algif_aead.type = "builtin" }'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is the bit that makes Puppet's model shine. You are not running a point-in-time scan. The data updates every run, automatically. As nodes get patched, rebooted, or have blocks applied, the numbers go down on their own. You can watch your exposure shrink in real time without maintaining anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  What makes a system vulnerable?
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Dirty Frag
&lt;/h3&gt;

&lt;p&gt;The Dirty Frag vulnerability sits in three Linux kernel modules used for IP fragment reassembly: &lt;code&gt;esp4&lt;/code&gt;, &lt;code&gt;esp6&lt;/code&gt;, and &lt;code&gt;rxrpc&lt;/code&gt;. If any of those modules is loaded, the system is exposed. The two CVEs chain together to allow privilege escalation and remote code execution through these modules.&lt;/p&gt;

&lt;p&gt;These modules are almost universally loaded on production Linux servers. If you are running iptables, firewalld, Docker, Kubernetes, or any network policy enforcement, odds are at least &lt;code&gt;esp4&lt;/code&gt; is there. This one affects nearly everything.&lt;/p&gt;

&lt;h3&gt;
  
  
  Copy Fail
&lt;/h3&gt;

&lt;p&gt;The vulnerability is a use-after-free in the AF_ALG subsystem's AEAD interface (&lt;code&gt;algif_aead&lt;/code&gt;). An unprivileged user can trigger it by opening an AF_ALG socket and exercising the AEAD code path in a specific way.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;algif_aead&lt;/code&gt; is present on most stock kernels but rarely used for legitimate work. The main consumers are some VPN implementations that offload crypto to the kernel, &lt;code&gt;kcapi-tools&lt;/code&gt;, and certain FIPS-certified configurations. On systems where the module is loadable, an attacker can force-load it themselves just by opening the socket. On systems where it is built-in, it may already be active regardless of use.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why module presence, not kernel version?
&lt;/h3&gt;

&lt;p&gt;The real fix for both vulnerabilities is a kernel patch, but every distribution ships their own patched versions on their own schedules with their own version numbers. Tracking "which kernel version is safe" across Red Hat, Ubuntu, SUSE, Amazon Linux, and the rest is a moving target that goes stale the day you publish it. Module presence is the reliable signal: if the vulnerable module is active, you are exposed, regardless of kernel version.&lt;/p&gt;

&lt;p&gt;This is also what the vendor advisories recommend. Red Hat's &lt;a href="https://access.redhat.com/security/vulnerabilities/RHSB-2026-003" rel="noopener noreferrer"&gt;RHSB-2026-003&lt;/a&gt; and the equivalent Ubuntu and SUSE bulletins all point to the same interim mitigation: prevent the modules from loading using &lt;code&gt;install /bin/false&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  When you want Puppet to enforce the block
&lt;/h2&gt;

&lt;p&gt;The facts give you visibility. If you also want Puppet to actively prevent the modules from loading, include the classes.&lt;/p&gt;

&lt;h3&gt;
  
  
  Dirty Frag
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight puppet"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s1"&gt;'dirty_frag'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="py"&gt;mitigate_esp4&lt;/span&gt;  &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="py"&gt;mitigate_esp6&lt;/span&gt;  &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="py"&gt;mitigate_rxrpc&lt;/span&gt; &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Copy Fail
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight puppet"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s1"&gt;'copy_fail'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="py"&gt;mitigate_algif_aead&lt;/span&gt; &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Both classes write &lt;code&gt;install &amp;lt;module&amp;gt; /bin/false&lt;/code&gt; directives to modprobe.d configuration files. From that point on, any attempt to load those modules (whether by autoloading, a dependency chain, or an explicit &lt;code&gt;modprobe&lt;/code&gt;) will run &lt;code&gt;/bin/false&lt;/code&gt; instead of loading the actual module code. The module simply cannot get into the kernel.&lt;/p&gt;

&lt;p&gt;This is worth understanding because it is stronger than the kernel's &lt;code&gt;blacklist&lt;/code&gt; directive. A &lt;code&gt;blacklist&lt;/code&gt; entry only prevents autoloading, it does not stop explicit &lt;code&gt;modprobe&lt;/code&gt; calls. &lt;code&gt;install /bin/false&lt;/code&gt; intercepts the load at every entry point.&lt;/p&gt;

&lt;p&gt;If the module was already loaded before the block was applied, it stays in memory until the next reboot. The &lt;code&gt;reboot_required&lt;/code&gt; flag in both facts makes this gap visible.&lt;/p&gt;

&lt;p&gt;This is declarative and safe. Puppet manages the config file, ensures the desired state on every run, and the classes are entirely opt-in. If your patching process handles the fix, or you are rolling out kernel updates, you might not need them at all. The facts alone give you the visibility to track progress.&lt;/p&gt;

&lt;p&gt;Both modules ship with Hiera data defaults, so if you prefer to drive parameters through your hierarchy, see each module's README for examples.&lt;/p&gt;

&lt;h2&gt;
  
  
  The built-in module problem (Copy Fail only)
&lt;/h2&gt;

&lt;p&gt;There is a wrinkle with Copy Fail that Dirty Frag does not have. On most stock distribution kernels, &lt;code&gt;algif_aead&lt;/code&gt; is compiled directly into the kernel image as a built-in module. It is not a loadable &lt;code&gt;.ko&lt;/code&gt; file, so modprobe.d has no effect on it.&lt;/p&gt;

&lt;p&gt;For built-in modules, the mitigation is &lt;code&gt;initcall_blacklist=algif_aead_init&lt;/code&gt; on the kernel command line (via GRUB configuration). This prevents the module's init function from running on boot. It requires a reboot to take effect.&lt;/p&gt;

&lt;p&gt;The copy_fail module does not automate GRUB changes. Getting boot configuration wrong can render a system unbootable. You can manage GRUB with Puppet (using &lt;code&gt;file_line&lt;/code&gt; or &lt;code&gt;augeas&lt;/code&gt;), but that is out of scope for this module. The fact will report &lt;code&gt;initcall_blacklisted: true&lt;/code&gt; once the parameter is in place, and &lt;code&gt;reboot_required: true&lt;/code&gt; until the reboot completes.&lt;/p&gt;

&lt;p&gt;The &lt;code&gt;type&lt;/code&gt; field in the fact output tells you which mitigation path applies to each node:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;builtin&lt;/code&gt;&lt;/strong&gt;: needs &lt;code&gt;initcall_blacklist&lt;/code&gt;, modprobe.d has no effect&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;loadable&lt;/code&gt;&lt;/strong&gt;: the class and modprobe.d approach works&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;absent&lt;/code&gt;&lt;/strong&gt;: not vulnerable, nothing to do&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Force-unload a module right now (with care)
&lt;/h2&gt;

&lt;p&gt;Sometimes you cannot wait for a reboot. Both modules ship a Bolt task that removes the module from the running kernel immediately:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;bolt task run dirty_frag::unload &lt;span class="nv"&gt;module&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;esp4 &lt;span class="nt"&gt;--targets&lt;/span&gt; servers
bolt task run copy_fail::unload &lt;span class="nv"&gt;module&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;algif_aead &lt;span class="nt"&gt;--targets&lt;/span&gt; servers
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A word of caution. Unloading a kernel module is not the same as flipping a config switch. If the module is actively in use (for example, &lt;code&gt;esp4&lt;/code&gt; underpins IPsec tunnels, &lt;code&gt;esp6&lt;/code&gt; handles IPv6 ESP, and &lt;code&gt;rxrpc&lt;/code&gt; is used by AFS), pulling it out from under a running service can drop connections, crash VPN tunnels, or cause dependent subsystems to fail. This is exactly why the Puppet classes do not do it automatically. The classes apply the block and wait for a reboot. The Bolt tasks give you the lever to pull when you have decided the tradeoff is worth it.&lt;/p&gt;

&lt;p&gt;Each task validates the module name, checks it is actually loaded, and gives you structured output. If the module is in use by another kernel subsystem and cannot be unloaded, the task will tell you, and you are back to "apply the block and schedule a reboot".&lt;/p&gt;

&lt;p&gt;Test on non-production nodes first. Know what the module is doing on that system before you yank it. If in doubt, the block-and-reboot path is always the safer option.&lt;/p&gt;

&lt;p&gt;The unload task only works for loadable modules. Built-in modules cannot be unloaded.&lt;/p&gt;

&lt;h2&gt;
  
  
  The workflow
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Add the modules to your Puppetfile.&lt;/strong&gt; Deploy. Done. Every node now reports its exposure.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Query PuppetDB&lt;/strong&gt; to see where you stand. One-liner, fleet-wide answer.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Decide on mitigation.&lt;/strong&gt; Apply the classes via Hiera for persistent blocking.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Use Tasks&lt;/strong&gt; to immediately unload modules on critical systems that cannot wait.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Watch the numbers drop.&lt;/strong&gt; The facts update every run. No manual tracking, no spreadsheets, no scheduled scans.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The whole approach is declarative. You tell Puppet what state you want, and it converges towards it. The facts keep reporting reality. The gap between the two is your exposure, and it is always visible.&lt;/p&gt;

&lt;h2&gt;
  
  
  Get started
&lt;/h2&gt;

&lt;p&gt;Both of these open source modules are available on GitHub:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://github.com/albatrossflavour/dirty_frag" rel="noopener noreferrer"&gt;albatrossflavour/dirty_frag&lt;/a&gt; (Puppet 7/8, no dependencies)&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/albatrossflavour/copy_fail" rel="noopener noreferrer"&gt;albatrossflavour/copy_fail&lt;/a&gt; (Puppet 7/8, no dependencies)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both modules have been built for RedHat, CentOS, Ubuntu, Debian, Amazon Linux, and SLES.&lt;/p&gt;

</description>
      <category>puppet</category>
      <category>vulnerabilities</category>
      <category>automation</category>
      <category>devops</category>
    </item>
    <item>
      <title>Using Let's Encrypt with the Puppet Enterprise console</title>
      <dc:creator>Tony Green</dc:creator>
      <pubDate>Thu, 30 Mar 2023 22:28:40 +0000</pubDate>
      <link>https://dev.to/puppet/using-lets-encrypt-with-the-puppet-enterprise-console-21l4</link>
      <guid>https://dev.to/puppet/using-lets-encrypt-with-the-puppet-enterprise-console-21l4</guid>
      <description>&lt;p&gt;Had an itch I've been meaning to scratch for a while.  I build my &lt;a href="https://puppet.com" rel="noopener noreferrer"&gt;Puppet&lt;/a&gt; environment using &lt;a href="https://terraform.io" rel="noopener noreferrer"&gt;Terraform&lt;/a&gt;, which makes it nice and easy to tear things down and rebuild them.  That is great, but it does leave me with an issue when it comes to the console SSL certificates.&lt;/p&gt;

&lt;p&gt;Puppet will generate self-signed certs for the console, which work &lt;em&gt;fine&lt;/em&gt;, but it was always a niggle that the certs couldn't be automagically coaxed into being valid.&lt;/p&gt;

&lt;p&gt;Since moving over to &lt;a href="https://github.com/albatrossflavour/k3s-cluster" rel="noopener noreferrer"&gt;kubernetes&lt;/a&gt; for my home lab, I've come to expect managed SSL certificates for any public facing services, without me having to do anything.&lt;/p&gt;

&lt;p&gt;Finally set some time aside to look at the options and thought I'd publish the details of the journey as well as where I ended up.&lt;/p&gt;

&lt;h1&gt;
  
  
  Step 1 - Let's Encrypt
&lt;/h1&gt;

&lt;p&gt;Obviously I wasn't going to reinvent the wheel.  If I wanted to manage and use Let's Encrypt on a Puppet Enterprise server, I'd be using the &lt;a href="https://forge.puppet.com/modules/puppet/letsencrypt" rel="noopener noreferrer"&gt;Let's Encrypt module&lt;/a&gt;.  The module makes it very simple to get the relevant packages installed and configured.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight puppet"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s1"&gt;'letsencrypt'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="py"&gt;config&lt;/span&gt;     &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="py"&gt;email&lt;/span&gt;  &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'certs@albatrossflavour.com'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="py"&gt;config_dir&lt;/span&gt; &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'/etc/letsencrypt'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One of the beauties of this module is that it also sets up a &lt;code&gt;cron&lt;/code&gt; job to renew the generated certs, so you don't need to keep an eye on it.&lt;/p&gt;

&lt;p&gt;Classify your server with that and you'll end up with &lt;code&gt;certbot&lt;/code&gt; and it's dependencies.  Great start and feeling confident about the future!&lt;/p&gt;

&lt;p&gt;Then we need to generate a certificates:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight puppet"&gt;&lt;code&gt;&lt;span class="n"&gt;letsencrypt::certonly&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s1"&gt;'puppet.gcp.albatrossflavour.com'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="py"&gt;domains&lt;/span&gt;       &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'puppet.gcp.albatrossflavour.com'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="py"&gt;manage_cron&lt;/span&gt;   &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="py"&gt;plugin&lt;/span&gt;        &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'webroot'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="py"&gt;webroot_paths&lt;/span&gt; &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="err"&gt;'&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;var&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;www&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Annnnnnnd that's where things start to go south.&lt;/p&gt;

&lt;p&gt;When you request a cert, the most simple method of validating you are who you say you are is to have a web server on the host respond to a query sent to port &lt;code&gt;80&lt;/code&gt;.  Sure we say, easy, couldn't take much to do!&lt;/p&gt;

&lt;p&gt;By default, the Puppet Enterprise &lt;code&gt;nginx&lt;/code&gt; config does a &lt;code&gt;301&lt;/code&gt; redirect for &lt;code&gt;http&lt;/code&gt; requests to &lt;code&gt;https&lt;/code&gt;.  This means any queries from Let's Encrypt to validate the requests will end up as &lt;code&gt;https&lt;/code&gt; requests and the cert request will fail.&lt;/p&gt;

&lt;h1&gt;
  
  
  Step 2 - &lt;code&gt;nginx&lt;/code&gt;
&lt;/h1&gt;

&lt;p&gt;Let me introduce you to &lt;code&gt;puppet_enterprise::profile::console::proxy::http_redirect::enable_http_redirect&lt;/code&gt;.  This parameter controls if that redirect is in place.  So step .... 6(?) was to use &lt;code&gt;hiera&lt;/code&gt; to disable the &lt;code&gt;http&lt;/code&gt; redirect:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="s"&gt;❯ cat data/role/role::pe::master.yaml&lt;/span&gt;
&lt;span class="na"&gt;puppet_enterprise::profile::console::proxy::http_redirect::enable_http_redirect&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Once we run this through a &lt;code&gt;puppet&lt;/code&gt; run, the redirect gets removed!  Score!&lt;/p&gt;

&lt;p&gt;Only problem is, without the redirect, the &lt;code&gt;nginx&lt;/code&gt; server doesn't listen on port &lt;code&gt;80&lt;/code&gt;.  OK, we can fix that easily.  We could use the &lt;code&gt;pe_nginx::directive&lt;/code&gt; type, but I found it to be a bit of an overkill for what I needed.  Instead I opted for a simple template:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight puppet"&gt;&lt;code&gt;  &lt;span class="n"&gt;file&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s1"&gt;'/etc/puppetlabs/nginx/conf.d/certs.conf'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="py"&gt;ensure&lt;/span&gt;  &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;file&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;owner&lt;/span&gt;   &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'root'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;group&lt;/span&gt;   &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'root'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;mode&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'0644'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;content&lt;/span&gt; &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;template&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;${module_name}&lt;/span&gt;&lt;span class="s2"&gt;/cert_vhost.conf.erb"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="kp"&gt;notify&lt;/span&gt;  &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;Exec&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'pe_nginx'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;and the template is just:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight erb"&gt;&lt;code&gt;server {
  listen       80;
  server_name  &lt;span class="cp"&gt;&amp;lt;%=&lt;/span&gt; &lt;span class="vi"&gt;@fqdn&lt;/span&gt; &lt;span class="cp"&gt;%&amp;gt;&lt;/span&gt;;
  index        index.html;
  location /.well-known {
    root /var/www;
  }
  location / {
    return 301 https://$server_name$request_uri;
  }
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;(Sorry, still using &lt;code&gt;erb&lt;/code&gt;, I will &lt;strong&gt;at some point&lt;/strong&gt; rewrite everthing in &lt;code&gt;epp&lt;/code&gt;)&lt;/p&gt;

&lt;p&gt;The template keeps the &lt;code&gt;301&lt;/code&gt; redirect in place for anything &lt;em&gt;other&lt;/em&gt; than a request to &lt;code&gt;/.well-known&lt;/code&gt;, which is where Let's Encrypt looks for the validation info.&lt;/p&gt;

&lt;p&gt;Run this through and the &lt;code&gt;nginx&lt;/code&gt; vhost gets created.  However the &lt;code&gt;letsencrypt::certonly&lt;/code&gt; call still fails on the first run.  The &lt;code&gt;notify&lt;/code&gt; to the &lt;code&gt;pe_nginx&lt;/code&gt; service, which is done when we create the &lt;code&gt;cert_vhost.conf&lt;/code&gt;, doesn't happen in the order we need.  The Let's Encrypt module is trying to get a response &lt;em&gt;before&lt;/em&gt; we've setup the vhost.  Now this would work on subsequent runs, but &lt;strong&gt;Golden Rule #1&lt;/strong&gt; is to make sure, whenever possible, that you get a clean puppet run in one pass.  Plus this was a challenge.&lt;/p&gt;

&lt;p&gt;Before I worked on fixing that, I wanted to make sure I could make the rest of it work.  Let's sum up where we are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;I've got the Let's Encrypt client installed and configured&lt;/li&gt;
&lt;li&gt;I've got a new &lt;code&gt;nginx&lt;/code&gt; vhost running that allows the Let's Encrypt web validation queries through and redirects any other &lt;code&gt;http&lt;/code&gt; traffic.&lt;/li&gt;
&lt;li&gt;After a couple of puppet runs, I've got valid SSL certs in &lt;code&gt;/etc/letsencrypt&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h1&gt;
  
  
  Step 3 - The console certs
&lt;/h1&gt;

&lt;p&gt;It'd been a while since I played with the Puppet console SSL certs, so I checked in with &lt;a href="https://www.puppet.com/docs/pe/latest/use_a_custom_ssl_cert_for_the_console.html" rel="noopener noreferrer"&gt;the source of truth&lt;/a&gt;, which outlines the steps we need to go through to use custom SSL certs with the console:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Retrieve the custom certificate and private key.&lt;/li&gt;
&lt;li&gt;Move the certificate to &lt;code&gt;/etc/puppetlabs/puppet/ssl/certs/console-cert.pem&lt;/code&gt;, replacing any existing file named &lt;code&gt;console-cert.pem&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Move the private key to &lt;code&gt;/etc/puppetlabs/puppet/ssl/private_keys/console-cert.pem&lt;/code&gt;, replacing any existing file named &lt;code&gt;console-cert.pem&lt;/code&gt;.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;So we can just create a &lt;code&gt;file&lt;/code&gt; resource that takes the Let's Encrypt cert/key and places them into the console SSL directory structure.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight puppet"&gt;&lt;code&gt;  &lt;span class="n"&gt;file&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s1"&gt;'/etc/puppetlabs/puppet/ssl/certs/console-cert.pem'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="py"&gt;ensure&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;file&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;owner&lt;/span&gt;     &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'pe-puppet'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;group&lt;/span&gt;     &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'pe-puppet'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;links&lt;/span&gt;     &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'follow'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;mode&lt;/span&gt;      &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'0640'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;source&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s2"&gt;"/etc/letsencrypt/live/&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="s2"&gt;{facts['puppet_server']}/cert.pem"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;backup&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'.puppet_bak'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="kp"&gt;notify&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;Service&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'pe-nginx'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="kp"&gt;subscribe&lt;/span&gt; &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;Letsencrypt&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nc"&gt;Certonly&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$facts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'puppet_server'&lt;/span&gt;&lt;span class="p"&gt;]],&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="n"&gt;file&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s1"&gt;'/etc/puppetlabs/puppet/ssl/private_keys/console-cert.pem'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="py"&gt;ensure&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;file&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;owner&lt;/span&gt;     &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'pe-puppet'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;group&lt;/span&gt;     &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'pe-puppet'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;links&lt;/span&gt;     &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'follow'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;mode&lt;/span&gt;      &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'0644'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;source&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s2"&gt;"/etc/letsencrypt/live/&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="s2"&gt;{facts['puppet_server']}/privkey.pem"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="py"&gt;backup&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'.puppet_bak'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="kp"&gt;notify&lt;/span&gt;    &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;Service&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'pe-nginx'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="kp"&gt;subscribe&lt;/span&gt; &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;Letsencrypt&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nc"&gt;Certonly&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$facts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'puppet_server'&lt;/span&gt;&lt;span class="p"&gt;]],&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Sure enough, when I try this on the Puppet server, life is good and we have a console with valid SSL certs.&lt;/p&gt;

&lt;p&gt;Time to crack open a bottle of red and tick an item off my to do list.&lt;/p&gt;

&lt;p&gt;Damn it, &lt;strong&gt;Golden Rule #2&lt;/strong&gt; raises it's head.  It's not finished until you know it works fine from scratch... &lt;strong&gt;&lt;em&gt;WITHOUT&lt;/em&gt;&lt;/strong&gt; breaking &lt;strong&gt;Golden Rule #1&lt;/strong&gt;.&lt;/p&gt;

&lt;h1&gt;
  
  
  Step 4 - Rebuild
&lt;/h1&gt;

&lt;p&gt;Nuked the Puppet server and rebuilt it.  Lots of failures on the first run (we kinda expected that), however they didn't go away.  The &lt;code&gt;nginx&lt;/code&gt; service never gets restarted as the various dependencies can be resolved, but not in the way we need.&lt;/p&gt;

&lt;p&gt;This is the &lt;em&gt;fun&lt;/em&gt; (?) of declarative configuration management.  I can only &lt;strong&gt;&lt;em&gt;manage&lt;/em&gt;&lt;/strong&gt; things once, which includes only being able to &lt;code&gt;notify&lt;/code&gt; the &lt;code&gt;pe-nginx&lt;/code&gt; service once.  Then the compiler will figure out when the service is restarted, based on all of the dependencies in the catalog. &lt;/p&gt;

&lt;p&gt;I played around with a lot of options, some &lt;em&gt;waaaaay&lt;/em&gt; hackier than I wanted to go with.  Plus this was becoming a fun exercise.&lt;/p&gt;

&lt;p&gt;All I needed to be able to do was to inject a restart of the &lt;code&gt;pe-nginx&lt;/code&gt; service twice in a single run.&lt;/p&gt;

&lt;p&gt;To get there, I bent a few of the future &lt;strong&gt;Golden Rules&lt;/strong&gt; and came up with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight puppet"&gt;&lt;code&gt;&lt;span class="n"&gt;exec&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s1"&gt;'restart_nginx'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="py"&gt;command&lt;/span&gt;     &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'/bin/systemctl restart pe-nginx'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="py"&gt;refreshonly&lt;/span&gt; &lt;span class="p"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This &lt;code&gt;exec&lt;/code&gt; resource will allow me to do a restart of &lt;code&gt;pe-nginx&lt;/code&gt; outside of, and before, the &lt;code&gt;notify =&amp;gt; Service['pe-nginx']&lt;/code&gt; parameters.&lt;/p&gt;

&lt;p&gt;Yes, it's a hack, but it's a hack on the side of the angels.  Not only will it solve the issue of the process not working at all, but it will also allow the certs to be generated, and installed, in a single run.&lt;/p&gt;

&lt;h1&gt;
  
  
  Step 5 - Quick Robin, to the &lt;code&gt;pdk&lt;/code&gt; mobile
&lt;/h1&gt;

&lt;p&gt;I did a fair bit of testing and found the end result to be far more reliable and useful than I thought it would be.&lt;/p&gt;

&lt;p&gt;I first created a fully parameterised profile to manage the certs.  Once I got that working, it was a no-brainer to create a module to share the love.&lt;/p&gt;

&lt;p&gt;That's how I ended up with the &lt;a href="https://forge.puppet.com/modules/albatrossflavour/pe_console_letsencrypt/readme" rel="noopener noreferrer"&gt;pe_console_letsencrypt&lt;/a&gt; module.  You can also just &lt;a href="https://github.com/albatrossflavour/pe_console_letsencrypt/blob/main/manifests/init.pp" rel="noopener noreferrer"&gt;check out the code&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I've done a fair bit of testing, but I'm certainly not saying it's bulletproof.&lt;/p&gt;

&lt;h1&gt;
  
  
  Step 6 - &lt;a href="https://en.wikipedia.org/wiki/Gnomes_(South_Park)" rel="noopener noreferrer"&gt;Profit&lt;/a&gt;?
&lt;/h1&gt;

&lt;p&gt;I'm still scratching my head to think of another way I could get the certs generated and in place in a single run &lt;strong&gt;without&lt;/strong&gt; the &lt;code&gt;exec&lt;/code&gt; hack.  That being said, I've had hacks a lot worse go live in the past!&lt;/p&gt;

&lt;h1&gt;
  
  
  Note
&lt;/h1&gt;

&lt;p&gt;While researching the links for this post, I came across a &lt;a href="http://blog.graypockets.com/2017/10/lets-encrypt-puppet-enterprise-console.html" rel="noopener noreferrer"&gt;blog less frequently updated than mine&lt;/a&gt; which has a post from 2017 showing a similar way of achieving the same thing, but without the hack to make it work in a single run.&lt;/p&gt;

</description>
      <category>puppet</category>
      <category>ssl</category>
      <category>nginx</category>
      <category>tutorial</category>
    </item>
  </channel>
</rss>
