Updating modules to Puppet 8

Puppet's content team recently upgraded all of our Supported Puppet modules to Puppet 8 compatibility. Since this is perhaps the largest effort in a Puppet infrastructure update, we thought that we'd write out a quick guide for how we went about it.

The process happened in three different stages. The preliminary work stage, the dependency update stage, and then finally the testing stage to make sure we didn't break anything!

The Puppet 8 compatibility guide detailed the major changes we had to be aware of and we found it invaluable the effort. It's probably worth bookmarking that page until you're done with the upgrade.

Preliminary work

During this stage, we aimed at getting the modules ready for the jump to the new versions by removing deprecated code constructs and rewriting code accordingly. The biggest changes in Puppet 8 are in the Ruby layer, so if you haven't written custom types/providers or functions or the like, your most common required changes are likely to be:

1. Assigning datatypes to class parameters
2. Replacing legacy facts with structured facts
3. General lint cleanup of top scope facts/variables
4. Cleaning up code to work with strict mode

You'll be happy to know that puppet-lint will help you by highlighting nearly all of these areas of concern. This means that a clean pdk validate run should show you code that needs to be corrected. However, you should first ensure that your modules don't have exceptions in place suppressing these warnings.

Here are a few pull requests demonstrating code updates, including removing lint exceptions from the Rakefile, .sync.yml and .puppet-lint.rc files.

• (CONT-359) Syntax update by LukasAud · Pull Request #1532 · puppetlabs/puppetlabs-mysql
• (CONT-362) Syntax update by LukasAud · Pull Request #273 · puppetlabs/puppetlabs-registry
• (Maint) Addressing Selinux legacy facts by LukasAud · Pull Request #2428 · puppetlabs/puppetlabs-apache

These resources will help you make the required changes:

• A list of Puppet data types and what they represent.
• A full list of the core facts, including both modern and legacy versions. The legacy_fact lint check will often recommend modern facts to transition to, but there's not always a direct one-to-one translation, so you may need this page to help identify which modern fact to use.
• The puppetlabs-stdlib 9.0 release removed quite a few long deprecated functions. They've been printing deprecation warnings for quite some time, but those can be easy to miss. It's likely that you'll still have a few in your codebase. This blog post lists most of them, but you can also refer to the latest 8.x reference page and search for the deprecation warnings.
• For a short period of time, two puppet-lint checks could have been installed twice, leading to already Initialized constant warnings. If you run into this issue, see this guide to resolve it.
• We used to require a weird meta gem to help ensure that the proper gems were installed for the versions of Puppet & Ruby your testing platform is using. This is no longer needed, and you can use this guide to remove the puppet-modules-gems meta-gem.

Dependency update

Now we're on to dependencies! This means ensuring that the proper versions of Puppet, dependent modules, and testing pipeline gems are all referenced correctly. Much of this is handled by pdk update, which will update all the PDK managed files to their latest versions. However, you'll also want to go through any modules dependencies in metadata.json and update any module dependencies to current versions that also support Puppet 8. You'll want to validate these as you go because sometimes major module releases might require downstream changes in your module to accommodate.

Here are some examples of this update, including automated updates and manual changes:

• (CONT-850) Puppet 8 support / Drop Puppet 6 by LukasAud · Pull Request #441 · puppetlabs/puppetlabs-accounts
• (CONT-850) Puppet 8 support / Drop Puppet 6 by LukasAud · Pull Request #441 · puppetlabs/puppetlabs-accounts

Testing

The final stage consisted of a series of manual and automated test runs to ensure that the module continues to work correctly once all the updates are applied. Most of our testing infrastructure is automated. It tests against a large set of containers representing all operating systems and Puppet versions the module supports. It runs the PDK's comprehensive test suite and our set of acceptance tests. Because this was a major version boundary, we also did some manual testing by running tests locally.

So long as the module also has a comprehensive testing coverage, passing tests locally and in CI should be a good indicator of a successful Puppet 8 update.

Third party Forge modules

Now comes the less satisfying part. You've made sure that all your in-house modules will work with Puppet 8. Now you need to check the Forge to validate all the community modules you use. For the most part, we recommend trusting the upstream author to do the update and validation.

Go to the Forge page for each module listed in your Puppetfile and validate that it claims Puppet 8 compatibility. If it does not, then add a comment to that line of your Puppetfile so that you know which modules to check next time you look at it.

After you're finished, if there are any modules marked as not Puppet 8 compatible, just save the file and come back to it in a week or a month and try again. You might consider following the Project URL link to the source repository and filing an issue asking for Puppet 8 support. If you're truly adventurous, you might consider cloning the repository and following this process on that module and submitting a pull request to add Puppet 8 support yourself!

Wrap up

Good luck on your Puppet 8 journey and we look forward to seeing you on the other side. It's beyond the scope of this guide, but you might also consider setting up Onceover to do full control repository smoke testing for major changes like this. You might even look into Octocatalog-diff to get fine-grained understanding of exactly what might change on each node.

Watch this space for a more detailed and advanced guide for some of the edge cases you might run into!