Puppet Code Quality (3 Part Series)
After months and years of using Puppet, the code base becomes increasingly complex and cluttered. How can you ensure its quality, as well as clean up unused code?
In the Puppet world, puppet-lint is the reference for code quality. It is used as a standard to check that modules follow the style guide, ensuring consistency in coding style and practices. puppet-lint can also be used in your control repository, to check your private modules (such as roles & profiles).
$ pdk convert $ pdk validate
Rakefile method is a easy way to automate
Rake::Task[:lint].clear PuppetLint::RakeTask.new :lint do |config| config.ignore_paths = [ 'modules/**/*.pp', 'vendor/**/*', ] config.disable_checks = [ '80chars', 'documentation', ] config.fail_on_warnings = true config.fix = true if ENV['PUPPETLINT_FIX'] == 'yes' end
Gemfile in your repository to install
source ENV['GEM_SOURCE'] || "https://rubygems.org" group :development, :test do gem 'rake', :require => false gem 'puppet-lint', :require => false # Other lint plugins (optional) gem 'puppet-lint-spaceship_operator_without_tag-check', :require => false gem 'puppet-lint-unquoted_string-check', :require => false gem 'puppet-lint-undef_in_function-check', :require => false gem 'puppet-lint-leading_zero-check', :require => false gem 'puppet-lint-trailing_comma-check', :require => false gem 'puppet-lint-file_ensure-check', :require => false gem 'puppet-lint-version_comparison-check', :require => false # You can also use the voxpupuli-test gem, # which pulls rake, puppet-lint & plugins as dependencies gem 'voxpupuli-test', :require => false end
You can then run the lint with:
$ bundle install --path vendor/bundle $ bundle exec rake lint # And if you want to autofix the detected mistakes $ PUPPETLINT_FIX=yes bundle exec rake lint
Onceover is a toolbox to automate tasks for Puppet control repositories. Among other things, its code quality plugin allows to run syntax checks and invoke
In order to use it, update your
source ENV['GEM_SOURCE'] || "https://rubygems.org" group :development, :test do gem 'voxpupuli-test', :require => false gem 'onceover', :require => false gem 'onceover-codequality', :require => false end
Refresh your bundle and run onceover:
$ bundle update $ bundle exec onceover run codequality --no-docs
Ideally, run this on every commit in a Continuous Integration/Continuous Deployment setup. At Camptocamp, we use a GitLab CI pipeline to check our control repo using Onceover before deploying it with r10k (also running a GitLab CI runner).
You've checked the quality of your existing code. Good! But what if you're actually maintaining and cleaning code that you don't use anymore? This would be quite the waste of time... At Camptocamp, we've built on
puppet-lint to provide a system to detect unused code and help us clean it up.
This is what the puppet-ghostbuster project is for. Under the hood,
puppet-ghostbuster is a collection of
puppet-lint plugins, distributed in a single
These plugins analyze your Puppet code and then connect to your PuppetDB to check if that code is actually used for any known node. It can also check Hiera data for unused keys. Just as previously, you can set it up as a Rake task, but our current setup requires a patch to
puppet-lint in order to whitelist the
puppet-lint checks activated (the current release of
puppet-lint only supports blacklisting checks in Rake tasks).
source ENV['GEM_SOURCE'] || "https://rubygems.org" group :development, :test do gem 'rake', :require => false gem 'puppet-lint', :require => false, :git => 'https://github.com/raphink/puppet-lint', :ref => '2cac4fb' # Includes patch for whitelisting checks gem 'puppet-ghostbuster', :require => false end
You can then set up a Rake task such as this one:
PuppetLint::RakeTask.new :ghostbuster do |config| config.pattern = ['./site/**/*'] config.only_checks = [ 'ghostbuster_classes', 'ghostbuster_defines', 'ghostbuster_facts', 'ghostbuster_files', 'ghostbuster_functions', 'ghostbuster_hiera_files', 'ghostbuster_templates', 'ghostbuster_types', ] config.fail_on_warnings = true end
puppet-ghostbuster requires info to connect to your PuppetDB, so you need to provide the following environment variables:
PUPPETDB_URL: URLs to PuppetDB
PUPPETDB_CERT_FILE: path to the certificate to use to connect to PuppetDB
PUPPETDB_KEY_FILE: path to the key to use to connect to PuppetDB
PUPPETDB_CACERT_FILE: path to the Puppet CA certificate
HIERA_YAML_PATH: path to
If you don't want to provide certificates and keys, you can connect to the PuppetDB through the unencrypted port 8080, for example by forwarding it through SSH. At Camptocamp, we're automating this setup by using summon as a wrapper to launch the command.
We store the certificates and keys to connect to PuppetDB in gopass, then provide a
secrets.yml file like so:
PUPPETDB_URL: https://puppetdb.example.com:8081 PUPPETDB_CERT_FILE: !var:file path/to/secret:cert PUPPETDB_KEY_FILE: !var:file path/to/secret:key PUPPETDB_CACERT_FILE: !var:file path/to/secret:cacert HIERA_YAML_PATH: ./hiera.yaml
Which allows to run:
$ summon bundle exec rake ghostbuster
This returns a list of classes, defines, files, templates, etc. that are unused in our code. We can then check these results and clean up our code!
This post was originally published on https://www.camptocamp.com/actualite/cleaning-up-puppet-code/
Level up every day