Distar - A CPAN Distribution Author Tool

Graham Knop — Wed, 31 Mar 2021 01:42:35 +0000

Basic CPAN Tooling

When releasing a distribution on CPAN, authors need to include an installer script. Classically, this would be a Makefile.PL using ExtUtils::MakeMaker to generate a Makefile, which would then be used with make. Later, Module::Build was created, using a Build.PL script which served a similar purpose. The installer script would include information about the distribution, such as the name and prerequisites. This script, as well as the information inside it, would be used by a tool like CPAN.pm or cpanm to install the distribution and its prerequisites. This is the primary purpose of the tools, as something users interact with to test and install modules.

But since authors always had the install tool at hand, it became a convenient place to put author tools as well. The primary of these would be make dist which generates a distribution archive suitable for uploading to CPAN. For a long time, this was the primary way authors would create releases.

However, as an author tool, ExtUtils::MakeMaker leaves a lot to be desired. It uses a MANIFEST file to list the files to include in the distribution, which won't include new files unless the author remembers to run make manifest. And there are other sanity checks that authors should do before making a release that it doesn't provide for at all. Bumping version numbers, updating change logs, tagging and pushing to a VCS server. Some authors would build these tools into their Makefile.PL script, but that can become a maintenance burden. Other tools, like Module::Install were created to address some of this, by having a split between the author tools and user code, but it made some other problematic architectural choices.

Dist::Zilla

These issues led to the creation of tools like Dist::Zilla which entirely separate the author tooling from the user tooling. It would also take over writing the Makefile.PL script, since writing a proper Makefile.PL is not trivial, and being able to include arbitrary code is usually unneeded. It is based on a plugin system, allowing authors to tailor its behavior to whatever release process they prefer.

Dist::Zilla can be a fantastic tool, but comes with its own problems. There can be unexpected interactions between the multitude of plugins and versions, which can make the results of builds hard to reproduce without significant effort. It can also be difficult to express the desired build process via the plugin system, or to understand how it will behave by reading its configuration. The combination of these plugins often leads to a significant number of prerequisites, increasing the complexity and avenues of failure. It also regularly gets updated to take advantage of new perl features, which prevents it from being used on older perl versions¹. These issues are all trade-offs made to allow it to be very feature full and extremely flexible.

Distar

Not all authors agree with the trade-offs made by tools like Dist::Zilla. Matt S Trout wanted an author tool, but started from a different perspective. What needs to be added on top of what ExtUtils::MakeMaker provides to make it a viable author tool? The result of this thought process was Distar.

Distar is a tool that adds just enough to ExtUtils::MakeMaker so you can't bungle a release. And it is small enough to be able to read and understand the entire codebase at the same time.

It uses an allowlist to include files in the dist, rather than ExtUtils::MakeMaker's normal denylist, preventing extraneous files in the working directory from being shipped. It makes certain that the MANIFEST is up to date. It ensures you are testing against the same thing you are releasing. It forces you to add a Changelog entry for your release. It uploads to CPAN, and tags and pushes to git. And it does very little else.

Since its creation, Distar has seen very little adoption. The only other CPAN author using it that I'm aware of is me. I've also done most of the development for the last several years, making it more robust, and including a few quality of life improvements like version bumping.

Distar does not make any changes from files in the repository vs the files shipped in the distribution. It generates a README, a LICENSE file, and the default files created by ExtUtils::MakeMaker, META.json and META.yml.

How Does Distar Work?

To use Distar, you start with a normal Makefile.PL:

use ExtUtils::MakeMaker;
WriteMakefile(NAME => "My::Cool::Module");

Before the call to WriteMakefile, you add:

(do './maint/Makefile.PL.include' or die $@) unless -f 'META.yml';

Since the META.yml file is generated by ExtUtils::MakeMaker in the distribution directory it creates, but not the working directory, it is a reliable check for "author mode" in a pure EUMM environment.

You then create the file maint/Makefile.PL.include with the content:

BEGIN { -e 'Distar' or system qw(git clone https://github.com/p5sagit/Distar.git) }
use lib 'Distar/lib';
use Distar;

author 'A. U. Thor <author@cpan.org>';

Distar provides essentially no configuration aside from this. Any customization needed in the build process needs to be written in the Makefile.PL, the same way it would be for raw ExtUtils::MakeMaker.

To make a release using Distar, the process is:

make bump           # bump version number
make nextrelease    # add changelog version heading
make release        # test & release

make release

The core of Distar is the release target. It does a number of sanity checks, then creates a release.

Verify that the version numbers in all of the modules are matching.
Verify that the MANIFEST file is up to date.
Verify that cpan-upload is available to push to CPAN.
Fetches any changes from the remote git server
Verify that no git tag exists for the version being published.
Verify that the release is being made from the master branch.
Verify that there are no new commits on the remote git server.
Verify that there is a version line in the Changelog that has been staged in git.
Verify that there are no other uncommitted changes.
Generate a dist directory, including a README and LICENSE file.
Run the tests in the dist directory, with RELEASE_TESTING set.
Create a git commit for the release.
Create a git tag for the release.
Create a dist tarball.
Upload the dist tarball to CPAN.
Push the new tag and commit to the remote git server.

make bump

Distar provides a version bumping tool to modify any modules or Makefile.PL containing the version number. It is meant to work with both numeric and dotted-decimal style versions. By default, it will bump the last segment of the dotted-decimal version, or the highest precision portion of a numeric version. It also provides other targets to bump specific parts of a version. When bumping a specific part of a version, numeric versions will be treated as X.YYYZZZ versions, the same way perl versions work. Any components following the one specified will be reset to 0.

make bumpmajor                # bump the first digit component
make bumpminor                # bump the second digit component
make bumpbugfix               # bump the third digit component

Other options can be used via the V option.

make bump V=1.200             # bump to specific version
make bump V=--alpha           # bump last component, but add an underscore
                              # to indicate a developer release
make bump V='major --alpha'   # bump first digit component, but also add
                              # an underscore.

After bumping the version, the changes will be committed to git. This will work even if there are other changes in the working directory, leaving those changes uncommitted. The Makefile.PL will then be re-run pick up the new version.

make nextrelease

The release process of a Distar dist requires you to add a version line to your Changelog file, and stage that change (git add). The nextrelease target will do this for you.

It adds the version line, then uses git's interactive add on it, so you will need to confirm the changes. The diff shown should include all of the changelog entries written for the release.

git clone Distar

The current way Distar is distributed is by adding code to Makefile.PL which clones the repository and uses that. Some might think this would present a compatibility or stability issue, but the authors of Distar are very committed to maintaining stability and compatibility. Users have to trust that the authors will always keep the code in git in a working state. This trust has been easy to maintain so far, since the known users of Distar are its authors.

There may be other users, since its code is small enough that others could understand it without needing any discussion with Matt S Trout or myself.

Not being published on CPAN limits the visibility of Distar. I have, at times, thought of this as a feature. I've been uncertain if I wanted the pressure of additional users, and having to either accomodate or push back against alternate use cases and feature requests.

Should You Use Distar?

I have no idea. It works well for me, and fits the model I have for creating releases. It isn't as fully automatic as many Dist::Zilla setups are, but it is extremely predictable, backwards compatible, and lightweight. It is very opinionated. It is unlikely to become more configurable to cover differing use cases.

If others did like the idea of Distar, it may be time to look at improving the documentation and publishing it on CPAN.

Distar is currently available on GitHub at https://github.com/p5sagit/Distar.

It prevents authors from using older perl versions, but the CPAN releases it creates are still backward compatible. ↩

DEV Community: Graham Knop