DEV Community: Mark Gardner

Porting from Perl to Go: Simplifying for Platform Engineering

Mark Gardner — Wed, 15 Oct 2025 16:18:32 +0000

Rewriting a script for the Homebrew package manager taught me how Go’s design choices align with platform-ready tools.

The problem with `brew upgrade`

By default, the brew upgrade command updates every formula (terminal utility or library). It also updates every cask (GUI application) it manages. All are upgraded to the latest version — major, minor, and patch. That’s convenient when you want the newest features, but disruptive when you only want quiet patch-level fixes.

Last week I solved this in Perl with brew-patch-upgrade.pl, a script that parsed brew upgrade’s JSON output, compared semantic versions, and upgraded only when the patch number changed. It worked, but it also reminded me how much Perl leans on implicit structures and runtime flexibility.

This week I ported the script to Go, the lingua franca of DevOps. The goal wasn’t feature parity — it was to see how Go’s design choices map onto platform engineering concerns.

Why port to Go?

Portfolio practice : I’m building a body of work that demonstrates platform engineering skills.
Operational focus : Go is widely used for tooling in infrastructure and cloud environments.
Learning by contrast : Rewriting a working Perl script in Go forces me to confront differences in error handling, type safety, and distribution.

The journey

Error handling philosophy

Perl gave me try/catch (experimental in the Perl v5.34.1 that ships with macOS, but since accepted into the language in v5.40). Go, famously, does not. Instead, every function returns an error explicitly.

use v5.34;
use warnings;
use experimental qw(try);
use Carp;
use autodie;

…

try {
  system ‘brew’, ‘upgrade’, $name;
  $result = ‘upgraded’;
}
catch ($e) {
  $result = ‘failed’;
  carp $e;
}

package main

import (
  “os/exec”
  “log”
)

…

cmd := exec.Command(“brew”, “upgrade”, name)
if output, err := cmd.CombinedOutput(); err != nil {
  log.Printf(“failed to upgrade %s: %v\n%s”,
    name,
    err,
    output)
}

The Go version is noisier, but it forces explicit decisions. That’s a feature in production tooling: no silent failures.

Dependency management

Perl : cpanfile + CPAN modules. Distribution means “install Perl (if it’s not already), install modules, run script.” Tools like carton and the cpan or cpanm commands help automate this. Additionally, one can use further tooling like fatpack and pp to build more self-contained packages. But those are neither common nor (except for cpan) distributed with Perl.
Go : go.mod + go build. Distribution is a single (platform-specific) binary.

For operational tools, that’s a massive simplification. No runtime interpreter, no dependency dance.

Type safety

Perl let me parse JSON into hashrefs and trust the keys exist. Go required a struct:

type Formula struct {
  Name string `json:"name"`
  CurrentVersion string `json:"current_version"`
  InstalledVersions []string `json:"installed_versions"`
}

The compiler enforces assumptions that Perl left implicit. That friction is valuable — it surfaces errors early.

Binary distribution

This is where Go shines. Instead of telling colleagues “install Perl v5.34 and CPAN modules,” I can hand them a binary. No need to worry about scripting runtime environments — just grab the right file for your system.

homebrew-semver-guard-darwin (Universal Binary for macOS)
homebrew-semver-guard-linux-amd64 (Intel/AMD 64-bit binary for Linux)
homebrew-semver-guard-linux-arm64 (Arm 64-bit binary for Linux)

Available on the release page. Download, run, done.

Semantic versioning logic

In Perl, I manually compared arrays of version numbers. In Go, I imported golang.org/x/mod/semver:

import (
  golang.org/x/mod/semver
)

…

if semver.MajorMinor(toSemver(formula.InstalledVersions[0])) !=
  semver.MajorMinor(toSemver(formula.CurrentVersion)) {
  log.Printf(“%s is not a patch upgrade”, formula.Name)
  results.skipped++
  continue
}

Cleaner, more legible, and less error-prone. The library encodes the convention, so I don’t have to.

Deliberate simplification

I didn’t port every feature. Logging adapters, signal handlers, and edge-case diagnostics remained in Perl. The Go version focuses on the core logic: parse JSON, compare versions, run upgrades. That restraint was intentional — I wanted to learn Go’s idioms, not replicate every Perl flourish.

Platform engineering insights

Three lessons stood out:

Binary distribution matters. Operational tools should be installable with a single copy step. Go makes that trivial.
Semantic versioning is an operational practice. It’s not just a convention for library authors — it’s a contract that tooling can enforce.
Go’s design aligns with platform needs. Explicit errors, type safety, and static binaries all reduce surprises in production.

Bringing it home

This isn’t a “Perl vs. Go” story. It’s a story about deliberate simplification, taking a working Perl script and recasting it in Go. The aim is to see how the language’s choices shape a solution to the same problem.

The result is homebrew-semver-guard v0.1.0, a small but sturdy tool. It’s not feature-finished, but it’s production-ready in the ways that matter.

Next up: I’m considering more Go tools, maybe even Kubernetes for services on my home server. This port was a practice, an artifact demonstrating platform engineering in action.

Verifying my cryptographic key for Keyoxide

Mark Gardner — Fri, 18 Nov 2022 22:54:38 +0000

[Verifying my cryptographic key: openpgp4fpr:4dadfefe8e4ad0f92bcbbb8142f3ca898cb08a45]

Perl lightning talk: ”Don’t Fear map and grep”

Mark Gardner — Fri, 24 Jun 2022 21:25:44 +0000

This week’s Perl and Raku Conference 2022 in Houston was packed with great presentations, and I humbly added to them with a five-ish minute lightning talk on two of Perl’s more misunderstood functions: map and grep.

(Sorry about the ”um”s and ”ah”s…)

Don’t Fear map & grep: List processing for fun and profit | PPTX

The document discusses the use of 'map' and 'grep' in Perl for processing lists and arrays, highlighting their ability to simplify code and improve readability compared to traditional loops. It provides examples of how to transform and filter lists using these functions, emphasizing the importance of avoiding side effects. The author also suggests refactoring code for better clarity and efficiency when dealing with list processing. - Download as a PPTX, PDF or view online for free

slideshare.net

I’ve written much about list processing in Perl, and this talk was based on the following blog posts:

Overall I loved attending the conference, and it really invigorated my participation in the Perl community. Stay tuned as I resume regular posting!

Cover image: "shopping list" by Ex-Smith is licensed under CC BY-SA 2.0.

How much is that BLÅHAJ in the (terminal) window?

Mark Gardner — Tue, 12 Apr 2022 14:00:00 +0000

IKEA’s toy BLÅHAJ shark has become a beloved Internet icon over the past several years. I thought it might be cute to write a little Perl to get info about it and even display a cuddly picture right in the terminal where I’m running the code. Maybe this will give you some ideas for your own quick web clients. Of course, you could accomplish all of these things using a pipeline of individual command-line utilities like curl, jq, and GNU coreutils’ base64. These examples focus on Perl as the glue, though.

Warning: dodgy API ahead

I haven’t found a publicly-documented and ‑supported official API for querying IKEA product information but others have deconstructed the company’s web site AJAX requests so we can use that instead. The alternative would be to scrape the IKEA web site directly which, although possible, would be more tedious and prone to failure should their design change. An unofficial API is also unreliable but the simpler client code is easier to change should any errors surface.

Enter the Mojolicious

My original goal was to do this in a single line issued to the perl command, and luckily the Mojolicious framework’s ojo module is tailor-made for such things. By adding a -Mojo switch to the perl command, you get over a dozen quick single-character functions for spinning up a quick web application or, in our case, making and interpreting web requests without a lot of ceremony. Here’s the start of my one-line request to the IKEA API for information on their BLÅHAJ product, using ojo’s g function to perform an HTTP GET and displaying the JSON from the response body to the terminal.

perl -Mojo -E 'say g("https://sik.search.blue.cdtapps.com/us/en/search-result-page",
  form => {types => "PRODUCT", q => "BLÅHAJ"})->body'

This currently returns over 2,400 lines of data, so after reading it over I’ll convert the response body JSON to a Perl data structure and dump only the main product information using ojo’s r function:

perl -Mojo -E 'say r g("https://sik.search.blue.cdtapps.com/us/en/search-result-page",
  form => {types => "PRODUCT", q => "BLÅHAJ"})
  ->json->{searchResultPage}{products}{main}{items}[0]{product}'

{
  "availability" => [],
  "breathTaking" => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),
  "colors" => [
    {
      "hex" => "0058a3",
      "id" => 10007,
      "name" => "blue"
    },
    {
      "hex" => "ffffff",
      "id" => 10156,
      "name" => "white"
    }
  ],
  "contextualImageUrl" => "https://www.ikea.com/us/en/images/products/blahaj-soft-toy-shark__0877371_pe633608_s5.jpg",
  "currencyCode" => "USD",
  "discount" => "",
  "features" => [],
  "gprDescription" => {
    "numberOfVariants" => 0,
    "variants" => []
  },
  "id" => 90373590,
  "itemMeasureReferenceText" => "39 \x{bc} \"",
  "itemNo" => 90373590,
  "itemNoGlobal" => 30373588,
  "itemType" => "ART",
  "lastChance" => $VAR1->{"breathTaking"},
  "mainImageAlt" => "BL\x{c5}HAJ Soft toy, shark, 39 \x{bc} \"",
  "mainImageUrl" => "https://www.ikea.com/us/en/images/products/blahaj-soft-toy-shark__0710175_pe727378_s5.jpg",
  "name" => "BL\x{c5}HAJ",
  "onlineSellable" => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),
  "pipUrl" => "https://www.ikea.com/us/en/p/blahaj-soft-toy-shark-90373590/",
  "price" => {
    "decimals" => 99,
    "isRegularCurrency" => $VAR1->{"breathTaking"},
    "prefix" => "\$",
    "separator" => ".",
    "suffix" => "",
    "wholeNumber" => 19
  },
  "priceNumeral" => "19.99",
  "quickFacts" => [],
  "tag" => "NONE",
  "typeName" => "Soft toy"
}

If I just want the price I can do:

$ perl -Mojo -E 'say g("https://sik.search.blue.cdtapps.com/us/en/search-result-page",
  form => {types => "PRODUCT", q => "BLÅHAJ"})->json
  ->{searchResultPage}{products}{main}{items}[0]{product}
  ->@{qw(currencyCode priceNumeral)}'
USD19.99

That ->@{qw(currencyCode priceNumeral)} towards the end uses the postfix reference slicing syntax introduced experimentally in Perl v5.20 and made official in v5.24. If you’re using an older perl, you’d say:

$ perl -Mojo -E 'say @{g("https://sik.search.blue.cdtapps.com/us/en/search-result-page",
  form => {types => "PRODUCT", q => "BLÅHAJ"})->json
  ->{searchResultPage}{products}{main}{items}[0]{product}}{qw(currencyCode priceNumeral)}'
USD19.99

I prefer the former, though, because it’s easier to read left-to-right.

But I’m not in the United States! Where’s my native currency?

You can either replace the ”us/en” in the URL above or use the core I18N::LangTags::Detect module added in Perl v5.8.5 if you’re really determined to be portable across different users’ locales. This is really stretching the definition of ”one-liner,” though.

$ LANG=de_DE.UTF-8 perl -Mojo -MI18N::LangTags::Detect \
  -E 'my @lang = (split /-/, I18N::LangTags::Detect::detect)[1,0];
  say g("https://sik.search.blue.cdtapps.com/"
  . join("/", @lang == 2 ? @lang : ("us", "en"))
  . "/search-result-page",
  form => {types => "PRODUCT", q => "BLÅHAJ"})->json
  ->{searchResultPage}{products}{main}{items}[0]{product}
  ->@{qw(currencyCode priceNumeral)}'
EUR27.99

Window dressing

It’s hard to envision cuddling a number, but luckily the product information returned above links to a JPEG file in the mainImageUrl key. My favorite terminal app iTerm2 can display images inline from either a file or Base64 encoded data, so adding an extra HTTP request and encoding from the core MIME::Base64 module yields:

perl -Mojo -MMIME::Base64 \
  -E 'say "\c[]1337;File=inline=1;width=100%:",
  encode_base64(g(g("https://sik.search.blue.cdtapps.com/us/en/search-result-page",
  form => {types => "PRODUCT", q => "BLÅHAJ"})->json
  ->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl})->body),
  "\cG"'

(You could just send the image URL to iTerm2’s bundled imgcat utility, but where’s the fun in that?)

imgcat --url `perl -Mojo \
  -E 'print g("https://sik.search.blue.cdtapps.com/us/en/search-result-page",
  form => {types => "PRODUCT", q => "BLÅHAJ"})->json
  ->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl}'`

But I don’t have iTerm2 or a Mac!

I got you. At the expense of a number of other dependencies, here’s a version that will work on any terminal that supports 256-color mode with ANSI codes using Image::Term256Color from CPAN and a Unicode font with block characters. I’ll also use Term::ReadKey to size the image for the width of your window. (Again, this stretches the definition of “one-liner.”)

perl -Mojo -MImage::Term256Color -MTerm::ReadKey \
  -E 'say for Image::Term256Color::convert(g(g("https://sik.search.blue.cdtapps.com/us/en/search-result-page",
  form => {types => "PRODUCT", q => "BLÅHAJ"})->json
  ->{searchResultPage}{products}{main}{items}[0]{product}{mainImageUrl})->body,
  {scale_x => (GetTerminalSize)[0], utf8 => 1})'

I hate Mojolicious! Can’t you just use core modules?

Fine. Here’s retrieving the product price using HTTP::Tiny and the pure-Perl JSON parser JSON::PP, which were added to core in version 5.14.

$ perl -MHTTP::Tiny -MJSON::PP \
  -E 'say @{decode_json(HTTP::Tiny->new->get("https://sik.search.blue.cdtapps.com/us/en/search-result-page?types=PRODUCT&q=BLÅHAJ")
  ->{content})
  ->{searchResultPage}{products}{main}{items}[0]{product}}{qw(currencyCode priceNumeral)}'
USD19.99

Fetching and displaying a picture of the huggable shark using MIME::Base64 or Image::Term256Color as above is left as an exercise to the reader.

Perl list processing is for hashes, too

Mark Gardner — Thu, 10 Feb 2022 15:00:00 +0000

This month I started a new job at Alert Logic, a cybersecurity provider with Perl (among many other things) at its beating heart. I’ve been learning a lot, and part of the process has been understanding the APIs in the code base. To that end, I’ve been writing small test scripts to tease apart data structures, using Perl array-processing, list-processing, and hash- (i.e., associative array)-processing functions.

I’ve covered map, grep, and friends a couple times before. Most recently, I described using List::Util’s any function to check if a condition is true for any item in a list. In the simplest case, you can use it to check to see if a given value is in the list at all:

use feature 'say';
use List::Util 'any';
my @colors =
  qw(red orange yellow green blue indigo violet);
say 'matched' if any { /^red$/ } @colors;

However, if you’re going to be doing this a lot with arbitrary strings, Perl FAQ section 4 advises turning the array into the keys of a hash and then checking for membership there. For example, here’s a simple script to check if the colors input (either from the keyboard or from files passed as arguments) are in the rainbow:

#!/usr/bin/env perl

use v5.22; # introduced <<>> for safe opening of arguments
use warnings;

my %in_colors = map {$_ => 1}
  qw(red orange yellow green blue indigo violet);

while (<<>>) {
  chomp;
  say "$_ is in the rainbow" if $in_colors{$_};
}

List::Util has a bunch of functions for processing lists of pairs that I’ve found useful when pawing through hashes. pairgrep, for example, acts just like grep but instead assigns $a and $b to each key and value passed in and returns the resulting pairs that match. I’ve used it as a quick way to search for hash entries matching certain value conditions:

use List::Util 'pairgrep';
my %numbers = (zero => 0, one => 1, two => 2, three => 3);
my %odds = pairgrep {$b % 2} %numbers;

Sure, you could do this by invoking a mix of plain grep, keys, and a hash slice, but it’s noisier and more repetitive:

use v5.20; # for key/value hash slice 
my %odds = %numbers{grep {$numbers{$_} % 2} keys %numbers};

pairgrep’s compiled C‑based XS code can also be faster, as evidenced by this Benchmark script that works through a hash made of the Unix words file (479,828 entries on my machine):

#!/usr/bin/env perl

use v5.20;
use warnings;
use List::Util 'pairgrep';
use Benchmark 'cmpthese';

my (%words, $count);
open my $fh, '<', '/usr/share/dict/words'
  or die "can't open words: $!";
while (<$fh>) {
  chomp;
  $words{$_} = $count++;
}
close $fh;

cmpthese(100, {
  grep => sub {
    my %odds = %words{grep {$words{$_} % 2} keys %words};
  },
  pairgrep => sub {
    my %odds = pairgrep {$b % 2} %words;
  },
} );

Benchmark output:

           Rate     grep pairgrep
grep     1.47/s       --     -20%
pairgrep 1.84/s      25%       --

In general, I urge you to work through the Perl documentation’s tutorials on references, lists of lists, the data structures cookbook, and the FAQs on array and hash manipulation. Then dip into the various list-processing modules (especially the included List::Util and CPAN’s List::SomeUtils) for ready-made functions for common operations. You’ll find a wealth of techniques for creating, managing, and processing the data structures that your programs need.

Get out early with Perl statement modifiers

Mark Gardner — Wed, 19 Jan 2022 15:00:00 +0000

When I first started writing Perl in my early 20’s, I tended to follow a lot of the structured programming conventions I had learned in school through Pascal, especially the notion that every function has a single point of exit. For example:

sub double_even_number {
    # not using signatures, this is mid-1990's code
    my $number = shift;

    if (not $number % 2) {
        $number *= 2;
    }

    return $number; 
}

This could get pretty convoluted, especially if I was doing something like validating multiple arguments. And at the time I didn’t yet grok how to handle exceptions with eval and die, so I’d end up with code like:

sub print_postal_address {
    # too many arguments, I know
    my ($name, $street1, $street2, $city, $state, $zip) = @_;
    # also this notion of addresses is naive and US-centric

    my $error;

    if (!$name) {
        $error = 'no name';
    }
    else {
        print "$name\n";

        if (!$street1) {
            $error = 'no street';
        }
        else {
            print "$street1\n";

            if ($street2) {
                print "$street2\n";
            }

            if (!$city) {
                $error = 'no city';
            }
            else {
                print "$city, ";

                if (!$state) {
                    $error = 'no state';
                }
                else {
                    print "$state ";

                    if (!$zip) {
                        $error = 'no ZIP code';
                    }
                    else {
                        print "$zip\n";
                    }
                }
            }
        }
    }

    return $error;
}

What a mess. Want to count all those braces to make sure they’re balanced? This is sometimes called the arrow anti-pattern, with the arrowhead(s) being the most nested statement. The default ProhibitDeepNests perlcritic policy is meant to keep you from doing that.

The way out (literally) is guard clauses: checking early if something is valid and bailing out quickly if not. The above example could be written:

sub print_postal_address {
    my ($name, $street1, $street2, $city, $state, $zip) = @_;

    if (!$name) {
        return 'no name';
    }
    if (!$street1) {
        return 'no street1';
    }
    if (!$city) {
        return 'no city';
    }
    if (!$state) {
        return 'no state';
    }
    if (!$zip) {
        return 'no zip';
    }

    print join "\n",
      $name,
      $street1,
      $street2 ? $street2 : (),
      "$city, $state $zip\n";

    return;
}

With Perl’s statement modifiers (sometimes called postfix controls) we can do even better:

    ...

    return 'no name'    if !$name;
    return 'no street1' if !$street1;
    return 'no city'    if !$city;
    return 'no state'   if !$state;
    return 'no zip'     if !$zip;

    ...

(Why if instead of unless? Because the latter can be confusing with double-negatives.)

Guard clauses aren’t limited to the beginnings of functions or even exiting functions entirely. Often you’ll want to skip or even exit early conditions in a loop, like this example that processes files from standard input or the command line:

while (<>) {
    next if /^SKIP THIS LINE: /;
    last if /^END THINGS HERE$/;

    ...
}

Of course, if you are validating function arguments, you should consider using actual subroutine signatures if you have a Perl newer than v5.20 (released in 2014), or one of the other type validation solutions if not. Today I would write that postal function like this, using Type::Params for validation and named arguments:

use feature qw(say state); 
use Types::Standard 'Str';
use Type::Params 'compile_named';

sub print_postal_address {
    state $check = compile_named(
        name    => Str,
        street1 => Str,
        street2 => Str, {optional => 1},
        city    => Str,
        state   => Str,
        zip     => Str,
    );
    my $arg = $check->(@_);

    say join "\n",
      $arg->{name},
      $arg->{street1},
      $arg->{street2} ? $arg->{street2} : (),
      "$arg->{city}, $arg->{state} $arg->{zip}";

    return;
}

print_postal_address(
    name    => 'J. Random Hacker',
    street1 => '123 Any Street',
    city    => 'Somewhereville',
    state   => 'TX',
    zip     => 12345,
);

Note that was this part of a larger program, I’d wrap that print_postal_address call in a try block and catch exceptions such as those thrown by the code reference $check generated by compile_named. This highlights one concern of guard clauses and other “return early” patterns: depending on how much has already occurred in your program, you may have to perform some resource cleanup either in a catch block or something like Syntax::Keyword::Try’s finally block if you need to tidy up after both success and failure.

Highlighting members of the Perl family

Mark Gardner — Tue, 11 Jan 2022 15:00:00 +0000

This past year of blogging has introduced me to a wide variety of people in the Perl community. Some I’ve admired from afar for years due to their published work, and even more I’ve “met” interacting on social media and other forums. So this will be the first in an occasional series highlighting not just the code, but the people that make up the Perl family.

Paul “LeoNerd” Evans

I first came across Paul’s work during his series last year on writing a core Perl feature; he’s responsible for Perl v5.32’s isa operator and v5.34’s experimental try/catch exception handling syntax. I interviewed him about the latter for Perl.com in March 2021. He’s been active on CPAN for so much longer, though, and joined the Perl Steering Council in July. He’s also often a helpful voice on IRC.

Elliot Holden

Renowned author and trainer Randal L. “merlyn” Schwartz linked over the weekend in a private Facebook group to Elliot’s impassioned YouTube video about his day job as a Perl web application developer. Through his alter ego Urban Guitar Legend Elliot is also a passionate musician; besides gigging and recording he’s been posting videos for nine years. (I’m a bit envious since I took a break from music almost twenty years ago and haven’t managed to recapture it.) Elliot seems like the quintessential needs-to-get-shit-done developer, and Perl is perfect for that.

Gábor Szabó

Gábor is a polyglot (both in human and computer languages) trainer, consultant, and author, writing about programming and devops on his Code Maven and Perl Maven websites. He’s also the founder and co-editor of Perl Weekly and recipient of a Perl White Camel award in 2008 thanks to his organizational and support contributions. Last year he introduced me to the world of live pair programming, working on a web application using the Mojolicious framework.

If you’re on Twitter and looking to connect with other Perl developers, please consider participating in the Perl community I’ve set up there. Twitter Communities are topic-specific moderated discussion groups, unlike the freewheeling #hashtags system that can be diluted by spam or topics that share the same name. Unfortunately, they’re still read-only on the Twitter Android app, but you can participate fully on iOS/iPadOS and the website.

Perl warnings and the warn function

Mark Gardner — Tue, 04 Jan 2022 15:00:00 +0000

I mentioned in passing last week that the next major release of Perl, v5.36, is set to enable warnings by default for code that opts in to use v5.35; or above. Commemorating Perl’s 34th birthday the week before that, I noted that the warnings system has been getting ever finer-grained since its introduction in 2000. And fellow Perl blogger and CPAN author Tom Wyant has been cataloging his favorites over the past several months—the latest as of this writing was on the “ambiguous” category of warnings, and you can find links to previous entries in his series at the bottom of that post.

It occurred to me afterward that there may be some confusion between the warnings pragma and the related warn function for reporting arbitrary runtime errors. warn outputs its arguments to the standard error (STDERR) stream, or if it’s not given any then you get a string with any exception from $@ ($EVAL_ERROR under use English) followed by a tab and then “...caught at <file> line x.” If that’s empty too, a plain warn just says, “Warning: something's wrong at <file> line x.”, which isn’t exactly helpful, but then again you didn’t give it much to go on.

warn output doesn’t have to go to STDERR, and this is where the relation to the warnings pragma comes in because both are governed by the __WARN__ signal handler in the %SIG hash. Normally, you might opt to only display runtime warnings if a debugging flag is set, like so:

#!/usr/bin/env perl

use strict;
use warnings;

my $DEBUG = 0;
$SIG{ __WARN__ } = sub { warn @_ if $DEBUG };
warn 'shhh'; # silenced

$DEBUG = 1;
warn 'hello warnings';

But if you set that signal handler in a BEGIN block, it catches compile-time warnings too, in which case flipping a flag after the fact has no effect—the compiler’s already run:

#!/usr/bin/env perl

use strict;
use warnings;

my $DEBUG = 0;
BEGIN { $SIG{ __WARN__ } = sub { warn @_ if $DEBUG } }
my $foo = 'hello';
my $foo = 'world'; # no warning issued here

$DEBUG = 1;
my $foo = 'howdy'; # still nothing

By the way, both __WARN__ and __DIE__ hooks are also used by the Carp module and its friends, so you can use the same technique with their enhanced output:

#!/usr/bin/env perl

use strict;
use warnings;
use Carp qw(carp cluck);

my $DEBUG = 0;
BEGIN { $SIG{ __WARN__ } = sub { warn @_ if $DEBUG } }
carp 'quiet fish';

$DEBUG = 1;
loud_chicken();

sub loud_chicken {
    cluck 'here comes a stack trace';
}

You could use these as stepping stones towards a debug log for larger applications, but at that point, I’d suggest looking into one of the logging modules on CPAN like Log::Log4perl (not to be confused with that lately-problematic Java library), Log::Dispatch (which can be wired into Log4perl), or something else to suit your needs.

Cover image: "Warning!" by John Goodridge is licensed under CC BY-NC-SA 2.0

Avoid Yoda conditions in Perl you should

Mark Gardner — Tue, 28 Dec 2021 15:00:00 +0000

I remember a brief time in the mid-2000s insisting on so-called “Yoda conditions” in my Perl. I would place constants to the left of equality comparisons. In case I accidentally typed a single = instead of ==, the compiler would catch it instead of blithely assigning a variable. E.g.:

if ( $foo == 42 ) { ... } # don’t do this
if ( 42 == $foo ) { ... } # do this
if ( $foo = 42  ) { ... } # to prevent this

And because a foolish consistency is the hobgoblin of little minds, I would even extend this to string and relational comparisons.

if ( 'bar' eq $foo ) { ... } # weirdo
if ( 42 > $foo )     { ... } # make it stop

It looks weird, and it turns out it’s unnecessary as long as you precede your code with use warnings;. Perl will then warn you: “Found = in conditional, should be ==“. (Sidenote: Perl v5.36, due in mid-2022, is slated to enable warnings by default if you do use v5.35; or above, in addition to the strictness that was enabled with use v5.11;. Yay for less boilerplate!)

If you want to fatally catch this and many other warnings, use the strictures module from CPAN in your code like this:

use strictures 2;

This will cause your code to throw an exception if it commits many categories of mistakes. If you’re running in a version control system’s working directory (specifically Git, Subversion, Mercurial, or Bazaar), the module also prevents you from using indirect object syntax, Perl 4‑style multidimensional arrays, and bareword filehandles.

Getting back to assignments vs. conditionals, there is one case where I’ve found it to be acceptable to use an assignment inside an if statement, and that’s when I need to use the result of a check inside the condition. For example:

if ( my $foo = some_truthy_function() ) {
    ... # do something further with $foo
}

This keeps the scope of some_truthy_function()‘s result inside the block so that I don’t pollute the outer scope with a temporary variable. Fortunately, Perl doesn’t warn on this syntax.

Cover image: Paul VanDerWerf from Brunswick, Maine, USA, CC BY 2.0, via Wikimedia Commons

34 at 34 for v5.34: Modern Perl features for Perl’s birthday

Mark Gardner — Tue, 21 Dec 2021 15:00:00 +0000

Friday, December 17, 2021, marked the thirty-fourth birthday of the Perl programming language, and coincidentally this year saw the release of version 5.34. There are plenty of Perl developers out there who haven’t kept up with recent (and not-so-recent) improvements to the language and its ecosystem, so I thought I might list a batch. (You may have seen some of these before in May’s post “Perl can do that now!”)

The `feature` pragma

Perl v5.10 was released in December 2007, and with it came feature, a way of enabling new syntax without breaking backward compatibility. You can enable individual features by name (e.g., use feature qw(say fc); for the say and fc keywords), or by using a feature bundle based on the Perl version that introduced them. For example, the following:

use feature ':5.34';

…gives you the equivalent of:

use feature qw(bareword_filehandles bitwise current_sub evalbytes fc indirect multidimensional postderef_qq say state switch unicode_eval unicode_strings);

Boy, that’s a mouthful. Feature bundles are good. The corresponding bundle also gets implicitly loaded if you specify a minimum required Perl version, e.g., with use v5.32;. If you use v5.12; or higher, strict mode is enabled for free. So just say:

use v5.34;

And lastly, one-liners can use the -E switch instead of -e to enable all features for that version of Perl, so you can say the following on the command line:

perl -E 'say "Hello world!"'

Instead of:

perl -e 'print "Hello world!\n"'

Which is great when you’re trying to save some typing.

The `experimental` pragma

Sometimes new Perl features need to be driven a couple of releases around the block before their behavior settles. Those experiments are documented in the perlexperiment page, and usually, you need both a use feature (see above) and no warnings statement to safely enable them. Or you can simply pass a list to use experimental of the features you want, e.g.:

use experimental qw(isa postderef signatures);

Ever-expanding `warnings` categories

March 2000 saw the release of Perl 5.6, and with it, the expansion of the -w command-line switch to a system of fine-grained controls for warning against “dubious constructs” that can be turned on and off depending on the lexical scope. What started as 26 main and 20 subcategories has expanded into 31 main and 43 subcategories, including warnings for the aforementioned experimental features.

As the relevant Perl::Critic policy says, “Using warnings, and paying attention to what they say, is probably the single most effective way to improve the quality of your code.” If you must violate warnings (perhaps because you’re rehabilitating some legacy code), you can isolate such violations to a small scope and individual categories. Check out the strictures module on CPAN if you’d like to go further and make a safe subset of these categories fatal during development.

Document other recently-introduced syntax with Syntax::Construct

Not every new bit of Perl syntax is enabled with a feature guard. For the rest, there’s E. Choroba’s Syntax::Construct module on CPAN. Rather than having to remember which version of Perl introduced what, Syntax::Construct lets you declare only what you use and provides a helpful error message if someone tries to run your code on an older unsupported version. Between it and the feature pragma, you can prevent many head-scratching moments and give your users a chance to either upgrade or workaround.

Make built-in functions throw exceptions with `autodie`

Many of Perl’s built-in functions only return false on failure, requiring the developer to check every time whether a file can be opened or a system command executed. The lexical autodie pragma replaces them with versions that raise an exception with an object that can be interrogated for further details. No matter how many functions or methods deep a problem occurs, you can choose to catch it and respond appropriately. This leads us to…

`try`/`catch` exception handling and Feature::Compat::Try

This year’s Perl v5.34 release introduced experimental try/catch syntax for exception handling that should look more familiar to users of other languages while handling the issues surrounding using block eval and testing of the special $@ variable. If you need to remain compatible with older versions of Perl (back to v5.14), just use the Feature::Compat::Try module from CPAN to automatically select either v5.34’s native try/catch or a subset of the functionality provided by Syntax::Keyword::Try.

Pluggable keywords

The abovementioned Syntax::Keyword::Try was made possible by the introduction of a pluggable keyword mechanism in 2010’s Perl v5.12. So was the Future::AsyncAwait asynchronous programming library and the Object::Pad testbed for new object-oriented Perl syntax. If you’re handy with C and Perl’s XS glue language, check out Paul “LeoNerd” Evans’ XS::Parse::Keyword module to get a leg up on developing your own syntax module.

Define `package`s with versions and blocks

Perl v5.12 also helped reduce clutter by enabling a package namespace declaration to also include a version number, instead of requiring a separate our $VERSION = ...; v5.14 further refined packages to be specified in code blocks, so a namespace declaration can be the same as a lexical scope. Putting the two together gives you:

package Local::NewHotness v1.2.3 {
    ...
}

Instead of:

{
    package Local::OldAndBusted;
    use version 0.77; our $VERSION = version->declare("v1.2.3");
    ...
}

I know which I’d rather do. (Though you may want to also use Syntax::Construct qw(package-version package-block); to help along with older installations as described above.)

The `//` defined-or operator

This is an easy win from Perl v5.10:

defined $foo ? $foo : $bar # replace this
$foo // $bar               # with this

And:

$foo = $bar unless defined $foo # replace this
$foo //= $bar                   # with this

Perfect for assigning defaults to variables.

`state` variables only initialize once

Speaking of variables, ever want one to keep its old value the next time a scope is entered, like in a sub? Declare it with state instead of my. Before Perl v5.10, you needed to use a closure instead.

Save some typing with `say`

Perl v5.10’s bumper crop of enhancements also included the say function, which handles the common use case of printing a string or list of strings with a newline. It’s less noise in your code and saves you four characters. What’s not to love?

Note unimplemented code with `...`

The ... ellipsis statement (colloquially “yada-yada”) gives you an easy placeholder for yet-to-be-implemented code. It parses OK but will throw an exception if executed. Hopefully, your test coverage (or at least static analysis) will catch it before your users do.

Loop and enumerate arrays with `each`, `keys`, and `values`

The each, keys, and values functions have always been able to operate on hashes. Perl v5.12 and above make them work on arrays, too. The latter two are mainly for consistency, but you can use each to iterate over an array’s indices and values at the same time:

while (my ($index, $value) = each @array) {
    ...
}

This can be problematic in non-trivial loops, but I’ve found it helpful in quick scripts and one-liners.

`delete local` hash (and array) entries

Ever needed to delete an entry from a hash (e.g, an environment variable from %ENV or a signal handler from %SIG) just inside a block? Perl v5.12 lets you do that with delete local.

Paired hash slices

Jumping forward to 2014’s Perl v5.20, the new %foo{'bar', 'baz'} syntax enables you to slice a subset of a hash with its keys and values intact. Very helpful for cherry-picking or aggregating many hashes into one. For example:

my %args = (
    verbose => 1,
    name    => 'Mark',
    extra   => 'pizza',
);
# don't frob the pizza
$my_object->frob( %args{ qw(verbose name) };

Paired array slices

Not to be left out, you can also slice arrays in the same way, in this case returning indices and values:

my @letters   = 'a' .. 'z';
my @subset_kv = %letters[16, 5, 18, 12];
# @subset_kv is now (16, 'p', 5, 'e', 18, 'r', 12, 'l')

`when` as a statement modifier

Starting in Perl v5.12, you can use the experimental switch feature’s when keyword as a postfix modifier. For example:

for ($foo) {
    $a =  1 when /^abc/;
    $a = 42 when /^dna/;
    ...
}

But I don’t recommend when, given, or given’s smartmatch operations as they were retconned as experiments in 2013’s Perl v5.18 and have remained so due to their tricky behavior. I wrote about some alternatives using stable syntax back in February.

Simple class inheritance with `use parent`

Sometimes in older object-oriented Perl code, you’ll see use base as a pragma to establish inheritance from another class. Older still is the direct manipulation of the package’s special @ISA array. In most cases, both should be avoided in favor of use parent, which was added to core in Perl v5.10.1.

Mind you, if you’re following the Perl object-oriented tutorial’s advice and have selected an OO system from CPAN, use its subclassing mechanism if it has one. Moose, Moo, and Class::Accessor’s “antlers” mode all provide an extends function; Object::Pad provides an :isa attribute on its class keyword.

Test for class membership with the `isa` operator

As an alternative to the isa() method provided to all Perl objects, Perl v5.32 introduced the experimental isa infix operator:

$my_object->isa('Local::MyClass')
# or
$my_object isa Local::MyClass

The latter can take either a bareword class name or string expression, but more importantly, it’s safer as it also returns false if the left argument is undefined or isn’t a blessed object reference. The older isa() method will throw an exception in the former case and might return true if called as a class method when $my_object is actually a string of a class name that’s the same as or inherits from isa()’s argument.

Lexical subroutines

Introduced in Perl v5.18 and de-experimentalized in 2017’s Perl v5.26, you can now precede sub declarations with my, state, or our. One use of the first two is truly private functions and methods, as described in this 2018 Dave Jacoby blog and as part of Neil Bowers’ 2014 survey of private function techniques.

Subroutine signatures

I’ve written and presented extensively about signatures and alternatives over the past year, so I won’t repeat that here. I’ll just add that the Perl 5 Porters development mailing list has been making a concerted effort over the past month to hash out the remaining issues towards rendering this feature non-experimental. The popular Mojolicious real-time web framework also provides a shortcut for enabling signatures and uses them extensively in examples.

Indented here-documents with `<<~`

Perl has had shell-style “here-document” syntax for embedding multi-line strings of quoted text for a long time. Starting with Perl v5.26, you can precede the delimiting string with a ~ character and Perl will both allow the ending delimiter to be indented as well as strip indentation from the embedded text. This allows for much more readable embedded code such as runs of HTML and SQL. For example:

if ($do_query) {
    my $rows_deleted = $dbh->do(<<~'END_SQL', undef, 42);
      DELETE FROM table
      WHERE status = ?
      END_SQL
    say "$rows_deleted rows were deleted."; 
}

Self-documenting named regular expression captures

Perl’s expressive regular expression matching and text-processing prowess are legendary, although overuse and poor use of readability enhancements often turn people away from them (and Perl in general). We often use regexps for extracting data from a matched pattern. For example:

if ( /Time: (..):(..):(..)/ ) { # parse out values
    say "$1 hours, $2 minutes, $3 seconds";
}

Named capture groups, introduced in Perl v5.10, make both the pattern more obvious and retrieval of its data less cryptic:

if ( /Time: (?<hours>..):(?<minutes>..):(?<seconds>..)/ ) {
    say "$+{hours} hours, $+{minutes} minutes, $+{seconds} seconds";
}

Set default regexp flags with the `re` pragma

Beginning with Perl v5.14, writing use re '/xms'; (or any combination of regular expression modifier flags) will turn on those flags until the end of that lexical scope, saving you the trouble of remembering them every time.

Non-destructive substitution with `s///r` and `tr///r`

The s/// substitution and tr/// transliteration operators typically change their input directly, often in conjunction with the =~ binding operator:

s/foo/bar/; # changes the first foo to bar in $_
$baz =~ s/foo/bar/; # the same but in $baz

But what if you want to leave the original untouched, such as when processing an array of strings with a map? With Perl v5.14 and above, add the /r flag, which makes the substitution on a copy and returns the result:

my @changed = map { s/foo/bar/r } @original;

Unicode case-folding with `fc` for better string comparisons

Unicode and character encoding in general are complicated beasts. Perl has handled Unicode since v5.6 and has kept pace with fixes and support for updated standards in the intervening decades. If you need to test if two strings are equal regardless of case, use the fc function introduced in Perl v5.16.

Safer processing of file arguments with `<<>>`

The <> null filehandle or “diamond operator” is often used in while loops to process input per line coming either from standard input (e.g., piped from another program) or from a list of files on the command line. Unfortunately, it uses a form of Perl’s open function that interprets special characters such as pipes (|) that would allow it to insecurely run external commands. Using the <<>> “double diamond” operator introduced in Perl v5.22 forces open to treat all command-line arguments as file names only. For older Perls, the perlop documentation recommends the ARGV::readonly CPAN module.

Safer loading of Perl libraries and modules from `@INC`

Perl v5.26 removed the ability for all programs to load modules by default from the current directory, closing a security vulnerability originally identified and fixed as CVE-2016–1238 in previous versions’ included scripts. If your code relied on this unsafe behavior, the v5.26 release notes include steps on how to adapt.

HTTP::Tiny simple HTTP/1.1 client included

To bootstrap access to CPAN on the web in the possible absence of external tools like curl or wget, Perl v5.14 began including the HTTP::Tiny module. You can also use it in your programs if you need a simple web client with no dependencies.

Test2: The next generation of Perl testing frameworks

Forked and refactored from the venerable Test::Builder (the basis for the Test::More library that many are familiar with), Test2 was included in the core module library beginning with Perl v5.26. I’ve experimented recently with using the Test2::Suite CPAN library instead of Test::More and it looks pretty good. I’m also intrigued by Test2::Harness’ support for threading, forking, and preloading modules to reduce test run times.

Task::Kensho: Where to start for recommended Perl modules

This last item may not be included when you install Perl, but it’s where I turn for a collection of well-regarded CPAN modules for accomplishing a wide variety of common tasks spanning from asynchronous programming to XML. Use it as a starting point or interactively select the mix of libraries appropriate to your project.

And there you have it: a selection of 34 features, enhancements, and improvements for the first 34 years of Perl. What’s your favorite? Did I miss anything? Let me know in the comments.

Cover image: "My birthday cake" by Wesley Bowler is licensed under CC BY-NC-SA 2.0

A brief note on Log4perl

Mark Gardner — Tue, 14 Dec 2021 15:00:00 +0000

The Java world had an… interesting weekend when security researchers revealed on December 9 a vulnerability in the popular Apache Log4j 2 software library for recording and debugging events. Systems as diverse as Amazon Web Services, Apple iCloud, and the Minecraft video game could be exploited to run arbitrary code on a server merely by sending a specially-crafted string of text. Information technology professionals have been scrambling ever since the initial disclosure to patch, upgrade, reconfigure, or otherwise protect affected servers. It’s bad, and past unpatched vulnerabilities like this have been responsible for the exposure of millions of people’s sensitive data.

Many Perl applications use the similarly-named and ‑designed Log::Log4perl library, and the good news is that as far as I can tell the latter doesn’t suffer from the type of vulnerability described above. This doesn’t mean poorly-written or ‑configured Perl-based systems are immune to all exploits, just this particular one. You should be safe to continue using Log4perl unless someone has deliberately configured it otherwise, and in fact, my work uses it extensively.

You might be surprised to read me suggesting a logging framework after writing multiple articles espousing the Perl step debugger as an alternative. Log4perl developer Mike Schilli’s 2002 introduction to the package for Perl.com came down on the opposite side of the argument. It can seem like one of those programmer religious issues like tabs vs. spaces, vim vs. Emacs, or Linux vs. Windows. (For the record, the correct answers are spaces, BBEdit, and macOS. 😉)

But in this case, you can and should have the best of both worlds—logging at different levels to appropriate destinations while still dropping into the interactive debugger when you need to do something trickier like examine program state or tweak a data structure on the fly. I use both techniques and only emphasize the advocacy of step debugging because it’s understood less.

Sweeter Perl exception classes

Mark Gardner — Tue, 07 Dec 2021 15:00:00 +0000

I mentioned at the Ephemeral Miniconf last month that as soon as I write about one Perl module (or five), someone inevitably brings up another (or seven) I’ve missed. And of course, it happened again last week: no sooner had I written in passing that I was using Exception::Class than the denizens of the Libera Chat IRC #perl channel insisted I should use Throwable instead for defining my exceptions. (I’ve already blogged about various ways of catching exceptions.)

Why Throwable? Aside from Exception::Class’s author recommending it over his own work due to a “nicer, more modern interface,” Throwable is a Moo role, so it’s composable into classes along with other roles instead of mucking about with multiple inheritance. This means that if your exceptions need to do something reusable in your application like logging, you can also consume a role that does that and not have so much duplicate code. (No, I’m not going to pick a favorite logging module; I’ll probably get that wrong too.)

However, since Throwable is a role instead of a class, I would have to define several additional packages in my tiny modulino script from last week, one for each exception class I want. The beauty of Exception::Class is its simple declarative nature: just use it and pass a list of desired class names along with options for attributes and whatnot. What’s needed for simple use cases like mine is a declarative syntax for defining several exception classes without the noise of multiple packages.

Enter Throwable::SugarFactory, a module that enables you to do just that by adding an exception function for declaring exception classes. (There’s also the similarly-named Throwable::Factory; see the above discussion about never being able to cover everybody’s favorites.) The exception function takes three arguments: the name of the desired exception class as a string, a description, and an optional list of instructions Moo uses to build the class. It might look something like this:

package Local::My::Exceptions;
use Throwable::SugarFactory;

exception GenericError => 'something bad happened';
exception DetailedError => 'something specific happened' =>
  ( has => [message => ( is => 'ro' )] );

1;

Throwable::SugarFactory takes care of creating constructor functions in Perl-style snake_case as well as functions for detecting what kind of exception is being caught, so you can use your new exception library like this:

#!/usr/bin/env perl

use experimental qw(isa);
use Feature::Compat::Try;
use JSON::MaybeXS;
use Local::My::Exceptions;

try {
    die generic_error();
}
catch ($e) {
    warn 'whoops!';
}

try {
    die detailed_error( message => 'you got me' );
}
catch ($e) {
    die encode_json( $e->to_hash )
      if $e isa DetailedError and defined $e->message;
    $e->throw if $e->does('Throwable');
    die $e;
}

The above also demonstrates a couple of other Throwable::SugarFactory features. First, you get a to_hash method that returns a hash reference of all exception data, suitable for serializing to JSON. Second, you get all of Throwable’s methods, including throw for re-throwing exceptions.

So where does this leave last week’s FOAAS.com modulino client demonstration of object mocking tests? With a little bit of rewriting to define and then use our sweeter exception library, it looks like this. You can review for a description of the rest of its workings.

#!/usr/bin/env perl

package Local::CallFOAAS::Exceptions;
use Throwable::SugarFactory;

BEGIN {
    exception NoMethodError =>
      'no matching WebService::FOAAS method' =>
      ( has => [ method => ( is => 'ro' ) ] );
    exception ServiceError =>
      'error from WebService::FOAAS' =>
      ( has => [ message => ( is => 'ro' ) ] );
}

package Local::CallFOAAS;  # this is a modulino
use Test2::V0;             # enables strict, warnings, utf8

# declare all the new stuff we're using
use feature qw(say state);
use experimental qw(isa postderef signatures);
use Feature::Compat::Try;
use Syntax::Construct qw(non-destructive-substitution);

use WebService::FOAAS ();
use Package::Stash;
BEGIN { Local::CallFOAAS::Exceptions->import() }

my $foaas = Package::Stash->new('WebService::FOAAS');

my $run_as =
    !!$ENV{CPANTEST}       ? 'test'
  : !defined scalar caller ? 'run'
  :                          undef;
__PACKAGE__->$run_as(@ARGV) if defined $run_as;

sub run ( $class, @args ) {
    try { say $class->call_method(@args) }
    catch ($e) {
        die 'No method ', $e->method, "\n"
          if $e isa NoMethodError;
        die 'Service error: ', $e->message, "\n"
          if $e isa ServiceError;
        die "$e\n";
    }
    return;
}

# Utilities

sub methods ($) {
    state @methods = sort map s/^foaas_(.+)/$1/r,
      grep /^foaas_/, $foaas->list_all_symbols('CODE');
    return @methods;
}

sub call_method ( $class, $method = '', @args ) {
    state %methods = map { $_ => 1 } $class->methods();
    die no_method_error( method => $method )
      unless $methods{$method};
    return do {
        try { $foaas->get_symbol("&$method")->(@args) }
        catch ($e) { die service_error( message => $e ) }
    };
}

# Testing

sub test ( $class, @ ) {
    state $stash = Package::Stash->new($class);
    state @tests = sort grep /^_test_/,
      $stash->list_all_symbols('CODE');

    for my $test (@tests) {
        subtest $test => sub {
            try { $class->$test() }
            catch ($e) { diag $e }
        };
    }
    done_testing();
    return;
}

sub _test_can ($class) {
    state @subs = qw(run call_method methods test);
    can_ok $class, \@subs, "can do: @subs";
    return;
}

sub _test_methods ($class) {
    my $mock = mock 'WebService::FOAAS' => ( track => 1 );

    for my $method ( $class->methods() ) {
        $mock->override( $method => 1 );

        ok lives { $class->call_method($method) },
          "$method lives";
        ok scalar $mock->sub_tracking->{$method}->@*,
          "$method called";
    }
    return;
}

sub _test_service_failure ($class) {
    my $mock = mock 'WebService::FOAAS';

    for my $method ( $class->methods() ) {
        $mock->override( $method => sub { die 'mocked' } );

        my $exception =
          dies { $class->call_method($method) };
        isa_ok $exception, [ServiceError],
          "$method throws ServiceError on failure";
        like $exception->message, qr/^mocked/,
          "correct error in $method exception";
    }
    return;
}

1;

[Updated, thanks to Dan Book, Karen Etheridge, and Bob Kleemann] The only goofy bit above is the need to put the exception calls in a BEGIN block and then explicitly call BEGIN { Local::CallFOAAS::Exceptions->import() }. Since the two packages are in the same file, I can’t do a use statement since the implied require would look for a corresponding file or entry in %INC. (You can get around this by messing with %INC directly or through a module like me::inlined that does that messing for you, but for a single-purpose modulino like this it’s fine.)

Do you like what you see? Leave a tip!