DEV Community: Toby Inkster

How can we make this Moose faster?

Toby Inkster — Sat, 17 Jan 2026 10:05:06 +0000

(I make no apologies for the ChatGPT images in my recent blog posts, by the way. No artists are missing out on being paid: I wasn’t going to hire an artist to illustrate these blog posts which will be read by like three people.)

A while back, I wrote MooseX::XSAccessor which you can add to Moose classes to inspect your attributes and try to replace the accessors with faster XS-based ones. Now I’ve done the same for constructors (new) and destructors (DESTROY) with MooseX::XSConstructor.

There are probably still bugs, but initial benchmarks look promising:

              Rate   Standard     XS
Standard    6392/s         --   -43%
XS         11228/s        76%     --

They’re mostly just a matter of importing the modules and forgetting about them. If the modules detect that your class is too complex for the XS modules to handle, they’ll sit back and let the regular Moose code work its magic. Give them a try. Let me know if they break things. They usually won’t.

Marlin Racing

Toby Inkster — Sun, 11 Jan 2026 15:40:29 +0000

When I first introduced Marlin, it seemed the only OO framework which could beat its constructor in speed was the one generated by the new Perl core class keyword. Which seems fair, as that’s implemented in C and is tightly integrated with the Perl interpreter. However, I’m pleased to say that Marlin’s constructors are now faster.

(Though also I forgot to include Mouse in previous benchmarks, so I’ve rectified that now.)

         Rate  Plain   Tiny    Moo  Moose   Core Marlin  Mouse
Plain  1357/s     --    -1%   -48%   -55%   -73%   -77%   -78%
Tiny   1374/s     1%     --   -48%   -54%   -72%   -77%   -78%
Moo    2617/s    93%    91%     --   -13%   -47%   -56%   -58%
Moose  3001/s   121%   118%    15%     --   -39%   -50%   -52%
Core   4943/s   264%   260%    89%    65%     --   -17%   -21%
Marlin 5976/s   340%   335%   128%    99%    21%     --    -4%
Mouse  6237/s   359%   354%   138%   108%    26%     4%     --

The main way I’ve squeezed out a bit of improved performance is by improving how Class::XSConstructor keeps its metadata.

Previously, if you called Local::Person->new(), the XS constructor would look up the list of supported attributes for the class in @Local::Person::__XSCON_HAS and loop through that array to initialize each attribute like "name", "age", etc. If the attribute had a type constraint, it would need to fetch the coderef to validate the value from $Local::Person::__XSCON_ISA{"name"}, and so on. All these involved looking things up in the class’s stash, which isn’t exactly slow when done via XS, but could be faster.

I’ve changed it so that the first time the constructor is called, the XS code pulls together all the data it needs into C structs.

typedef struct {
    char   *name;
    I32     flags;
    char   *init_arg;
    char  **aliases;
    I32     num_aliases;
    SV     *default_sv;
    SV     *trigger_sv;
    CV     *check_cv;
    CV     *coercion_cv;
} xscon_param_t;

typedef struct {
    char   *package;
    bool    is_placeholder;
    xscon_param_t *params;
    I32     num_params;
    CV    **build_methods;
    I32     num_build_methods;
    bool    strict_params;
    char  **allow;
    I32     num_allow;
} xscon_constructor_t;

Rather than having to deal with attribute names being Perl SVs, they’re just simple C strings (char*).

The flags field does a lot of heavy lifting. It is a bit field with booleans indicating whether an attribute is required or optional, whether it should be a weaken reference, and other features. A lot of common defaults (attributes which default to common values like undef, true, false, 0, 1, the empty string, an empty arrayref, or an empty hashref) and common type constraints (Str, Num, Int, ArrayRef, etc) are also encoded into the flags field, so the constructor can often skip even having to look at default_sv and check_cv.

At the same time, the number of features Class::XSConstructor supports has increased, so Marlin now never needs to fall back to generating Pure Perl constructors. (The code for generating Perl constructors has now been deleted!)

A second trick is one I learned from Mouse in how it implements its strict constructor check. As a reminder, a strict constructor check is like the ones implemented by MooseX::StrictConstructor, MooX::StrictConstructor, and MouseX::StrictConstructor, along these lines:

sub new {
  # Unpack @_
  my $class = shift;
  my %args  = ( @_ == 1 and ref($_[0]) eq 'HASH' ) ? %{+shift} : @_;

  # Create new object
  my $object = bless( {}, $class );

  # Initialize each attribute
  if ( exists $args{name} ) {
    $object->{name} = $args{name};
  }
  if ( exists $args{date} ) {
    $object->{date} = $args{date};
  }

  # Strict constructor check
  for my $key ( %args ) {
    die "Unrecognized key: $key" unless $key =~ /^(name|date)$/;
  }

  return $object;
}

Strict constructors are a really useful feature as a protection against mistyped attributes. But they do come with a speed penalty, which I guess is why Moose and Moo don’t have this feature built in. (Mouse does actually have the feature built in, but requires an extension (MouseX::StrictConstructor) to toggle it on.)

Mouse’s strict constructor check has virtually zero performance impact. I took a look at the source code to figure out how, and it is pretty smart. It just counts the number of arguments the constructor has used to initialize attributes, and only bothers with the strict constructor check if the total number of arguments is greater than that. Something like this:

sub new {
  # Unpack @_
  my $class = shift;
  my %args  = ( @_ == 1 and ref($_[0]) eq 'HASH' ) ? %{+shift} : @_;

  # Create new object
  my $object = bless( {}, $class );
  my $used_keys = 0;

  # Initialize each attribute
  if ( exists $args{name} ) {
    $object->{name} = $args{name};
    $used_keys++;
  }
  if ( exists $args{date} ) {
    $object->{date} = $args{date};
    $used_keys++;
  }

  # Strict constructor check
  if ( keys(%args) > $used_keys ) {
    for my $key ( %args ) {
      die "Unrecognized key: $key" unless $key =~ /^(name|date)$/;
    }
  }

  return $object;
}

Genius!

With these changes, Marlin is now significantly faster than the Perl core class keyword.

Mouse still has around 10% faster accessors than Marlin, which I think might be largely down to having an integrated type system allowing pure C function calls for type constraints instead of needing to use call_sv to call an XS or Perl type check function.

Marlin does however beat Mouse significantly (around 70% faster) when it comes to delegated methods. Things like:

use v5.36;

package API_Client {
  use Marlin
    -modifiers,
    _log => {
      isa          => 'ArrayRef[HashRef]',
      default      => [],
      handles_via  => 'Array',
      handles      => {
        add_to_log   => 'push',
        responses    => 'all',
      },
    },
    ua => {
      isa          => 'HTTP::Tiny',
      default      => sub { HTTP::Tiny->new },
      handles      => {
        http_get     => 'get',
        http_post    => 'post',
      },
    };

    around 'http_get', 'http_post' => sub ( $next, $self, @args ) {
      my $response = $self->$next( @args );
      $self->add_to_log( $response );
      return $response;
    };
}

my $client = API_Client->new;
$client->http_get( ... );
$client->http_get( ... );
$client->http_get( ... );
my @responses = $client->responses;

Marlin outperforms all other OO frameworks in this kind of method.

If you want a fast, concise OO framework, consider using Marlin.

Introducing Marlin

Toby Inkster — Thu, 18 Dec 2025 21:44:05 +0000

Does the Perl world need another object-oriented programming framework?

To be honest, probably not.

But here’s why you might want to give Marlin a try anyway.

Most of your constructors and accessors will be implemented in XS and be really, really fast.
If you accept a few basic principles like “attributes should usually be read-only”, it can be really, really concise to declare a class and its attributes.

An example

  use v5.20.0;
  use experimental qw(signatures);

  # Import useful constants, types, etc.
  use Marlin::Util -all, -lexical;
  use Types::Common -all, -lexical;

  package Person {
    use Marlin
      'given_name!'   => NonEmptyStr,
      'family_name!'  => NonEmptyStr,
      'name_style'    => { enum => [qw/western eastern/], default => 'western' },
      'full_name'     => { is => lazy, builder => true },
      'birth_date?';

    sub _build_full_name ( $self ) {
      return sprintf( '%s %s', uc($self->family_name), $self->given_name )
        if $self->name_style eq 'eastern';
      return sprintf( '%s %s', $self->given_name, $self->family_name );
    }
  }

  package Payable {
    use Marlin::Role
      -requires => [ 'bank\_details' ];

    sub make_payment ( $self ) {
      ...;
    }
  }

  package Employee {
    use Marlin
      -extends => [ 'Person' ],
      -with    => [ 'Payable' ],
      'bank_details!' => HashRef,
      'employee_id!'  => Int,
      'manager?'      => { isa => 'Employee' };
  }

  my $manager = Employee->new(
    given_name    => 'Simon',
    family_name   => 'Lee',
    name_style    => 'eastern',
    employee_id   => 1,
    bank_details  => {},
  );

  my $staff = Employee->new(
    given_name    => 'Lea',
    family_name   => 'Simons',
    employee_id   => 2,
    bank_details  => {},
    manager       => $manager,
  );

  printf(
    "%s's manager is %s.\n",
    $staff->full_name,
    $staff->manager->full_name,
  ) if $staff->has_manager;

Some things you might notice:

It supports most of the features of Moose… or most of the ones you actually use anyway.
Declaring an attribute is often as simple as listing it’s name on the use Marlin line.
It can be followed by some options, but if you’re happy with Marlin’s defaults (read-only attributes), it doesn’t need to be.
You can use the ! to quickly mark an attribute as required instead of the longer { required => true }.
You can use ? to request a predicate method instead of the longer { predicate => true }.

Benchmarks

My initial benchmarking shows that Marlin is fast.

Constructors

         Rate   Tiny  Plain    Moo  Moose Marlin   Core
Tiny   1317/s     --    -2%   -48%   -53%   -54%   -72%
Plain  1340/s     2%     --   -47%   -53%   -53%   -72%
Moo    2527/s    92%    89%     --   -11%   -12%   -47%
Moose  2828/s   115%   111%    12%     --    -2%   -40%
Marlin 2873/s   118%   114%    14%     2%     --   -39%
Core   4727/s   259%   253%    87%    67%    65%     --

Only the new Perl core class keyword generates a constructor faster than Marlin’s. And it is significantly faster; there’s no denying that. However, object construction is only part of what you are likely to need.

Accessors

          Rate   Tiny  Moose  Plain   Core    Moo Marlin
Tiny   17345/s     --    -1%    -3%    -7%   -36%   -45%
Moose  17602/s     1%     --    -2%    -6%   -35%   -44%
Plain  17893/s     3%     2%     --    -4%   -34%   -44%
Core   18732/s     8%     6%     5%     --   -31%   -41%
Moo    27226/s    57%    55%    52%    45%     --   -14%
Marlin 31688/s    83%    80%    77%    69%    16%     --

By accessors, I’m talking about not just standard getter and setters, but also predicate methods and clearers. Marlin and Moo both use Class::XSAccessor when possible, giving them a significant lead over the others. Marlin uses some sneaky tricks to squeeze out a little bit of extra performance by creating aliases for parent class methods directly in the child class symbol tables, allowing Perl to bypass a lot of the normal method resolution stuff.

I really expected class to do a lot better than it does. Its readers and writers are basically implemented in pure Perl currently, though I guess there’s scope to improve them in future releases.

Native Traits / Handles Via / Delegations

         Rate   Tiny   Core  Plain  Moose    Moo Marlin
Tiny    675/s     --   -56%   -57%   -59%   -61%   -61%
Core   1518/s   125%     --    -4%    -8%   -13%   -13%
Plain  1581/s   134%     4%     --    -4%    -9%   -10%
Moose  1642/s   143%     8%     4%     --    -5%    -6%
Moo    1736/s   157%    14%    10%     6%     --    -1%
Marlin 1752/s   160%    15%    11%     7%     1%     --

If you don’t know what I mean by native traits, it’s the ability to create small methods like this:

  sub add_language ( $self, $lang ) {
    push $self->languages->@*, $lang;
  }

As part of the attribute definition:

  use Marlin
    languages => {
      is           => 'ro',
      isa          => ArrayRef[Str],
      default      => [],
      handles_via  => 'Array',
      handles      => { add_language => 'push', count_languages => 'count' },
    };

There’s not an awful lot of difference between the performance of most of these, but Marlin slightly wins. Marlin and Moose are also the only frameworks that include this out of the box without needing extension modules.

By the way, that default => [] was not a typo. You can set an empty arrayref or empty hashref as a default, and Marlin will assume you meant something like default => sub { [] }, but it cleverly skips over needing to actually call the coderef (slow), instead creating a reference to a new empty array in XS (fast)!

Combined

         Rate   Tiny  Plain   Core  Moose    Moo Marlin
Tiny    545/s     --   -48%   -56%   -58%   -60%   -64%
Plain  1051/s    93%     --   -16%   -19%   -22%   -31%
Core   1249/s   129%    19%     --    -4%    -8%   -18%
Moose  1304/s   139%    24%     4%     --    -4%   -14%
Moo    1355/s   148%    29%     8%     4%     --   -11%
Marlin 1519/s   179%    45%    22%    17%    12%     --

A realistic bit of code that constructs some objects and calls a bunch of accessors and delegations on them. Marlin performs very well.

Lexical accessors and private attributes

Marlin has first class support for lexical methods!

  use v5.42.0;

  package Widget {
    use Marlin
      name        => { isa => Str },
      internal_id => { reader => 'my internal_id', storage => 'PRIVATE' };

    ...

    printf "%d: %s\n", $w->&internal_id, $w->name, 
  }

  # dies because internal_id is lexically scoped
  Widget->new->&internal_id;

Support for the ->& operator was added in Perl 5.42. On older Perls (from Perl 5.12 onwards), lexical methods are still supported but you need to use function call syntax (internal_id($w)).

The storage => "PRIVATE" hint tells Marlin to use inside-out storage for that attribute, meaning that trying to access the internal_id by poking into the object’s internals ($obj->{internal_id}) won’t work.

This gives you true private attributes.

On Perl 5.18 and above, you can of course declare lexical methods using the normal my sub foo syntax, so you have private attributes as well as private methods.

Constant attributes

  package Person {
    use Marlin
      name         => { isa => Str, required => true },
      species_name => { isa => Str, constant => "Homo sapiens" };
  }

Constant attributes are declared like regular attributes, but are always very read-only and illegal to pass to the constructor.

Like other attributes, they support delegations, provided the delegated method isn’t one which could change the value.

Perl version support

Although some of the lexical features need newer versions of Perl, Marlin runs on Perl versions as old as 5.8.8.

Future directions

Some ideas I’ve had:

If Moose is loaded, create meta object protocol stuff for Marlin classes and roles, like Moo does.

Type::Tiny 2.8.0 Released

Toby Inkster — Mon, 31 Mar 2025 16:26:15 +0000

What’s new?

The BoolLike type constraint accepts boolean.pm booleans.
Type::Params offers some improvements for DWIM named parameter processing.
More shortcuts are provided for exporting parameterized versions of type constraints.

Improvements to Type::Params

If your function or method takes named arguments, the list_to_named option allows these arguments to optionally be provided positionally as a shortcut:

  package My::Company;

  use v5.36;
  use Moo;
  use builtin qw( true false );
  use Types::Common qw( PositiveOrZeroNum InstanceOf );
  use Type::Params qw( signature_for );

  ...;

  signature_for pay_money => (
    method => true,
    named => [
      amount => PositiveOrZeroNum,
      employee => InstanceOf['Local::Person'],
    ],
    list_to_named => true,
  );

  sub pay_money ( $self, $arg ) {
    $self->payroll_account->withdraw( $arg->amount );
    $arg->employee->bank_account->deposit( $arg->amount );
    return $self;
  }

  ...;

  my $co = My::Company->new( ... );

  # Standard usage is named arguments:
  $co->pay_money( amount => 3000, employee => $alice );
  $co->pay_money( { amount => 3000, employee => $bob } );

  # Or provide them as positional arguments in the same order
  # they were declared in (amount then employee):
  $co->pay_money( 3000, $carol );

  # Or if the types are unambiguous, switch it up and provide
  # them in the wrong order instead. Still works!
  $co->pay_money( $dave, 3000 );

  # Or mix and match:
  $co->pay_money( $eve, amount => 3000 );
  $co->pay_money( $eve, { amount => 3000 } );
  $co->pay_money( 3000, employee => $eve );
  $co->pay_money( 3000, { employee => $eve } );

Exporting Parameterized Types

For certain parameterizable types, there are now shortcuts to export parameterized versions of them.

For example, supposing you need to deal with numeric arrayrefs quite a lot. That is, arrayrefs containing only numbers. Previously, you’d probably do something like this:

  use Types::Common qw( Num ArrayRef );

  ...;

  has favourite_numbers => ( is => 'ro', isa => ArrayRef[Num] );

  ...;

  if ( ArrayRef->of( Num )->check( \@my_array ) ) {
    ...;
  }

Now you can easily create a Nums type constraint and use it:

  use Types::Common qw( Num );
  use Types::Standard::ArrayRef Nums => { of => Num };

  ...;

  has favourite_numbers => ( is => 'ro', isa => Nums );

  ...;

  if ( is_Nums \@my_array ) {
    ...;
  }

Not all parameterizable types support this, but many of the common ones do.

Matching simply

Toby Inkster — Fri, 14 Jul 2023 20:42:30 +0000

A little over ten years ago, when Perl 5.18 was approaching its release date, I released match::simple. This was mostly in response to the smartmatch operator (~~) being deprecated, but also a solution to the incredibly arcane rules for how the smartmatch operator operated.

match::simple was intended to be... simpler. The operator looks a little funky because it uses some clever trickery to fake being an infix operator:

    use match::simple;

    if ( $this |M| $that ) {
        ...;
    }

For those who don't like the infix syntax, a regular function is also provided:

    use match::simple 'match';

    if ( match $this, $that ) {
        ...;
    }

Its behaviour was always determined by the operand on the right-hand side. Given:

    $a |M| $b

If $b is undef, $a only matches if it's also undef.
If $b is any other non-reference value, acts like the string equality (eq) operator.
If $b is a regexp, acts like the regexp binding (=~) operator.
If $b is a coderef, $a matches if $b->($a) returns true.
If $b is an object, $a matches if $b->MATCH($a) returns true or $a ~~ $b using ~~ operator overloading.
If $b is an arrayref, $a matches if $a |M| $_ for any element of @{$b}.
If $b is anything else, the operator croaks.

These seemed simple, sane, and predictable. Yes, some form of numeric matching would have been nice, but I ruled that out because of the ambiguity of numeric strings.

Plus because if you really need numeric matching, I knew you could wrap the number in a coderef:

    if ( $a |M| sub { $b == shift } ) {
        ...;
    }

Type::Tiny has always been compatible with match::simple. You can use:

    use match::simple;
    use Types::Standard -types;

    if ( $value |M| ArrayRef ) {
        ...;
    }
    elsif ( $value |M| HashRef ) {
        ...;
    }

Anyway, fast forward to now, and Perl 5.38 has now been released, which finally removes the long-deprecated smartmatch operator. Thankfully one of the new features it also provides allows for true user-defined infix operators!

Introducing Syntax::Operator::Matches!

    use Syntax::Operator::Matches;

    if ( $this matches $that ) {
        ...;
    }

This uses exactly the same logic as match::simple — logic that has been frozen since 2013, but finally has beautiful syntax.

You can do nice things like:

    use Syntax::Operator::Matches;
    use Types::Standard -types;

    if ( $value matches ArrayRef[Int] ) {
        ...;
    }
    elsif ( $value matches Str|Undef ) {
        ...;
    }

Some of it is just a wrapper around match::simple, but other parts happen purely in XS, so operate much faster.

If you're looking for something to replace smartmatch in your code, consider Syntax::Operator::Matches. If you need backwards compatibility, match::simple is still here and isn't going to disappear.

Increasing Perl’s Visibility, Redux

Toby Inkster — Wed, 05 Apr 2023 15:19:28 +0000

Quite a while ago, I blogged about how Perl projects should have websites to increase not only their visibility, but the visibility of Perl as a whole.

Perl has had the CPAN and awesome websites like MetaCPAN and its predecessor search.cpan.org for a long time, so unlike how things happen in other programming language ecosystems, many Perl projects have felt no need to start their own websites for documentation, package downloads, and community — all these things were already provided.

However, I do feel that this centralization keeps Perl content on the Internet very isolated and makes Perl less visible than other programming languages.

I’ve recently released Web::PerlDistSite to provide an easy way to generate a modern-looking website for your project based mainly on its existing pod. You can then publish your site on github.io or your own hosting.

I used it to create this website for Exporter-Tiny and also to revamp the existing website for Type-Tiny.

If you’ve got a CPAN distribution and want to create a website for it, Web::PerlDistSite could be a quick way to get started. The documentation is pretty limited right now, but you can open an issue if you need help.

Experiments in Overloading

Toby Inkster — Sun, 02 Apr 2023 12:03:44 +0000

Let’s play with overloading a little.

A simple class:

  package Local::Overloaded {
    use Moo;

    has number => ( is => 'ro' );

    use overload '0+' => sub {
      my $self = shift;
      return $self->number;
    };
  }

And let’s test it:

  use Test2::V0;
  my $obj = Local::Overloaded->new( number => 42 );
  is( 0+$obj, 42 );
  done_testing;

This test fails.

Why?

We tend to think of 0+ as the way to “cast” a Perl variable to a number, so much so that the overload pragma even calls the numeric overload “0+”. However it is of course actually an addition, and we haven’t overloaded the plus operator.

The simple solution is to just including the fallback => true option when overloading. This tells Perl to fill in as many missing overloaded operations it can based on the operations you’ve explicitly provided.

But I’m not interested in the simple solution. I’m interested in what we can do with the “0+” overload.

Let’s try rephrasing our test case:

  use Test2::V0;
  my $obj = Local::Overloaded->new( number => 42 );
  ok( $obj == 42 );
  done_testing;

This still fails. The equality operator isn’t overloaded either.

However, this one works:

  use Test2::V0;
  my $obj = Local::Overloaded->new( number => 42 );
  is( $obj, 42 );
  done_testing;

So how is Test2 comparing them?

The answer is that whenever the right hand side of an is comparison is a non-reference, Test2 compares them as strings. Specifically, it does this:

  "$left" eq "$right"

And for whatever reason, even though we didn’t set fallback => true, Perl will happily apply the numeric overload when an object is interpolated into a string and doesn’t have a string overload.

So let’s add some stringy overloading to our class:

  package Local::Overloaded {
    use Moo;
    use Lingua::EN::Numbers qw( num2en );

    has number => ( is => 'ro' );

    use overload (
      '0+' => sub {
        my $self = shift;
        return $self->number;
      },
      '""' => sub {
        my $self = shift;
        return num2en( $self->number );
      },
    );
}

Now our previously passing test fails:

  use Test2::V0;
  my $obj = Local::Overloaded->new( number => 42 );
  is( $obj, 42 );
  done_testing;

It is comparing “forty-two” and “42” as strings.

So this is where we stumble upon the safe way to cast to a number. And it’s still not 0+.

  use Test2::V0;
  my $obj = Local::Overloaded->new( number => 42 );
  is( sprintf( '%d', $obj ), 42 );
  is( sprintf( '%s', $obj ), 'forty-two' );
  done_testing;

Passing the object to sprintf( '%d' ) for integers or sprintf( '%f' ) will actually use the numeric overload without complaining about == and + not being overloaded. This seems the most reliable way to cast an object to a number if it doesn’t have overload fallbacks enabled, or you’re not sure if it does.

The lesson here is that overloading can be weird in Perl, but using fallback => true makes it a lot less weird.

The other lesson is to use sprintf( '%d' ) or sprintf( '%f' ) to cast to a number instead of 0+.

Introducing Exporter::Almighty

Toby Inkster — Sat, 25 Mar 2023 17:02:18 +0000

Consider a simple module like this:

  package MyApp::Util::Maths;

  use strict;
  use warnings;

  use constant PI => 3.14159265358979;
  use constant EULER => 2.71828182845905;

  use base 'Exporter';

  our @EXPORT_OK = qw( PI EULER add );
  our %EXPORT_TAGS = (
    arithmetic => [qw( add )],
    numbers => [qw( PI EULER )],
    all => @EXPORT_OK,
  );

  sub add {
    my ( $x, $y ) = @_;
    return $x + $y;
  }

  1;

You might use it like:

  use MyApp::Util::Maths qw( PI add );

  my $pi_plus_one = add( PI, 1 );

Exporter::Almighty is a module designed to reduce boilerplate in your utils-like modules, and increase their functionality.

The initial module can be rewritten as:

  package MyApp::Util::Maths;

  use Exporter::Almighty -setup => {
    const => {
      numbers => {
        PI => 3.14159265358979,
        EULER => 2.71828182845905,
      },
    },
    tag => {
      arithmetic => ['add'],
    },
  };

  sub add {
    my ( $x, $y ) = @_;
    return $x + $y;
  }

  1;

Exporter::Almighty sets up your exporting automatically (but using Exporter::Tiny instead of Exporter), and calls use strict and use warnings on your behalf.

Exporter::Almighty creates your constants for you, so you don’t need to duplicate your list of constants anywhere.

A bonus for your caller is that they can do:

  use MyApp::Util::Maths qw( $PI $EULER );

To import read-only $PI and $EULER variables instead of traditional constants.

Your caller can also import things lexically:

  my $pi_plus_one = do {
    use MyApp::Util::Maths -lexical, qw( add $PI );
    add( $PI, 1 );
  };

 # add() and $PI are not defined outside the above block

Your caller can also rename any imported functions:

  use MyApp::Util::Maths 'add' => { -as => 'sum_of' };

Exporter::Almighty has integrations with Type::Tiny making it easy to define and export Type::Tiny type constraints as part of your module. For example:

  package MyApp::Util::Maths;

  use Exporter::Almighty -setup => {
    const => { ... },
    tag => { ... },
    type => { 'Types::Standard' => ['Int', 'Num'] },
    class => ['Calc' => { class => 'MyApp::Calculator' }],
  };

  ...;

  1;

Now people can import the Int and Num type constraints from your module:

  use MyApp::Util::Maths qw( Int );

They can even import a is_Int function:

  use MyApp::Util::Maths qw( is_Int );

You’ve also defined a Calc class type constraint which can be used like this:

  has calculator => (
    is => 'ro',
    isa => Calc,
    default => sub { Calc->new },
  );

Exporter::Almighty makes defining enum-like data types easy:

  package My::Properties {
    use Exporter::Almighty -setup => {
      enum => { 'Status' => ['alive', 'dead', 'undead'] },
    };
  }

  package Story::Character {
    use Moo;
    use My::Properties -lexical, '+Status';
    use experimental 'signatures';

    has status => (
      is => 'ro',
      isa => Status,
      default => STATUS_ALIVE,
    );

    sub meet ( $self, $other ) {

      if ( $self->status eq STATUS_ALIVE
      and $other->status eq STATUS_UNDEAD ) {
        print "Help!\n";
      }

      return $self;
    }
  }

Next time you’re writing a module that needs to export things, consider Exporter::Almighty. It could make things very easy for you, while adding a bunch of useful features for your caller.

Prosody vs Synapse

Toby Inkster — Wed, 08 Mar 2023 22:21:05 +0000

I recently decided to experiment by setting up a Matrix account. And because I always dive head first into things, I of course set up my own Matrix server rather than making an account on a public one. At the same time, I noticed my Jabber/XMPP account had stopped working, so I decided to self-host an XMPP server too.

For Matrix, I used Synapse as that seems the most popular and well-supported implementation. It was mostly just a matter of installing the Ubuntu packages and starting the service, but there was a bit of manual work:

The default config file is set up to store data in SQLite, but using SQLite is advised against, so I needed to manually create a PostgreSQL database and username, and add those to the config file.
The default config is missing macaroon_secret_key and registration_shared_secret settings, which are kinda important.
Because I wanted the server to be hosted on a different VPS than my website, I needed to set up delegation and some of the documentation online on how to do that isn’t quite correct.

For XMPP, I used Prosody and the install went very smoothly. Again, just install the package and start the service. The only manual work was:

Needed to enter a host name in the example config file.
Ubuntu doesn’t give access to LetsEncrypt certificates to the prosody user, so I needed to fix permissions.

Overall, I would highly recommend Prosody as an XMPP server. Synapse is a good Matrix server too, but the Debian/Ubuntu packagers could take a look at Prosody for an example of a smooth install.

My Matrix account is at me colon toby.ink.

My XMPP account is me at toby.ink.

Using Type::Params Effectively

Toby Inkster — Fri, 17 Feb 2023 16:48:37 +0000

One of the modules bundled with Type::Tiny is Type::Params, a module
that allows you to validate subroutine signatures using type constraints.
It's one of the more popular parts of the suite.

This article provides a few hints for using it effectively.

When not to use Type::Params

There are a few places I occasionally see Type::Params being used where
it's really not the best fit.

You just want to know if a variable meets a type constraint

Let's say you have a variable $x and you want to know if it's
an integer. I have seen people do things like this:

    use v5.16;
    use Types::Standard qw( Int );
    use Type::Params qw( validate );

    if ( eval { validate [ $x ], Int } ) {
      say "x is an integer!";
    }

(The eval is used because validate will throw an exception, when what
we want is a boolean yes/no.)

This works, yes, but it can be made so much simpler!

The Int() function returns an Int type constraint object which we can
call methods on. In particular, the check method.

    use v5.16;
    use Types::Standard qw( Int );

    if ( Int->check( $x ) ) {
      say "x is an integer!";
    }

Because type libraries also export an is_X() function for each type
constraint, we can even go one step simpler:

    use v5.16;
    use Types::Standard qw( is_Int );

    if ( is_Int $x ) {
      say "x is an integer!";
    }

This is not just easier to read; it will also benchmark much, much faster.

You want to assert that a variable meets a type constraint

This is a similar situation, except that here we want an exception to be
thrown.

I see people do things like this:

    use v5.16;
    use Types::Standard qw( Int );
    use Type::Params qw( compile );

    state $check_int = compile( Int );
    $check_int->( $x );                 # assert $x is an integer

Again, we can call methods on the Int type constraint to achieve the
same result.

    use v5.16;
    use Types::Standard qw( Int );

    Int->assert_valid( $x );

Or we can use the assert_X() functions exported by type constraint
libraries:

    use v5.16;
    use Types::Standard qw( assert_Int );

    assert_Int $x;

Once again, this is not only much clearer, but will perform better in
benchmarks.

When to use Type::Params

Type::Params is intended for validating and unpacking an entire list
in one logical step. In particular this is useful for validating the
@_ array of subroutine arguments.

Consider we have a class something like this:

    use v5.16;

    package Point {
      use Moo;
      use Types::Common qw( Num );

      has x => ( is => 'rw', isa => Num, default => 0 );
      has y => ( is => 'rw', isa => Num, default => 0 );

      sub move_to {
        my ( $self, $new_x, $new_y, $reason ) = @_;
        $self->x( $new_x );
        $self->y( $new_y );
        say $reason;
        return $self;
      }
    }

This looks safe because the type constraints will ensure that x and y
will always be numeric.

However, what if somebody does $point->move_to( 1, '', 'whatever' )?
Setting y to the empty string will throw an exception, but x has
already been changed, so the object is left in an inconsistent state. It
hasn't moved to where we wanted it to move, but it hasn't stayed where
it was either!

So it's a good idea to validate all the parameters to move_to up front.
Here's how you could do it using Type::Params:

    use v5.16;

    package Point {
      use Moo;
      use Types::Common -types, -sigs;

      has x => ( is => 'rw', isa => Num, default => 0 );
      has y => ( is => 'rw', isa => Num, default => 0 );

      sub move_to {
        state $check = signature(
          method     => Object,
          positional => [ Num, Num, Str ],
        );
        my ( $self, $new_x, $new_y, $reason ) = &$check;
        $self->x( $new_x );
        $self->y( $new_y );
        say $reason;
        return $self;
      }
    }

The signature function compiles the function's signature into a coderef
called $check. This is stored in a state variable so that it is
compiled only once.

The method option indicates that this sub will be called as a method,
so will have an invocant (which must be an Object). The positional
option indicates that this sub expects two numbers followed by a string.

Calling &$check with no parentheses takes advantage of the Perl
feature where @_ is automatically passed forward. The $check
coderef will now validate @_ and return the validated values. If
validation fails, it will throw an exception.

Adding defaults

Type::Params also supports defaults:

    state $check = signature(
      method     => Object,
      positional => [
        Num,
        Num,
        Str, { default => 'cos I want to' },
      ],
    );

Now if the function is called without a $reason, a default reason
can be supplied. Simple non-reference defaults can be provided as above.
If you need to build a more complex default, you can use:

    default => sub { ... }

Switching to named parameters

Let's assume we would prefer to call the method like this:

    $point->move_to( x => 1, y => 2, reason => 'whatever' )

Type::Params handles named parameters easily.

    use v5.16;

    package Point {
      use Moo;
      use Types::Common -types, -sigs;

      has x => ( is => 'rw', isa => Num, default => 0 );
      has y => ( is => 'rw', isa => Num, default => 0 );

      sub move_to {
        state $check = signature(
          method => Object,
          named  => [
            x      => Num,
            y      => Num,
            reason => Str, { default => 'cos I want to' },
          ],
        );
        my ( $self, $arg ) = &$check;
        $self->x( $arg->x );
        $self->y( $arg->y );
        say $arg->reason;
        return $self;
      }
    }

The $check coderef will now return just the invocant $self
and an object $arg which allows access to the named arguments.

However, perhaps we want to accept named parameters without rewriting
the guts of move_to to deal with $arg. We still want our
$new_x, $new_y, and $reason variables. That is
also possible!

    use v5.16;

    package Point {
      use Moo;
      use Types::Common -types, -sigs;

      has x => ( is => 'rw', isa => Num, default => 0 );
      has y => ( is => 'rw', isa => Num, default => 0 );

      sub move_to {
        state $check = signature(
          method        => Object,
          named_to_list => 1,
          named         => [
            x      => Num,
            y      => Num,
            reason => Str, { default => 'cos I want to' },
          ],
        );
        my ( $self, $new_x, $new_y, $reason ) = &$check;
        $self->x( $new_x );
        $self->y( $new_y );
        say $reason;
        return $self;
      }
    }

The named_to_list option tells $check that it needs to accept
named parameters, but return them in a positional list.

Transitioning

Now we have rewritten move_to to accept named parameters, but perhaps
there are still places all over our codebase that call move_to with
positional parameters. How can we accept both?

Type::Params allows multiple signatures to be combined into one using the
multiple option.

    state $check = signature(
      method   => Object,
      multiple => [
        {
          named_to_list => 1,
          named         => [
            x      => Num,
            y      => Num,
            reason => Str, { default => 'cos I want to' },
          ],
        },
        {
          positional => [
            Num,
            Num,
            Str, { default => 'cos I want to' },
          ],
        },
      ],
    );
    my ( $self, $new_x, $new_y, $reason ) = &$check;

Now $check will try each signature in order and see which seems to
make sense. move_to will accept both named and positional arguments.

Extra flexibility

For some reason, we now also want to allow people to call:

    my @coordinates = ( $x, $y );
    $point->move_to( \@coordinates, $reason )

And still preserve the two existing ways to call the function.
Type::Params can still handle this situation. Firstly, we add
another option to the multiple list:

    state $check = signature(
      method   => Object,
      multiple => [
        {
          named_to_list => 1,
          named         => [
            x      => Num,
            y      => Num,
            reason => Str, { default => 'cos I want to' },
          ],
        },
        {
          positional => [
            Num,
            Num,
            Str, { default => 'cos I want to' },
          ],
        },
        {
          positional => [
            Tuple[ Num, Num ],
            Str, { default => 'cos I want to' },
          ],
          goto_next => sub { ... },
        },
      ],
    );
    my ( $self, $new_x, $new_y, $reason ) = &$check;

An issue with this new calling style is that instead of returning
four arguments (the invocant, two numbers, and the reason), it returns
three arguments (the invocant, an arrayref, and the reason). But we can
use goto_next to unpack the arrayref:

    goto_next => sub {
      my ( $self, $pair, $reason ) = @_;
      return ( $self, $pair->[0], $pair->[1], $reason );
    },

And now it all works!

Perl 5.20 subroutine signatures

Perl 5.20 introduced subroutine signatures as an experimental feature.
As of Perl 5.36, the feature is no longer experimental.

At first glance, this doesn't seem to fit well with Type::Params.
Consider:

    use v5.16;

    package Point {
      use Moo;
      use Types::Common -types, -sigs;

      has x => ( is => 'rw', isa => Num, default => 0 );
      has y => ( is => 'rw', isa => Num, default => 0 );

      sub move_to ( $self, $new_x, $new_y, $reason ) {
        state $check = signature(
          method     => Object,
          positional => [ Num, Num, Str ],
        );
        ( $self, $new_x, $new_y, $reason )
          = $check->( $self, $new_x, $new_y, $reason );
        $self->x( $new_x );
        $self->y( $new_y );
        say $reason;
        return $self;
      }
    }

It's... not neat. And we've lost the flexibility to support multiple
different calling styles.

However, Type::Params also provides a signature_for function which
inverts its usual declaration style, putting the signature check
outside the sub, and allowing you to use Perl subroutine signatures
for the sub itself.

Here's how we'd use it with named + positional calling style:

    use v5.16;

    package Point {
      use Moo;
      use Types::Common -types, -sigs;

      has x => ( is => 'rw', isa => Num, default => 0 );
      has y => ( is => 'rw', isa => Num, default => 0 );

      signature_for move_to => (
        method   => Object,
        multiple => [
          {
            named_to_list => 1,
            named         => [
              x      => Num,
              y      => Num,
              reason => Str, { default => 'cos I want to' },
            ],
          },
          {
            positional => [
              Num,
              Num,
              Str, { default => 'cos I want to' },
            ],
          },
        ],
      );

      sub move_to ( $self, $new_x, $new_y, $reason ) {
        $self->x( $new_x );
        $self->y( $new_y );
        say $reason;
        return $self;
      }
    }

The signature_for keyword operates at run-time, wrapping around
the move_to sub declared below it. The signature check takes care
of ensuring the parameters are always sent as a list, so the Perl
subroutine signature never needs to deal with named arguments.

Currently goto_next is not supported by signature_for, so the
third calling style we had before will not work as expected. However,
support is planned.

Hopefully this article has given you an idea of what Type::Params
is capable of, how you can get the most out of it, and when it's better
to use something else.

Setting up WordPress in 2023

Toby Inkster — Tue, 07 Feb 2023 10:57:26 +0000

Here is my recipe for quickly spinning up a WordPress site with:

Plugins and themes managed by PHP Composer, so they can be checked into version control.
Several of my favourite plugins installed, including FooGallery and Advanced Custom Fields.
A minimal theme combining Roots Sage 10 and Bootstrap 5.2 (at the time of writing this article 5.3 was an alpha release). and making use of the plugins in places.
Avoiding shiny WYSIWYG things like the WordPress block editor, page builder GUIs, etc.

Assumptions

You have SSH access to a Linux+Apache web host
You have a blank MySQL/MariaDB database
PHP and PHP Composer are installed and working on the web host, along with any optional components WordPress and Roots Sage need
You have a domain name (or subdomain) pointed at the host
Node JS, npm, and yarn are installed and working on the web host
git is installed and working on the web host

WordPress Setup

Let’s clone a skeleton site from Github. This includes our composer.json file along with a minimal index.php and an example wp-config.php file.

cd $HOME/sites/test.site.example
git clone https://github.com/tobyink/wp-base2023.git .
composer install
cp wp-config.php.example wp-config.php

Now we edit wp-config.php to add:

The domain name
Database connection info
Fresh salt from here

That should be enough for us to be able to visit https://$domain/wordpress/wp-admin/ in a browser and complete WordPress’s browser-based setup procedure.

Theme Setup

My minimal theme is called Thyme. It’s based on Roots Sage 10 and is pretty flexible in terms of appearance.

If you plan on using a different theme, you can skip straight to the conclusion.

To install Thyme:

cd $HOME/sites/test.site.example
cd wp-content/plugins/
git clone https://github.com/roots/acorn.git acorn
cd acorn
git checkout 2.x
composer install
cd ../../..
git clone https://github.com/tobyink/wp-thyme-theme.git thyme
cd wp-content/themes/
ln -s ../../thyme/ thyme
cd ../../thyme/
composer install
yarn install
yarn build
chmod -R ugo+rwX public/ resources/styles/common/_wp_theme.scss

Yes, that’s a fair few commands, but don’t let them scare you.

Once it’s installed, go to Plugins in the WordPress backend and make sure the following key plugins are activated (because the theme needs them!):

Acorn
Advanced Custom Fields
Classic Editor

Only after activating these plugins should you activate the theme.

Some optional features of the theme require these plugins to be activated:

Breadcrumb NavXT
FooGallery

After activating the theme, you can use the Theme Options and Sections forms in the backend to customize site appearance. After making any changes, all your CSS and Javascript files are automatically recompiled.

(There’s currently a bug in Thyme where recompilation is broken until you’ve gone to Theme Options and Sections and hit “Update” on both. This is because there’s a few nulls hanging around in the default values which breaks SASS syntax causing CSS compilation errors. Visiting both pages and setting some values flushes out the nulls.)

Conclusion

This is a pretty simple way to set up an easily-manageable no-nonsense WordPress blog.

Perl Testing in 2023

Toby Inkster — Tue, 24 Jan 2023 17:38:36 +0000

With my open source work, I’ve historically taken an approach which relies more on integration testing than unit testing, but with some of my newer projects, I’ve tried adopting principles from $paidwork and applying them to my free software.

This is a quick run-down of how I’m structuring my test suite in newer projects. It’s likely that many of my existing projects will never adopt this structure, but some may.

Out with the old, in with the new

First step is ditching Test::More, Test::Fatal, Test::Warnings, and other pre-Test2 testing libraries.

Test2::V0 provides a good base to work with, so we add that to our project’s requirements list straight away.

We’ll round that out with:

Test2::Require::AuthorTesting to skip certain tests when run on the end user’s machine, and require the AUTHOR_TESTING environment variable to run. This is useful for tests which are very slow or require a highly specific environment to run in.
Test2::Require::Module to skip certain tests when optional modules are unavailable.
Test2::Tools::Spec to better structure our unit tests.
Test2::Plugin::BailOnFail for when your tests simply cannot carry on. Use sparingly.

There are other Test2 modules that can be nice to have, but that’s a good starter set. The ones I list above are distributed alongside Test2::V0, so you get them “for free”. It is worth explicitly listing them in your project’s dependencies though, in case they are split into separate distributions in the future.

Test directory structure

Perl distributions typically keep their tests in a directory called t found inside the project root. We won’t change that. We will however create two subdirectories within it, t/unit and t/integration.

You may have other categories of tests which need their own subdirectories too, but these two will be sufficient for most projects.

If support modules are needed for testing, they can live in t/lib. If any data files are needed, put them in t/share.

Any especially important tests can included in the t directory itself for maximum visibility. I recommend giving them numerically-prefixed filenames for better control over the order they run in.

I will typically include two such tests:

t/00start.t which performs no real testing, but prints relevant information about the system it’s being run on, such as the version numbers of dependencies, the values of important environment variables, etc.
t/01basic.t which loads all of your modules, or at least the important ones, then passes without any real testing being done. The purpose of this is to quickly check for syntax errors so extreme that they prevent your code from even compiling. This is a good place for Test2::Plugin::BailOnFail.

Unit tests

For each module lib/Foo/Bar.pm there should be a corresponding unit test script t/unit/Foo/Bar.t.

The prelude of this file will look something like this:

  use Test2::V0 -target => 'Foo::Bar';
  use Test2::Tools::Spec;
  use Data::Dumper;

The footer will look exactly like this:

  done_testing;

And in between will be tests for each unit of code. A “unit” would typically be a function, a method, or an important package variable. Tests for a unit should avoid exercising much code outside their unit, and should especially avoid exercising code outside the target module.

A fairly extensive unit test for a method:

  describe "method `foo_bar`" => sub {

    my ( $foo, $bar, $expected_foo_bar );

    case 'both defined' => sub {
      $foo = 'Hello';
      $bar = 'world';
      $expected_foo_bar = 'Hello world';
    };

    case 'both defined, but foo is empty string' => sub {
      $foo = '';
      $bar = 'world';
      $expected_foo_bar = ' world';
    };

    case 'both defined, but bar is empty string' => sub {
      $foo = 'Hello';
      $bar = '';
      $expected_foo_bar = 'Hello ';
    };

    case 'both defined, but both are empty string' => sub {
      $foo = '';
      $bar = '';
      $expected_foo_bar = ' ';
    };

    case 'foo is undefined' => sub {
      $foo = undef;
      $bar = 'world';
      $expected_foo_bar = 'world';
    };

    case 'bar is undefined' => sub {
      $foo = 'Hello';
      $bar = undef;
      $expected_foo_bar = 'Hello';
    };

    case 'both are undefined' => sub {
      $foo = undef;
      $bar = undef;
      $expected_foo_bar = '';
    };

    tests 'it works' => sub {
      my ( $object, $got_foo_bar, $got_exception, $got_warnings );

      $got_exception = dies {
        $got_warnings = warns {
          $object = $CLASS->new( foo => $foo, bar => $bar );
          $got_foo_bar = $object->foo_bar;
        };
      };
      is( $got_exception, undef, 'no exception thrown' );
      is( $got_warnings, 0, 'no warnings generated' );
      is( $got_foo_bar, $expected_foo_bar, 'expected string returned' );
      is(
        $object,
        object {
          call foo => $foo;
          call bar => $bar;
        },
        "method call didn't alter the values of the attributes",
      ) or diag Dumper( $object );
    };
  };

The module itself as a whole can also be considered a “unit” so that very basic module-wide concerns can be tested there.

A fairly simple module-wide unit test:

  describe "class `$CLASS`" => sub {

    tests 'it inherits from Moo::Object' => sub {
      isa_ok( $CLASS, 'Moo::Object' );
    };

    tests 'it can be instantiated' => sub {
      can_ok( $CLASS, 'new' );
    };
  };

Integration tests

Integration tests can take a far more freeform approach. These tests should ensure that the system as a whole, or subsystems within it (which may involve multiple modules), work when used together.

A good place to start is to look at the SYNOPSIS sections of your documentation and test that they work as advertised.

Integration tests should use Test2::V0 but will often be small enough not to benefit much from Test2::Tools::Spec.

Running tests locally

The Test2 tool for running your test suite is yath, but old stalwart prove also works well. In your project roor directory, you should be able to run yath test at the command-line to run your entire test suite. prove -lr should also work.

If you need more verbose output to see exactly which tests are passing, then append -v to each command.

Continuous integration with GitHub Actions

I will normally use GitHub Actions to automatically run my test suite on each push, on every major version of Perl I support. One of the test runs will load Devel::Cover and use it to upload test coverage data to Codecov and Coveralls.

This gives me (almost) instant feedback on whether recent commits have broken things or reduced test coverage.

100% coverage on Coveralls is totally achieveable. 100% coverage on Codecov often takes a lot more work as it measures branch coverage instead of statement coverage. It’s certainly a good goal though.

Exactly how to set up continuous integration will depend a lot on your built tools, so I won’t go into specifics here.

Summary

I feel this setup provides a pretty good basis for test-driven Perl development in 2023.