DEV Community: LNATION

Enums for Perl: Adopting Devel::CallParser and Building Enum::Declare

LNATION — Tue, 14 Apr 2026 10:15:35 +0000

Adopting one from Zefram

Back in 2011, Andrew Main, known to the Perl community as Zefram, released Devel::CallParser. It was a quiet piece of infrastructure: a C API that let XS modules attach custom argument parsers to Perl subroutines. Where the core's PL_keyword_plugin API was awkward to work with directly, CallParser gave you a structured way to extend Perl's syntax from C.

Zefram maintained it through 2013, fixing compatibility issues with indirect and Sub::StrictDecl, working around padrange optimiser changes, and shipping version 0.002. Then silence. The module sat on CPAN, unmaintained, while Perl kept moving.

By the current year and perl version it was breaking. I personally could not install it locally on my hardened macOS runtimes, many reports of issues on threaded builds, shifted qerror internals, and many red reports on CPAN testers. I needed CallParser for Object::Proto::Sugar, so I adopted it and so far have shipped six dev releases (0.003_01 through 0.003_06) to try get it passing green again on all envs. Not glamorous work, but Zefram built something worth preserving.

(RIP Zefram... I didn't know them personally but the infrastructure they left behind is still making new things possible.)

The Idea

With CallParser working again, I decided to implement an idea I'd thought about for a long time: give Perl a proper enum keyword.

Not a hash. Not a bunch of use constant lines. Not a class/object pretending to be an enumeration. An actual keyword that declares an enum at compile time, generates real constants, and gives you a meta object for introspection.

Enum::Declare

Here's what it looks like:

use Enum::Declare;

enum Colour {
    Red,
    Green,
    Blue
}

say Red;    # 0
say Green;  # 1
say Blue;   # 2

That's it. enum is a real keyword, parsed at compile time by an XS callback wired through cv_set_call_parser. The constants are true constants, not subroutine calls, not tied variables. The compiler sees them.

Explicit Values

enum HttpStatus {
    OK        = 200,
    Created   = 201,
    NotFound  = 404,
    Internal  = 500
}

String Enums

enum LogLevel :Str {
    Debug,
    Info,
    Warn = "warning",
    Error,
    Fatal
}

say Debug;  # "debug"
say Warn;   # "warning"

Without an explicit value, :Str lowercases the constant name. With one, it uses what you gave it.

Bitflags

enum Perms :Flags {
    Read,
    Write,
    Execute
}

my $rw = Read | Write;
say "can read"  if $rw & Read;
say "can write" if $rw & Write;

:Flags assigns powers of two automatically. Combine them with bitwise operators as you'd expect.

Exporting

# In your module:
enum StatusCode :Export {
    OK      = 200,
    NotFound = 404
}

# Consumers get the constants automatically, or use tags:
use MyModule qw(:StatusCode);

Meta Objects

Every enum gets a meta object accessible by calling the enum name as a function:

my $meta = Colour();

say $meta->count;           # 3
say $meta->name(0);         # "Red"
say $meta->value('Blue');   # 2
say $meta->valid(1);        # true

my @pairs = $meta->pairs;  # (Red => 0, Green => 1, Blue => 2)

Exhaustive Matching

Colour()->match($val, {
    Red   => sub { "stop" },
    Green => sub { "go" },
    Blue  => sub { "sky" },
});

Miss a variant and it dies. Every key/branch must be covered.

How It Works

Under the hood, use Enum::Declare installs an XS stub named enum into the calling package, then attaches a custom parser via cv_set_call_parser. When Perl encounters enum during compilation, the parser callback fires and:

Reads the enum name (lex_read_ident)
Reads optional attributes - :Str, :Flags, :Export, :Type
Reads the { Name = Value, ... } variant block
Builds the constant subs and enum data structures
Installs the meta object
Optionally wires up @EXPORT / @EXPORT_OK

All of this happens at compile time. By the time Perl starts executing your code, the constants exist, the meta object is ready, and the exports are in place.

Enum::Declare::Common

Once the keyword worked, the obvious next step was a library of common enums:

use Enum::Declare::Common::HTTP qw(:StatusCode :Method);
use Enum::Declare::Common::Calendar qw(:Weekday :Month);
use Enum::Declare::Common::Color qw(:CSS);

say OK;        # 200
say GET;       # "get"
say Monday;    # 1
say January;   # 1

Enum::Declare::Common ships 20 submodules covering HTTP status codes and methods, ISO country and currency codes, MIME types, 148 named CSS hex colours, timezone offsets, Unix permissions, log levels, and more. All built on the same enum keyword, all with meta objects, all exportable.

Integration with Object::Proto

Every enum in the Common collection is declared with the :Type attribute:

enum StatusCode :Type :Export {
    OK      = 200,
    Created = 201,
    ...
}

This registers the enum as a type in Object::Proto at load time, so you can use enum names directly as slot types:

use Enum::Declare::Common::HTTP qw(:StatusCode :Method);
use Enum::Declare::Common::LogLevel qw(:Level);
use Object::Proto;

object 'APIRequest',
    'method:Method:required',
    'status:StatusCode',
    'log_level:Level:default(' . Info . ')',
;

my $req = new APIRequest method => GET;
$req->status(OK);       # valid
$req->status(9999);     # dies - not a valid StatusCode
$req->status(200);      # coercion - resolves to OK

The type checks and coercions run in C via object_register_type_xs_ex. No Perl callback overhead. A single pair of C functions serves every enum type, only the data pointer differs.

If you're writing Perl and you've been using hashes or use constant blocks to fake enums, give Enum::Declare a try. In my opinion it's enums the way they should have always worked.

As always if you have any questions just post below.

Learning XS - Custom Ops

LNATION — Sat, 11 Apr 2026 18:34:39 +0000

This is the next tutorial in my Learning XS series. This post explores one of Perl's most powerful optimisation techniques: Custom Ops. We'll build a real-world example, reimplementing one of my first XS cpan modules - a Shannon entropy calculator - and explain how to bypass Perl's subroutine call overhead entirely.

So firstly... What are Custom Ops?

When you call a Perl subroutine, Perl executes an entersub op that:

Sets up a new stack frame
Pushes arguments onto the stack
Magical things happens
Jumps to the subroutine code
More Magical things can happens
Cleans up and returns

For simple functions called millions of times (like mathematical operations), this overhead dominates execution time. Custom ops let you replace the entire entersub call with a single, specialised op - eliminating that overhead.

Next.. What is Shannon Entropy?

Shannon entropy measures the information content (or "randomness") of a string:

"aaaa" → 0 bits (no randomness, completely predictable)
"abcd" → 2 bits (4 equally likely symbols = log₂(4))
Random data → ~8 bits per byte (maximum entropy)

It's used in compression, cryptography, and password strength estimation. A perfect candidate for a small, focused XS module.

In Perl the algorithm for Shannon entropy can look something like:

sub entropy {
    my ($entropy, $len, $p, %t) = (0, length($_[0]));
    $t{$_}++ foreach split '', $_[0];
    foreach (values %t) {
        $p = $_/$len;
        $entropy -= $p * log $p;
    } 
    return $entropy / log 2;
}

Creating the Distribution

First we will need to create a new perl distribution. As we have done in the past we will use Module::Starter.

module-starter --module="Shannon::Entropy::XS" --author="Your Name" --email="your@email.com"

This will create a new directory called Shannon-Entropy-XS with the basic structure of a perl module.

Next we will need to update the Makefile.PL so that it knows we are using XS. One simple way to do this is to add XSMULTI => 1 to the WriteMakefile call. Which will tell MakeMaker to look for XS files in the lib/Shannon/Entropy/ directory.

use ExtUtils::MakeMaker;

my %WriteMakefileArgs = (
    NAME             => 'Shannon::Entropy::XS',
    AUTHOR           => q{Your Name <your@email.com>},
    VERSION_FROM     => 'lib/Shannon/Entropy/XS.pm',
    XSMULTI          => 1,
    LIBS             => [''],
    # ... rest of your config
);

WriteMakefile(%WriteMakefileArgs);

Next lets update the lib/Shannon/Entropy/XS.pm file to load our XS:

package Shannon::Entropy::XS;

use strict;
use warnings;
our $VERSION = '0.01';

require XSLoader;
XSLoader::load("Shannon::Entropy::XS", $VERSION);

use Exporter qw(import);
our @EXPORT_OK = qw(entropy);

1;

Now we can create the XS file. Create a new file called lib/Shannon/Entropy/XS.xs and add the basic structure:

#define PERL_NO_GET_CONTEXT
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"

MODULE = Shannon::Entropy::XS  PACKAGE = Shannon::Entropy::XS
PROTOTYPES: ENABLE

That defines the package. If we compile this now, the basic tests added by Module::Starter will pass. To test that:

perl Makefile.PL
make test

Adding the Entropy Function

With the basic structure working, lets write a test for our entropy function. Create or update t/01-test.t:

use Test::More;
use strict;
use warnings;
use Shannon::Entropy::XS qw/entropy/;

is(entropy(''), 0);
is(entropy('0'), 0);
is(sprintf('%.3f', entropy('1223334444')), 1.846);
is(entropy('0123456789abcdef'), 4),
is(sprintf('%.3f', entropy('abcdefghijklmnopqrst123456789!@£[]"')), 5.170),

done_testing();

Now we need to implement the entropy calculation. First, add the C helper functions to our XS file, before the MODULE line:

#define PERL_NO_GET_CONTEXT
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include <math.h>

static int makehist(const unsigned char *str, int *hist, int len) {
    int chars[256];
    int histlen = 0, i;
    for (i = 0; i < 256; i++) chars[i] = -1;
    for (i = 0; i < len; i++) {
        int c = (int)str[i];
        if (chars[c] == -1) {
            chars[c] = histlen;
            histlen++;
        }
        hist[chars[c]]++;
    }
    return histlen;
}

static double entropy(const char *str) {
    int len = strlen(str);
    int hist[256] = {0};
    int histlen, i;
    double out = 0.0;
    if (len == 0) return 0.0;
    histlen = makehist((const unsigned char *)str, hist, len);
    for (i = 0; i < histlen; i++) {
        double p = (double)hist[i] / len;
        out -= p * log2(p);
    }
    return out;
}

MODULE = Shannon::Entropy::XS  PACKAGE = Shannon::Entropy::XS
PROTOTYPES: ENABLE

double
entropy(string)
    SV *string
    CODE:
        STRLEN len;
        const char *str = SvPV(string, len);
        RETVAL = entropy(str);
    OUTPUT:
        RETVAL

So what exactly are we doing here? The makehist() function builds a histogram of character frequencies by iterating through the string and mapping each unique character to a slot in the hist array. The chars[256] lookup table tracks which slot each byte value maps to, with -1 meaning "not yet seen". When we encounter a new character, we assign it the next available slot and increment histlen.

The entropy() function then implements the Shannon entropy formula. For each unique character in the histogram, we calculate its probability (count / total length) and accumulate the entropy value. Empty strings return 0 immediately to avoid division by zero.

Finally, the XS wrapper extracts the string from the Perl SV using SvPV() (which gives us both the string pointer and its length) and calls our C function. The result is returned as a double.

Now run make test and our entropy tests should pass. But every call still goes through Perl's entersub op. Let's optimise that with custom ops.

Adding Custom Ops

Custom ops require Perl 5.14+. We'll add three pieces:

An XOP structure (for debugging/introspection)
A PP function (the actual operation)
A call checker (transforms the op tree at compile time)

First we need to add some back porting as part of the API we are about to use was introduced in 5.26, after the last import add:

#ifndef OpSIBLING
#  define OpSIBLING(o)              ((o)->op_sibling)
#endif
#ifndef OpMORESIB_set
#  define OpMORESIB_set(o, sib)     ((o)->op_sibling = (sib))
#endif
#ifndef OpLASTSIB_set
#  define OpLASTSIB_set(o, parent)  ((o)->op_sibling = NULL)
#endif

Next, we can add the XOP declaration after the last endif from above add:

#if PERL_VERSION >= 14
static XOP entropy_xop;
#endif

The XOP (eXtended OP) structure provides metadata - the op's name and description that appear in tools like B::Concise.

The PP Function

Every Perl op has a "pp" (push-pop) function that does the actual work. Add this after your entropy functions, inside a #if PERL_VERSION >= 14 block:

#if PERL_VERSION >= 14

static OP* pp_entropy(pTHX) {
    dSP;
    SV *sv = TOPs;
    STRLEN len;
    const char *str = SvPV(sv, len);
    double result = entropy(str);
    POPs;
    mPUSHn(result);
    RETURN;
}

#endif

A line by line explanation of the above code:

dSP - Declares the stack pointer local variable
SV *sv = TOPs - Gets the top of stack (our argument) without popping it
SvPV(sv, len) - Extracts the string and its length
entropy(str) - Calls our C function
POPs - Pops the argument off the stack
mPUSHn(result) - Pushes the result as a mortal (auto-freed) numeric value
RETURN - Returns control to the next op in the chain

The Call Checker

This is where the real magic happens. A call checker is invoked at compile time when Perl sees a call to our function. We can inspect the op tree and transform it. Add this after the pp function:

static OP* entropy_call_checker(pTHX_ OP *entersubop, GV *namegv, SV *ckobj) {
    OP *pushop, *argop, *nextop, *newop;
    PERL_UNUSED_ARG(namegv);
    PERL_UNUSED_ARG(ckobj);

    pushop = cLISTOPx(entersubop)->op_first;
    if (!pushop) return entersubop;

    /* Handle null op wrapper in modern Perl */
    if (pushop->op_type == OP_NULL && cLISTOPx(pushop)->op_first) {
        pushop = cLISTOPx(pushop)->op_first;
    }

    argop = OpSIBLING(pushop);
    if (!argop) return entersubop;

    nextop = OpSIBLING(argop);
    if (!nextop) return entersubop;

    if (OpSIBLING(nextop)) return entersubop;

    OpMORESIB_set(pushop, nextop);
    OpLASTSIB_set(argop, NULL);
    newop = newUNOP(OP_CUSTOM, 0, argop);
    newop->op_ppaddr = pp_entropy;
    op_free(entersubop);
    return newop;
}

#endif

So what is actually happening here? The call checker receives the entersub op that Perl normally uses to call subroutines. Inside an entersub, the child ops form a linked list: pushmark -> arg1 -> arg2 -> ... -> cv. We use cLISTOPx(entersubop)->op_first to get the first child. In modern Perl (5.22+), this is often wrapped in a null op, so we check for that and unwrap it to find the actual pushmark. Then we use OpSIBLING() to navigate to our argument.

Before we do anything destructive, we validate that the call matches our expected signature. The series of null checks and the OpSIBLING(nextop) test ensure we have exactly one argument. If someone called entropy($a, $b) with two arguments, or used the &entropy sigil syntax, we bail out early by returning entersubop unchanged - Perl falls back to the normal subroutine call mechanism.

Once we've validated the call, we surgically detach the argument op from the entersub's child list. OpMORESIB_set(pushop, nextop) rewires pushmark to skip over our argument, linking it directly to the CV. OpLASTSIB_set(argop, NULL) disconnects argop from its siblings entirely, leaving it as a standalone op.

With the argument detached, we create our custom op using newUNOP(OP_CUSTOM, 0, argop). This creates a unary op with our argument as its child. We then assign our pp function to execute when this op runs: newop->op_ppaddr = pp_entropy. Finally, we free the original entersub with op_free() since it's no longer needed, and return our shiny new custom op to take its place in the op tree.

Wiring It Up in BOOT

The BOOT section runs when the module loads. We register our custom op and install the call checker. Add this after your XS entropy function:

BOOT:
#if PERL_VERSION >= 14
{
    CV *entropy_cv;
    XopENTRY_set(&entropy_xop, xop_name, "entropy");
    XopENTRY_set(&entropy_xop, xop_desc, "Shannon entropy calculation");
    Perl_custom_op_register(aTHX_ pp_entropy, &entropy_xop);
    entropy_cv = get_cv("Shannon::Entropy::XS::entropy", 0);
    if (entropy_cv) {
        cv_set_call_checker(entropy_cv, entropy_call_checker, (SV *)entropy_cv);
    }
}
#endif

The first thing we do is set up the XOP metadata using XopENTRY_set(). This registers the name "entropy" and a description "Shannon entropy calculation" with our custom op. These strings appear in debugging tools like B::Concise and Devel::Peek, making it much easier to understand op trees when things do go wrong.

Next, Perl_custom_op_register() connects our pp function to the XOP structure. This tells Perl's introspection system that whenever it encounters an OP_CUSTOM with our pp_entropy address, it should use our XOP metadata for display purposes.

Finally, we wire up the call checker itself. get_cv() retrieves the CV (code value) for our Shannon::Entropy::XS::entropy function - this is the subroutine object that Perl created when it compiled the XSUB. We then call cv_set_call_checker() to install our call checker on that CV. From now on, whenever Perl compiles a call to entropy(), it will invoke our entropy_call_checker function, giving us the chance to transform the op tree.

Now run make clean && make test and all tests should still pass - but now with custom ops!

Verifying It Works

You can use B::Concise to see the optimised op tree. Create a simple test script:

use B::Concise;
use Shannon::Entropy::XS qw(entropy);

B::Concise::compile('-exec', sub { entropy("test") })->();

Without custom ops, you'd see entersub. With them, you'll see our entropy custom op directly in the tree - proof that we've eliminated the subroutine call overhead!

B::Concise::compile(CODE(0x12b02b4f0))
1  <;> nextstate(main 1816 test.pl:4) v
2  <$> const[PV "test"] s
3  <0> entropy K/1
4  <1> leavesub[1 ref] K/REFC,1

Okay at this point you may or may not be asking why you would replace 9 lines of code with 111 lines... well it would be for the performance improvement you've just achieved. Take this quick benchmark as justification:

Benchmark: Short string (5 chars)
----------------------------------------
                           Rate     Shannon::Entropy Shannon::Entropy::XS
Shannon::Entropy       908579/s                   --                 -95%
Shannon::Entropy::XS 18023002/s                1884%                   --

Benchmark: Medium string (43 chars)
----------------------------------------
                          Rate     Shannon::Entropy Shannon::Entropy::XS
Shannon::Entropy      165257/s                   --                 -98%
Shannon::Entropy::XS 8525479/s                5059%                   --

Extension: Alternative Approaches to Custom Ops

So far we've covered what I consider the proper way to implement custom ops. But you may encounter other patterns in existing code, and it's worth understanding why some approaches are better than others.

The Proper Way (What We Did)

Our call checker restructures the op tree properly:

Detaches arguments from the entersub
Creates a new OP_CUSTOM unop
Frees the old entersub entirely

newop = newUNOP(OP_CUSTOM, 0, argop);
newop->op_ppaddr = pp_entropy;
op_free(entersubop);
return newop;

This is more code, but it's correct. The op tree is clean, B::Concise shows your custom op, and it works in all contexts.

The Quick Hack (Avoid This)

Some other modules take a shortcut - they just swap the op_ppaddr pointer without restructuring the tree:

static OP* hack_call_checker(pTHX_ OP *entersubop, GV *namegv, SV *protosv) {
    PERL_UNUSED_ARG(namegv);
    PERL_UNUSED_ARG(protosv);
    entersubop->op_ppaddr = my_custom_op;
    return entersubop;  /* Return unchanged! */
}

The problem is that your pp function now has to deal with the entersub's stack frame, using TOPMARK/POPMARK manipulation:

static OP* my_custom_op(pTHX) {
    dSP;
    I32 ax = TOPMARK + 1;
    I32 items = (SP - PL_stack_base - TOPMARK) - 1;
    if (items < 1) croak("requires at least 1 argument");
    SV *sv = PL_stack_base[ax];
    /* ... do work ... */
    SP = PL_stack_base + POPMARK;
    EXTEND(SP, 1);
    PUSHs(newSVnv(result));  /* Memory leak! */
    PUTBACK;
    return NORMAL;
}

Why avoid this?

Fragile - Breaks when your function is called inside other expressions
Memory leaks - Easy to forget mortalization (note newSVnv() above leaks!)
Debugging confusion - B::Concise still shows entersub, not your custom op
Complex code - The pp function is harder to understand and maintain

Shared Call Checker (For Multiple Functions)

If you're building a library with many similar functions (like a maths library), you can use a generic call checker that takes a function pointer:

static OP* vec_unary_call_checker(pTHX_ OP *entersubop, GV *namegv, SV *ckobj, OP* (*pp_func)(pTHX)) {
    OP *pushop, *selfop, *nextop, *newop;
    PERL_UNUSED_ARG(namegv);
    PERL_UNUSED_ARG(ckobj);

    pushop = cUNOPx(entersubop)->op_first;
    if (!OpHAS_SIBLING(pushop)) return entersubop;

    selfop = OpSIBLING(pushop);
    if (!selfop) return entersubop;

    nextop = OpSIBLING(selfop);
    if (!nextop) return entersubop;

    if (OpSIBLING(nextop)) return entersubop;

    OpMORESIB_set(pushop, nextop);
    OpLASTSIB_set(selfop, NULL);

    newop = newUNOP(OP_CUSTOM, 0, selfop);
    newop->op_ppaddr = pp_func;
    op_free(entersubop);
    return newop;
}

Then create thin wrappers for each function:

static OP* vec_sum_call_checker(pTHX_ OP *entersubop, GV *namegv, SV *ckobj) {
    return vec_unary_call_checker(aTHX_ entersubop, namegv, ckobj, pp_vec_sum);
}

static OP* vec_mean_call_checker(pTHX_ OP *entersubop, GV *namegv, SV *ckobj) {
    return vec_unary_call_checker(aTHX_ entersubop, namegv, ckobj, pp_vec_mean);
}

static OP* vec_min_call_checker(pTHX_ OP *entersubop, GV *namegv, SV *ckobj) {
    return vec_unary_call_checker(aTHX_ entersubop, namegv, ckobj, pp_vec_min);
}

This keeps your code DRY while still doing proper op tree restructuring.

Binary Ops (Two Arguments)

For functions taking two arguments, use newBINOP instead of newUNOP:

static OP* binary_call_checker(pTHX_ OP *entersubop, GV *namegv, SV *ckobj) {
    OP *pushop, *arg1, *arg2, *nextop, *newop;
    PERL_UNUSED_ARG(namegv);
    PERL_UNUSED_ARG(ckobj);

    pushop = cUNOPx(entersubop)->op_first;
    if (!OpHAS_SIBLING(pushop)) return entersubop;

    arg1 = OpSIBLING(pushop);
    if (!arg1) return entersubop;

    arg2 = OpSIBLING(arg1);
    if (!arg2) return entersubop;

    nextop = OpSIBLING(arg2);
    if (!nextop) return entersubop;

    if (OpSIBLING(nextop)) return entersubop;

    OpMORESIB_set(pushop, nextop);
    OpLASTSIB_set(arg1, NULL);
    OpLASTSIB_set(arg2, NULL);

    newop = newBINOP(OP_CUSTOM, 0, arg1, arg2);
    newop->op_ppaddr = pp_my_binary_func;
    op_free(entersubop);
    return newop;
}

The pp function for a binary op pops two values:

static OP* pp_my_binary_func(pTHX) {
    dSP;
    SV *right = POPs;
    SV *left = POPs;
    /* combine left and right */
    mPUSHn(result);
    RETURN;
}

If you have any further questions then please do post them below.

LRU::Cache - A Fast Least Recently Used Cache For Perl

LNATION — Wed, 08 Apr 2026 12:08:47 +0000

Caching is one of those things every application needs eventually. Whether it's DNS lookups, database rows, or computed results, keeping frequently accessed data close avoids expensive recomputation. The classic data structure for this is the Least Recently Used (LRU) cache: a fixed-size store that automatically evicts the oldest unused entry when full.

LRU::Cache is a new CPAN module that implements an LRU cache entirely in C via XS. Every operation — get, set, delete, exists — is O(1). No Perl-level hash ties, no linked list objects, no method dispatch on the hot path if you don't want it.

Quick Start

use LRU::Cache;

my $cache = LRU::Cache::new(1000);

$cache->set('user:42', { name => 'Alice', role => 'admin' });
$cache->set('user:43', { name => 'Bob',   role => 'editor' });

my $user = $cache->get('user:42');   # promotes to front
print $user->{name};                 # "Alice"

The constructor takes a single argument: the maximum number of entries. Once the cache hits capacity, the next set evicts whatever was accessed longest ago.

The Eviction Model

Every get or set promotes an entry to the front of the list. The entry at the tail — the one untouched for the longest — is the first to go when the cache is full.

my $cache = LRU::Cache::new(3);

$cache->set('a', 1);
$cache->set('b', 2);
$cache->set('c', 3);

# Access 'a' — it moves to the front
$cache->get('a');

# Cache is full. This evicts 'b' (the least recently used).
$cache->set('d', 4);

# 'b' is gone
print $cache->get('b');   # undef

After the get('a'), the internal order is a → c → b. Adding d evicts b from the tail.

Peeking Without Promoting

Sometimes you want to check a value without affecting the eviction order. peek does exactly that:

my $cache = LRU::Cache::new(3);
$cache->set('a', 1);
$cache->set('b', 2);
$cache->set('c', 3);

# peek returns the value but doesn't promote
my $val = $cache->peek('a');   # 1

# 'a' is still the oldest
my ($oldest_key) = $cache->oldest;   # 'a'

# Now get('a') promotes it
$cache->get('a');
($oldest_key) = $cache->oldest;      # 'b'

This is useful when you're iterating over cache contents for monitoring or debugging without disturbing the access pattern.

Inspecting the Cache

oldest and newest return both the key and value:

my $cache = LRU::Cache::new(5);
$cache->set('first',  'I was first');
$cache->set('second', 'I was second');
$cache->set('third',  'I was third');

my ($newest_key, $newest_val) = $cache->newest;
# $newest_key = 'third', $newest_val = 'I was third'

my ($oldest_key, $oldest_val) = $cache->oldest;
# $oldest_key = 'first', $oldest_val = 'I was first'

keys returns all keys in most-recently-used order:

$cache->get('first');   # promote 'first'
my @keys = $cache->keys;
# ('first', 'third', 'second')

And delete hands back whatever was removed:

my $deleted = $cache->delete('second');
# $deleted = 'I was second'

The Function-Style API

Method dispatch in Perl isn't free. For a cache that sits in a tight loop, the overhead of $cache->get(...) — which goes through entersub, method resolution, and argument shifting — adds up. LRU::Cache offers a function-style API that is custom OP optimised that eliminates this entirely:

use LRU::Cache qw(import);

my $cache = LRU::Cache::new(10_000);

lru_set($cache, 'key', $value);
my $v   = lru_get($cache, 'key');
my $hit = lru_exists($cache, 'key');
my $p   = lru_peek($cache, 'key');
lru_delete($cache, 'key');

As I say these aren't just wrappers. At compile time, Perl's call checker mechanism replaces the entersub op with a custom op that goes directly to the C implementation. No method lookup, no @_ construction, no argument unpacking. The cache pointer is read directly off the pad.

The result is roughly 2x faster than the method-style API, and 5–20x faster than a pure Perl implementation.

Benchmarks

Single-operation rates, no batching loops — each iteration is one cache call:

Operation	Pure Perl	XS Method	XS Function
set (existing)	1,715,893/s	21,445,932/s	45,124,820/s
get (hit)	5,271,976/s	18,302,304/s	33,017,565/s
get (miss)	8,133,594/s	22,125,453/s	46,998,887/s
exists (hit)	7,080,776/s	19,521,882/s	38,745,035/s
peek	6,410,022/s	16,842,291/s	32,437,007/s

A Practical Example: Caching Expensive Lookups

Here's a pattern that comes up constantly — wrapping an expensive operation with a cache:

use LRU::Cache qw(import);

my $dns_cache = LRU::Cache::new(1000);

sub resolve {
    my ($domain) = @_;
    my $ip = lru_get($dns_cache, $domain);
    if (!defined $ip) {
        $ip = expensive_dns_lookup($domain);
        lru_set($dns_cache, $domain, $ip);
    }
    return $ip;
}

The cache transparently sits in front of the expensive call. Hits are served from C-backed memory in constant time. Misses fall through to the real lookup and get cached for next time. When the cache fills up, stale domains are evicted automatically.

Install

cpanm LRU::Cache

Heap::PQ — A Binary Heap (Priority Queue) Implementation for Perl

LNATION — Tue, 07 Apr 2026 07:09:03 +0000

So What Is A Binary Heap?

A binary heap is really just an sorted array pretending to be a tree. Each element has a parent and children, but instead of pointers you find them with simple array indexes. The trick is that every parent follows one rule relative to its children, and that rule decides what kind of heap you get:

Min heap - every parent is less than or equal to its children. The smallest element always lives at the root/top.
Max heap - every parent is greater than or equal to its children. The largest element always lives at the root/top.

Because the tree is complete, the parent of element at index i sits at index floor((i−1)/2), and its children sit at 2i+1 and 2i+2. No pointers, no linked lists — just arithmetic on array indices. That cache friendly layout is one reason heaps are so fast in practice.

The key operations:

Operation	Time	What it does
push	O(log n)	Insert a value, then sift it up to restore order
pop	O(log n)	Remove the root, move the last element up, sift down
peek	O(1)	Read the root without removing it
make_heap	O(n)	Turn an existing unsorted array into a valid heap (`make_heap_min` / `make_heap_max`, `new('min')`, `new('max')`)

This makes a heap the natural backbone of a priority queue — a collection where you always want the next most important item and you need insertions to stay fast.

Why Heap::PQ?

Heap::PQ brings binary heaps to Perl as a C extension. Where a pure Perl heap tops out around 300 operations per second, Heap::PQ's functional interface clocks over 11,000 when pushing a 1000 random integers — and if you are just handling plain integers then there is an optimised NV heap implementation that squeezes out 50% more performance. It ships with three interfaces (object oriented, functional custom ops or a raw array) so you can trade readability for throughput depending on what your code needs.

Installation

cpanm Heap::PQ

Getting Started

Create a heap, push values in, and they come back out in priority order:

use Heap::PQ;

my $pq = Heap::PQ::new('min');

$pq->push(5);
$pq->push(1);
$pq->push(3);

print $pq->peek, "\n";  # 1 — the smallest value is always at the root

while (!$pq->is_empty) {
    print $pq->pop, "\n";  # 1, 3, 5
}

Switch to 'max' and the largest element comes out first instead.

Three Ways to Use It

OO Interface

The object oriented API is the clearest to read if you're a traditional perl developer that likes OO. push returns the heap so calls can be chained:

my $heap = Heap::PQ::new('min');
$heap->push(10)->push(20)->push(5);
$heap->push_all(8, 2, 15);

my $top = $heap->pop;    # 2
my $size = $heap->size;  # 5

Functional Interface

Import with 'import' and Heap::PQ installs heap_push, heap_pop, heap_peek, heap_size, and heap_is_empty as custom ops. Perl compiles them down to optimised opcodes so there is no method dispatch overhead at runtime:

use Heap::PQ 'import';

my $h = Heap::PQ::new('min');

heap_push($h, 42);
heap_push($h, 7);

my $val  = heap_pop($h);      # 7
my $size = heap_size($h);     # 1
my $top  = heap_peek($h);     # 42

This is the fastest path as stated functional custom ops removes the method->dispatch overhead.

Raw Array Interface

You can also operate directly on a Perl array. Import with 'raw' to get make_heap_min, push_heap_min, pop_heap_min (and their _max counterparts). This skips the object entirely but the functional interface is still faster thanks to custom op optimisation:

use Heap::PQ 'raw';

my @data = (9, 4, 7, 1, 3);
Heap::PQ::make_heap_min(\@data);  # O(n) heapify in place

my $min = Heap::PQ::pop_heap_min(\@data);   # 1
Heap::PQ::push_heap_min(\@data, 2);

No object, no opaque struct — just an array whose layout satisfies the heap property. Useful when you already have data in an array and want to avoid copying it.

Custom Comparators

By default the heap compares values numerically. Pass a code reference as a second argument to new and you can heap order anything — hash references, objects, strings:

my $tasks = Heap::PQ::new('min', sub {
    $a->{due} <=> $b->{due}
});

$tasks->push({ name => 'Write tests',  due => 3 });
$tasks->push({ name => 'Ship release', due => 5 });
$tasks->push({ name => 'Fix bug',      due => 1 });

while (!$tasks->is_empty) {
    my $t = $tasks->pop;
    print "$t->{name}\n";
}
# Fix bug
# Write tests
# Ship release

The comparator receives two elements in $a and $b — exactly like Perl's sort — and should return a negative, zero, or positive value. String ordering works the same way:

my $alpha = Heap::PQ::new('min', sub { $a cmp $b });
$alpha->push_all('banana', 'apple', 'cherry');
print $alpha->pop, "\n";  # apple

Key Path Comparators

When your heap contains hash references and you just want to compare by a numeric field, pass the field name as a string instead of a code reference. The comparison happens entirely in C — no Perl callback overhead at all:

my $tasks = Heap::PQ::new('min', 'priority');

$tasks->push({ name => 'Write tests',  priority => 3 });
$tasks->push({ name => 'Ship release', priority => 5 });
$tasks->push({ name => 'Fix bug',      priority => 1 });

while (!$tasks->is_empty) {
    my $t = $tasks->pop;
    print "$t->{name}\n";
}
# Fix bug
# Write tests
# Ship release

Dot separated paths reach into nested hashes:

my $heap = Heap::PQ::new('min', 'meta.score');
$heap->push({ id => 'a', meta => { score => 42 } });
$heap->push({ id => 'b', meta => { score => 17 } });
print $heap->pop->{id};  # 'b'

The key is extracted once when you push, so sift operations run at the same speed as a plain numeric heap. In benchmarks, key path heaps match the no-comparator path (~6,400/s) while custom Perl comparators top out around ~1,200/s — a 5× difference.

NV Heaps — Native Doubles, No SV Overhead

When every value is a number, new_nv stores them as raw C doubles instead of Perl scalars. That eliminates SV allocation and reference counting entirely:

my $h = Heap::PQ::new_nv('min');
$h->push_all(3.14, 2.71, 1.41, 1.73);
print $h->pop, "\n";  # 1.41

In benchmarks the NV heap runs roughly 1.5× faster than the functional interface for push heavy workloads, and nearly 57× faster than pure Perl.

Searching and Deleting

Both search and delete accept a callback that receives each element in $_. search returns matching elements; delete removes them and returns how many were dropped:

my $pq = Heap::PQ::new('min');
$pq->push_all(1, 5, 10, 15, 20, 25);

# Find all elements greater than 12
my @big = $pq->search(sub { $_ > 12 });
# @big = (15, 20, 25)

# Remove them from the heap
my $removed = $pq->delete(sub { $_ > 12 });
# $removed = 3, heap now contains (1, 5, 10)

After a delete, Heap::PQ rebuilds the heap in O(n) with Floyd's algorithm so the ordering invariant is always intact.

Peeking at the Top N

peek_n returns the top N elements in sorted order without removing them:

my $scores = Heap::PQ::new('max');
$scores->push_all(88, 95, 72, 99, 84, 91);

my @top3 = $scores->peek_n(3);  # (99, 95, 91)
# Heap is unchanged — still has all 6 elements

peek_n works with the heap's built-in numeric comparison, so it is best suited to plain numeric heaps. For heaps that use a custom comparator, pop and re-push to inspect the top N.

Putting It Together: An Event Scheduler

A priority queue is a natural fit for an event loop — push events with a timestamp, pop them in chronological order:

use Heap::PQ 'import';

my $scheduler = Heap::PQ::new('min', 'time'); # compare by the 'time' key in C

heap_push($scheduler, { time => 1712500800, action => 'start_server' });
heap_push($scheduler, { time => 1712497200, action => 'load_config' });
heap_push($scheduler, { time => 1712504400, action => 'open_connections' });
heap_push($scheduler, { time => 1712502600, action => 'warm_cache' });

while (!heap_is_empty($scheduler)) {
    my $event = heap_pop($scheduler);
    printf "t=%d  %s\n", $event->{time}, $event->{action};
}

# t=1712497200  load_config
# t=1712500800  start_server
# t=1712502600  warm_cache
# t=1712504400  open_connections

Benchmarks

All numbers below are from pushing 1,000 random integers then popping them all, measured with Benchmark:

Implementation	Rate	vs Pure Perl
Pure Perl	305/s	—
Custom comparator	1,194/s	4x
Array::Heap	6,087/s	20x
Heap::PQ OO key path	6,359/s	21x
Heap::PQ OO	6,685/s	22x
Heap::PQ raw	8,665/s	28x
Heap::PQ func	11,416/s	37x
Heap::PQ NV	16,747/s	55x

The custom op overhead reduction is clearly visible in the functional row, and the NV heap's custom op implementation plus avoidance of SV allocation pushes throughput higher still.

If you have any questions please do comment.

A Side note: I'm currently on look out for a new contract or permanent opportunity. If you know of any relevant openings, I'd appreciate hearing from you - email@lnation.org

Chandra: Cross-Platform Desktop GUIs in Perl

LNATION — Sat, 04 Apr 2026 18:53:20 +0000

Building desktop applications has traditionally been a challenge in the Perl ecosystem. While we have excellent tools for web development, backend services, and system administration, creating native feeling desktop apps often meant learning and wrestling with complex GUI toolkits or abandoning Perl altogether.

Chandra attempts to change that equation.

What is Chandra?

Chandra is a Perl module that lets you build cross-platform desktop applications using the web technologies you already know — HTML, CSS, and JavaScript — while keeping your application logic in Perl. Under the hood, it leverages native webview components (WebKit on macOS/Linux, Edge on Windows) to render your UI, giving you the best of both worlds: native performance with web flexibility.

use Chandra::App;

my $app = Chandra::App->new(
    title  => 'My First App',
    width  => 800,
    height => 600,
);

$app->set_content('<h1>Hello from Perl!</h1>');
$app->run;

That's it. Five lines to create a windowed application.

The Bridge Between Worlds

The real power of Chandra lies in its seamless bidirectional communication between Perl and JavaScript. You can expose Perl functions to your frontend with a simple bind call:

my $app = Chandra::App->new(title => 'Counter');

my $count = 0;

$app->bind('increment', sub {
    $count++;
    return $count;
});

$app->bind('decrement', sub {
    $count--;
    return $count;
});

$app->set_content(<<'HTML');
<!DOCTYPE html>
<html>
<body>
    <h1 id="counter">0</h1>
    <button onclick="inc()">+</button>
    <button onclick="dec()">-</button>
    <script>
        async function inc() {
            let val = await window.chandra.invoke('increment');
            document.getElementById('counter').textContent = val;
        }
        // or
        function dec() {
            window.chandra.invoke('decrement').then(function(val) {
                document.getElementById('counter').textContent = val;
            });
        }
    </script>
</body>
</html>
HTML

$app->run;

Your JavaScript calls window.chandra.invoke('increment'), Perl increments the counter, and the result flows back as a Promise. JSON serialization happens automatically.

Building UIs the Perl Way

If you prefer constructing your UI in Perl rather than writing raw HTML strings, Chandra::Element provides a DOM-like interface:

use Chandra::Element;

my $ui = Chandra::Element->new({
    tag => 'div',
    class => 'container',
    style => { padding => '20px', background => '#f5f5f5' },
    children => [
        { tag => 'h1', data => 'Welcome' },
        {
            tag => 'button',
            data => 'Click Me',
            onclick => sub {
                my ($event, $app) = @_;
                print "Button clicked!\n";
            },
        },
    ],
});

$app->set_content($ui);

Event handlers are automatically wired up — when the button is clicked, your Perl callback fires.

Single-Page Apps with Built-in Routing

Modern desktop apps often benefit from SPA-style navigation. Chandra has you covered:

my $app = Chandra::App->new(title => 'My SPA');

$app->layout(sub {
    my ($body) = @_;
    return qq{
        <nav>
            <a href="/">Home</a>
            <a href="/settings">Settings</a>
        </nav>
        <div id="chandra-content">$body</div>
    };
});

$app->route('/' => sub {
    return '<h1>Home</h1><p>Welcome!</p>';
});

$app->route('/settings' => sub {
    return '<h1>Settings</h1><p>Configure your app...</p>';
});

$app->route('/user/:id' => sub {
    my (%params) = @_;
    return "<h1>User $params{id}</h1>";
});

$app->run;

Navigation happens client-side with no page reloads — just like a web SPA, but in a native window.

Native Integration Done Right

Chandra doesn't just wrap a browser. It provides genuine desktop integration:

Native Dialogs

# File picker
my $path = $app->dialog->open_file(title => 'Select a file');

# Directory picker  
my $dir = $app->dialog->open_directory;

# Save dialog
my $save_path = $app->dialog->save_file(
    title   => 'Save As',
    default => 'document.txt',
);

# Alert dialogs
$app->dialog->info(message => 'Operation complete!');
$app->dialog->error(message => 'Something went wrong');

System Tray

use Chandra::Tray;

my $tray = Chandra::Tray->new(
    app     => $app,
    icon    => 'icon.png',
    tooltip => 'My App',
);

$tray->add_item('Show Window' => sub { $app->show });
$tray->add_separator;
$tray->add_item('Quit' => sub { $app->terminate });
$tray->show;

Persistent Storage

my $store = $app->store;  # ~/.chandra/<app-name>/store.json

$store->set('theme', 'dark');
$store->set('window.width', 1200);
$store->set('recent_files', ['/path/a', '/path/b']);

my $theme = $store->get('theme');  # 'dark'

Dot notation for nested keys, automatic JSON serialization, atomic writes with file locking — storage that just works.

Hot Reload

Wtch your source files and refresh automatically:

$app->watch('lib/', sub {
    my ($changed) = @_;
    print "Files changed: @$changed\n";
    $app->set_content(build_ui());
    $app->refresh;
});

$app->run;

DevTools

Need to inspect elements or debug JavaScript? Enable developer tools:

my $app = Chandra::App->new(
    title => 'Debug App',
    debug => 1,  # Press F12 to open DevTools
);

Platform Support

Chandra works across the major desktop platforms:

macOS — Uses WebKit via WKWebView
Linux — Uses WebKitGTK
Windows — Uses MSHTML/Edge

Getting Started

Install from CPAN:

cpanm Chandra

Or from source:

perl Makefile.PL
make
make install

Then dive into the examples directory for working demonstrations of all major features or I also have several modules already available for you to test:

Chandra::EPUB

Ascii::Text::Chandra

Chandra::Game::Tetris

A Side note: I'm currently on look out for a new contract or permanent opportunity. If you know of any relevant openings, I'd appreciate hearing from you - email@lnation.org

Introducing Object::Proto — A Prototype Object System for Perl

LNATION — Fri, 03 Apr 2026 11:54:34 +0000

Perl's object system is famously flexible. bless a reference, add some accessors, and you're in business. Over the years, modules like Moose, Moo, and Mouse have built increasingly sophisticated layers on top of that foundation — giving us type constraints, roles, lazy attributes, and declarative syntax.

But they all share the same bedrock: a blessed hash reference.

Object::Proto takes a less used approach. It stores objects as blessed arrays, maps property names to slot indices at compile time, and compiles accessors down to custom ops — the same mechanism Perl uses internally for built-in operations. The result is an object system that is type-safe, feature-rich, and significantly faster than the previously mentioned alternatives.

The Basics

use Object::Proto;

object 'Animal',
    'name:Str:required',
    'sound:Str:default(silence)',
    'age:Int';

my $cat = new Animal name => 'Whiskers', age => 3;
print $cat->name;      # Whiskers
print $cat->sound;     # silence
$cat->age(4);          # setter

The object keyword defines a class at compile time. Property specifications use a colon-separated format: name:Type:modifier. No hash lookups at runtime — $cat->name compiles to a direct array slot access.

Both positional and named constructors are supported:

my $cat = new Animal 'Whiskers', 'meow', 3;     # positional (fastest)
my $cat = new Animal name => 'Whiskers', age => 3; # named pairs

What You Get

Built-in Types (Zero Overhead)

Any  Defined  Str  Int  Num  Bool  ArrayRef  HashRef  CodeRef  Object

These are checked inline at the C level — no function call, no callback. You can also register custom types in Perl or in XS.

Slot Modifiers

object 'Person',
    'name:Str:required',
    'email:Str:required:readonly',
    'age:Int:default(0)',
    'nickname:Str:clearer:predicate',
    'settings:HashRef:lazy:builder(_build_settings)',
    'parent:Object:weak',
    'internal_id:Int:arg(_id)',       # constructor uses _id, accessor uses internal_id
    'score:Num:trigger(on_score_change)',
    'rank:Str:reader(get_rank):writer(set_rank)';

Everything you'd expect from a modern object system: required, readonly, default, lazy, builder, clearer, predicate, reader, writer, trigger, weak references, and arg (init_arg) — all compiled to C/XS.

Inheritance

object 'Animal', 'name:Str:required', 'sound:Str';

object 'Dog',
    extends => 'Animal',
    'breed:Str';

my $dog = new Dog name => 'Rex', sound => 'Woof', breed => 'Lab';
print $dog->isa('Animal');  # 1

Multiple inheritance and multi-level chains work as youl would expect:

object 'Triathlete',
    extends => ['Swimmer', 'Runner'],
    'event:Str';

Important note: because Object::Proto objects are arrays and Moo/Moose/Mouse objects are hashes, you cannot inherit from Moo, Moose, or Mouse classes (or vice versa). The internal representations are fundamentally incompatible. If you need to interoperate, use composition (roles or delegation via slots) rather than inheritance.

Roles

Roles work the way you'd expect — define a bundle of slots and method requirements, then compose them into any class. A role can carry its own attributes and demand that consuming classes implement specific methods:

use Object::Proto;

role('Printable', 'format:Str:default(text)');
requires('Printable', 'to_string');

object 'Document',
    'title:Str:required',
    'body:Str';

with('Document', 'Printable');

package Document;
sub to_string { $_[0]->title . ': ' . $_[0]->body }

The format slot from Printable is merged into Document at define time. If Document didn't provide a to_string method, the with call would croak. You can check role consumption at runtime with Object::Proto::does($obj, 'Printable').

Method Modifiers

You can wrap existing methods without subclassing. before runs code ahead of the original, after runs code after it, and around gives you full control — you receive the original method as $orig and decide whether (and how) to call it:

before 'bark' => sub {
    my ($self) = @_;
    warn "About to bark...";
};

after 'bark' => sub {
    my ($self) = @_;
    warn "Done barking.";
};

around 'bark' => sub {
    my ($orig, $self, @args) = @_;
    return uc $self->$orig(@args);
};

Multiple modifiers can be stacked on the same method. They're stored as linked lists in C, so the dispatch overhead is minimal.

Prototype Chains

This is where Object::Proto gets its name. Like JavaScript's prototype model, any object can delegate to another object. When you access a property that is undef in the current object, the lookup walks up the prototype chain until it finds a defined value:

my $defaults = new Animal name => 'Default', sound => 'silence';
my $cat = new Animal name => 'Whiskers';
$cat->set_prototype($defaults);

print $cat->sound;  # 'silence' — resolved from prototype

$cat has no sound of its own, so the accessor walks up to $defaults and returns 'silence'. Setting $cat->sound('meow') writes to $cat directly — prototypes are never mutated by child writes. You can inspect the full chain with Object::Proto::prototype_chain($obj) and measure its depth with Object::Proto::prototype_depth($obj).

Mutability Controls

Objects can be locked (preventing structural changes) or frozen (permanent deep immutability). A frozen object's setters will croak on any attempt to modify — useful for configuration objects, default prototypes, or any value you want to guarantee stays constant:

$obj->lock;       # prevent structural changes
$obj->unlock;
$obj->freeze;     # permanent immutability — setters will croak
$obj->is_frozen;  # true

Locking is reversible; freezing is not. The frozen check is implemented as a single bitflag test in C — it adds no measurable overhead to the normal (unfrozen) code path.

Cloning & Introspection

Object::Proto provides a full introspection API. You can clone objects (the clone is always fresh, unfrozen and unlocked, even if the original wasn't), list properties, inspect slot metadata, and walk the inheritance tree:

my $clone = Object::Proto::clone($obj);     # shallow copy, unfrozen
my @props = Object::Proto::properties('Dog');
my $info  = Object::Proto::slot_info('Dog', 'name');
my @chain = Object::Proto::ancestors('Dog');

slot_info returns a hashref with everything about a slot: its type, whether it's required, readonly, lazy, has a default, trigger, builder, and so on. Useful for building serializers, form generators, or debugging tools.

Singletons

Need a class that only ever has one instance? Mark it as a singleton and use ->instance() instead of new. The first call constructs the object; every subsequent call returns the same one:

object 'Config', 'db_host:Str', 'db_port:Int';
Object::Proto::singleton('Config');

my $c = Config->instance(db_host => 'localhost', db_port => 5432);
# Config->instance returns the same object every time after first call

This is handled at the C level — no Perl-side state variable or CHECK block trickery.

Function-Style Accessors

For the absolute fastest access, Object::Proto offers function-style accessors that bypass method dispatch entirely. These are compiled to custom ops at compile time — a direct array slot read/write with no entersub, no method resolution, no hash lookup:

use Object::Proto;

object 'Point', 'x:Num', 'y:Num';
Object::Proto::import_accessors('Point');

my $p = new Point 1.0, 2.0;
print x($p);         # 1.0 — custom op, not a method call
y($p, 3.0);          # set y to 3.0

Performance

Here are benchmarks from a mixed new + set + get workload:

                       Rate    Pure Hash  Pure Array    XS OO   Typed  Raw Hash Ref  XS func  Raw Hash
Pure Hash         1824921/s           --       -14%     -48%     -64%       -66%       -66%     -70%
Pure Array        2124357/s          16%         --     -40%     -58%       -60%       -61%     -65%
Object::Proto OO  3541833/s          94%        67%       --     -31%       -34%       -35%     -41%
Object::Proto T   5114883/s         180%       141%      44%       --        -4%        -6%     -15%
Raw Hash Ref      5331773/s         192%       151%      51%       4%         --        -2%     -12%
Object::Proto fn  5427162/s         197%       156%      53%       6%        2%          --     -10%
Raw Hash          6024884/s         230%       184%      70%      18%       13%        11%       --

The function-style path (XS func) runs slightly faster than raw hash reference access — while giving you a real object with type constraints, constructors, and a class hierarchy. The method-style path (XS OO) is nearly 2x faster than a pure-Perl hash-based object.

For hot set + get loops on pre-constructed objects, the XS function path has been measured at over 32 million operations per second — faster than a hash $hash{key} access.

Extending the Type System from XS

If you write XS modules, you can register types with C-level check functions that run at near-zero cost (~5 CPU cycles vs ~100 for Perl callbacks):

/* In your BOOT section */
#include "object_types.h"

static bool check_positive_int(pTHX_ SV *val) {
    return SvIOK(val) && SvIV(val) > 0;
}

BOOT:
    object_register_type_xs(aTHX_ "PositiveInt", check_positive_int, NULL);

use MyTypes;  # registers in BOOT
use Object::Proto;

object 'Counter', 'value:PositiveInt:required';

For Moo/Moose Users: Object::Proto::Sugar

If you prefer declarative Moo-style syntax, Object::Proto::Sugar wraps the XS layer with familiar keywords. Everything compiles down to the same custom ops underneath:

package Animal;
use Object::Proto::Sugar;

has name  => ( is => 'rw', isa => Str, required => 1 );
has sound => ( is => 'rw', isa => Str, default  => 'silence' );
has age   => ( is => 'rw', isa => Int );

sub speak { $_[0]->sound }

package Dog;
use Object::Proto::Sugar;

extends 'Animal';

has breed => ( is => 'rw', isa => Str );

package main;

my $dog = new Dog name => 'Rex', sound => 'Woof', breed => 'Lab';
print $dog->speak;         # Woof
print $dog->isa('Animal'); # 1

Every feature from the core module is available through the Sugar interface: lazy, builder, coerce, trigger, predicate, clearer, reader, writer, weak_ref, init_arg, roles (use Object::Proto::Sugar -role / with / requires), method modifiers (before, after, around), and function-style accessors via accessor => 1. The Sugar layer runs at compile time via BEGIN::Lift and then Devel::Hook via BIGIN and UNITCHECK hooks — by the time your first line of runtime code executes, everything has been compiled down to the same custom ops as the raw object keyword.

Sugar with Function-Style Accessors

Add accessor => 1 to a has declaration to get a function style accessor alongside the method style one. Then call import_accessors in your package so the functions are visible when your calling code compiles.

package Point;
use Object::Proto::Sugar;

has x => ( is => 'rw', isa => Num, accessor => 1 );
has y => ( is => 'rw', isa => Num, accessor => 1 );

package main;

Point->import_accessors;     # install before compilation

my $p = new Point x => 1.0, y => 2.0;
print x($p);       # 1.0 — compiled to custom op
y($p, 3.0);        # direct array write

Sugar Benchmarks vs Moo and Mouse

                       Rate       Moo     Mouse   Sugar   Sugar (fn)
Moo               1264320/s        --       -2%    -55%       -60%
Mouse             1290237/s        2%        --    -54%       -59%
Sugar (method)    2784378/s      120%      116%      --       -12%
Sugar (func)      3174093/s      151%      146%     14%         --

Sugar method-style is 2.2x Moo. Sugar function-style is 2.5x Moo.

A Note on Compatibility

Once again a reminder: because Object::Proto objects are arrays and Moo/Moose/Mouse objects are hashes, you cannot inherit across the boundary. extends 'My::Moo::Class' will not work. Use role or delegation to bridge the two worlds if needed. Within its own ecosystem — Object::Proto classes inheriting from other Object::Proto classes — everything works seamlessly, including multiple inheritance.

Object::Proto is available on CPAN and licensed under the Artistic License 2.0.

cpanm Object::Proto
cpanm Object::Proto::Sugar   # optional, for Moo-like syntax

MetaCPAN

Loo: Yet Another Way To Introspect Data

LNATION — Wed, 01 Apr 2026 07:48:49 +0000

A Side note: I'm currently on look out for a new contract or permanent opportunity. If you know of any relevant openings, I'd appreciate hearing from you. I am a proficient front-end developer also - email@lnation.org

If you've spent any time debugging Perl, you've used Data::Dumper. It's one of those modules that ships with every Perl installation, gets loaded into every debugging session, and does its job without complaint. But it also hasn't changed much in a long time. The output is monochrome, the internals are pure Perl, and code references remain opaque sub { "DUMMY" } blobs unless you enable Deparse and even then you do not get the response one would expect.

Loo is a new take on the same problem: dump Perl data structures to readable output, but do it in C, with colour, and with a built-in code deparser that walks the op tree directly.

Why Another Dumper?

Three reasons drove the creation of Loo:

Speed. Loo is implemented entirely in XS. The dump logic, string escaping, colour code generation, and op tree walking all happen in C. For large or deeply nested structures, this matters.

Colour out of the box. When your terminal supports it, Loo's output is coloured by default. Strings, numbers, hash keys, braces, blessed class names, regex patterns, and code all get distinct colours that can be customised. There's no separate module to install, no formatter to configure. It auto-detects terminal capability, respects $ENV{NO_COLOR}, and falls back to plain text when appropriate.

Code deparsing without B::Deparse. When you pass a code reference to Loo with deparsing enabled, it walks Perl's internal op tree in C and reconstructs the source. This is not a wrapper around B::Deparse — it's a standalone implementation that lives in the same XS compilation unit as the introspecter itself.

Getting Started

Loo provides a functional interface that mirrors Data::Dumper closely enough that switching is straightforward:

use Loo qw(Dump cDump ncDump dDump);

# Colour auto-detected based on terminal
print Dump({ name => 'Perl', version => 5.40 });

# Force colour on (useful when piping to a pager that supports ANSI)
print cDump([1, 2, 3]);

# Force colour off (useful for logging or file output)
print ncDump(\%ENV);

# Dump with code deparsing enabled
print dDump(sub { my ($x) = @_; return $x * 2 });

The OO interface supports method chaining and gives you access to the full set of configuration options:

my $loo = Loo->new([{ key => 'value' }], ['data']);
$loo->Indent(1)->Sortkeys(1)->Theme('monokai');
print $loo->Dump;

Data::Dumper Compatibility

Loo implements the same accessor interface as Data::Dumper: Indent, Terse, Varname, Useqq, Quotekeys, Sortkeys, Maxdepth, Maxrecurse, Purity, Deepcopy, Pair, Freezer, Toaster, Bless, Deparse, Trailingcomma, Sparseseen, and Pad. If you have existing code that configures a Data::Dumper object, the same method calls work on a Loo object and is faster.

Beyond that, Loo adds a few options that Data::Dumper doesn't have:

Indentwidth($n) — control the number of characters per indent level (default 2)
Usetabs($bool) — indent with tabs instead of spaces

Colour Customisation

Loo ships with four built-in themes: default, light (optimised for light terminal backgrounds), monokai, and none.

For fine-grained control, the Colour method accepts a hash specifying foreground and background colours for 17 distinct syntax elements:

$loo->Colour({
    string_fg  => 'green',
    key_fg     => 'magenta',
    number_fg  => 'cyan',
    brace_fg   => 'bright_black',
    undef_fg   => 'red',
});

The colorable elements cover every visual component of the output: string, number, key, brace, bracket, paren, arrow, comma, undef, blessed, regex, code, variable, quote, keyword, operator, and comment.

Colour codes are pre-computed once at configuration time, so there's no per-character overhead during the dump.

The Deparser

The most unusual feature of Loo is its built-in code deparser. When you enable deparsing, Loo walks the Perl op tree directly in C — the same internal structure that the Perl interpreter executes — and reconstructs Perl source code from it.

my $loo = Loo->new([\&Some::function]);
$loo->Deparse(1);
print $loo->Dump;

This means code references in your data structures are no longer black boxes. You see the actual code that will be run from the code. NOTE: It may not be identical to the code written as some postfix operations are lost as I am deparsing the compiled op tree itself.

Getting this to work across Perl versions was one of the harder parts of the project. The op tree structure has changed across Perl releases — OP_PADSV_STORE appeared in 5.38, OP_EMPTYAVHV landed in 5.36 as an enum rather than a #define, and PADNAME typedefs shifted between 5.20 and 5.22. Loo handles all of this with version-conditional compilation, supporting Perl 5.10 through to the latest releases.

Reusable C Headers

Loo's XS code is organised into modular C headers:

loo.h — core definitions, themes, colour element names
loo_colour.h — ANSI colour code generation
loo_escape.h — string escaping
loo_dump.h — recursive data structure dumping
loo_deparse.h — op tree walking and code reconstruction

These headers are installed alongside the Perl module, and Loo->include_dir returns their path. This means other XS modules can reuse Loo's colour or escaping logic without duplicating the C code.

Auto-Detection Done Right

Loo follows the no-color.org convention and layers several checks to decide whether to emit ANSI codes:

If $Loo::USE_COLOUR is set, that takes precedence
If $ENV{NO_COLOR} is set, colour is disabled
If $ENV{TERM} is "dumb", colour is disabled
If STDOUT is not a terminal, colour is disabled
Otherwise, colour is enabled

This means Dump() does the right thing whether you're debugging interactively, piping to a file, or running in CI. And cDump() / ncDump() are there when you need to override.

Installation

Loo is available on CPAN:

cpanm Loo

It requires Perl 5.008003 or later and a C compiler (which you already have if you've ever installed an XS module).

Closing Thoughts

Data::Dumper is a workhorse that has served the Perl community well for decades. Loo isn't trying to replace it everywhere — but if you spend a lot of time reading dump output, colour and deparsing make a real difference. And if you're dumping large structures in production logging or tooling, the XS implementation gives you that output faster.

Give it a look. Your eyes may thank you.

Learning XS - Sharing Reusable C Headers Between Perl XS Distributions

LNATION — Sun, 29 Mar 2026 12:00:05 +0000

Introduction

Since I last wrote a XS tutorial my knowledge within C has increased this has come from improvements in LLM software that has assisted in improving my knowledge where previously I would be stuck. This knowledge and software has since enabled me to craft more elegant and efficient XS implementations.

Today I will share with you my technique for writing reusable C/XS code.

One of the most powerful patterns in XS development is writing your core logic in pure C header files. This gives you:

Zero-cost reuse - no runtime linking, no shared libraries, just a #include line.
No Perl dependency in the C layer - your headers work in any C project
Compile-time inlining - the compiler sees everything, optimises aggressively
Simple distribution - headers are installed alongside the Perl module via PM

This tutorial walks through the complete pattern step by step, using a minimal working example you can build and run yourself.

The Example

We will create two distributions:

Abacus - a provider distribution that ships a reusable pure-C abacus_math.h header containing simple arithmetic functions
Tally - a consumer distribution that #includes the Abacus header to build its own XS module, without duplicating any C code

As always lets start by creating the distributions that we will need for this tutorial. Open your terminal and run module-starter. If you are using a modern version of Module::Starter then the command has changed slightly since my last posts.

  module-starter --module=Abacus --author="LNATION <email@lnation.org>"
  module-starter --module=Tally --author="LNATION <email@lnation.org>"

Part 1: The Provider Distribution (Abacus)

Write the pure-C header

This is the reusable part. It has zero Perl dependencies - just standard C.

Now enter the Abacus and create the include directory:

  cd Abucus
  mkdir include

Then create a new file abacus_math.h:

  touch abacus_math.h
  vim abacus_math.h

Paste the following code into the file:

#ifndef ABACUS_MATH_H
#define ABACUS_MATH_H

/*
 * abacus_math.h - Pure C arithmetic library (no Perl dependencies)
 *
 * This header is the reusable entry point for any C or XS project
 * that needs basic arithmetic operations. It has ZERO Perl/XS
 * dependencies.
 *
 * Usage from another XS module:
 *
 *     #include "abacus_math.h"
 *
 * Build: add -I/path/to/Abacus/include to your compiler flags.
 */

#include <stdint.h>

/* ── Error handling hook ─────────────────────────────────────────
 *
 * Consumers can #define ABACUS_FATAL(msg) before including this
 * header to route errors through their own mechanism.
 *
 * In an XS module you would typically do:
 *
 *     #define ABACUS_FATAL(msg) croak("%s", (msg))
 *     #include "abacus_math.h"
 *
 * In plain C the default behaviour is fprintf + abort.
 */
#ifndef ABACUS_FATAL
#  include <stdio.h>
#  include <stdlib.h>
#  define ABACUS_FATAL(msg) do { \
       fprintf(stderr, "abacus fatal: %s\n", (msg)); \
       abort(); \
   } while (0)
#endif

/* ── Arithmetic operations ───────────────────────────────────── */

static inline int32_t
abacus_add(int32_t a, int32_t b) {
    return a + b;
}

static inline int32_t
abacus_subtract(int32_t a, int32_t b) {
    return a - b;
}

static inline int32_t
abacus_multiply(int32_t a, int32_t b) {
    return a * b;
}

static inline int32_t
abacus_divide(int32_t a, int32_t b) {
    if (b == 0) {
        ABACUS_FATAL("division by zero");
    }
    return a / b;
}

static inline int32_t
abacus_factorial(int32_t n) {
    int32_t result = 1;
    int32_t i;
    if (n < 0) {
        ABACUS_FATAL("factorial of negative number");
    }
    for (i = 2; i <= n; i++) {
        result *= i;
    }
    return result;
}

#endif /* ABACUS_MATH_H */

The code above demonstrates three critical design patterns for reusable C headers:

static inline functions eliminate linker complications by giving each translation unit its own copy of the function. The compiler can then inline these small arithmetic operations directly into the call site, producing zero-overhead abstractions. This is key to the "zero-cost reuse" principle—there is no shared library dependency, no function call overhead, just pure generated code.

The ABACUS_FATAL macro hook provides a customization point for error handling. By default, it calls fprintf() and abort() in standalone C programs; but consumers can #define ABACUS_FATAL(msg) croak("%s", (msg)) before including the header to integrate seamlessly with Perl's exception system. This single mechanism allows the same C header to work across Perl XS, plain C, and other environments without code duplication.

The use of only stdint.h integers and no Perl types ensures the header remains truly portable. There are no SV* pointers, no pTHX context variables, no XSUB.h includes—just standard C99 types. This purity is what allows the header to be #included into any C or XS project without creating hidden Perl dependencies at the C layer.

Write the Perl-facing XS header

Next we will add another header which will hold the Perl/XS specific logic. Inside the include directory create a new file called abacus.h. The rational behind this thin wrapper is to pull in Perl's headers and sets up the ABACUS_FATAL macro to use croak(). To reiterate only a XS distribution should include this header, whereas abacus_math.h is generic and could be used by other languages which bind C.

    touch abacus.h
    vim abacus.h

Paste the following code into the file

#ifndef ABACUS_H
#define ABACUS_H

/*
 * abacus.h - Perl XS wrapper header for the Abacus library
 *
 * This header sets up Perl-specific error handling and includes
 * the pure C core library.
 *
 * For reuse from OTHER XS modules without Perl overhead, include
 * abacus_math.h directly instead (see that header for usage).
 */

#define PERL_NO_GET_CONTEXT
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "ppport.h"

/* Route fatal errors through Perl's core croak() */
#define ABACUS_FATAL(msg) croak("%s", (msg))

/* Pull in the pure-C library */
#include "abacus_math.h"

#endif /* ABACUS_H */

Now any .xs file can #include "abacus.h" and get the full Perl/XS environment plus all the pure-C functions, with errors properly integrated.

Write the XS file

Next we will create the XS file, return to the root directory and then enter the lib directoy where you should see the Abacus.pm file already. Create a new XS file called Abacus.xs. This will be the glue that exposes the C functions to Perl.

  cd ../lib
  touch Abacus.xs
  vim Abacus.xs

#include "abacus.h"

MODULE = Abacus  PACKAGE = Abacus

PROTOTYPES: DISABLE

int
add(a, b)
    int a
    int b
  CODE:
    RETVAL = abacus_add(a, b);
  OUTPUT:
    RETVAL

int
subtract(a, b)
    int a
    int b
  CODE:
    RETVAL = abacus_subtract(a, b);
  OUTPUT:
    RETVAL

int
multiply(a, b)
    int a
    int b
  CODE:
    RETVAL = abacus_multiply(a, b);
  OUTPUT:
    RETVAL

int
divide(a, b)
    int a
    int b
  CODE:
    RETVAL = abacus_divide(a, b);
  OUTPUT:
    RETVAL

int
factorial(n)
    int n
  CODE:
    RETVAL = abacus_factorial(n);
  OUTPUT:
    RETVAL

As you can see we include abacus.h which pulls in all the C that we need to create our XS module. We then define add, subtract, multiply, divide and factorial as XSUBs. As you should know by now XSUBs can be called directly from your perl code.

Write the Perl module with `include_dir()`

Next open the pm file and update to add an exporter for the XSUBS we have just created.

package Abacus;

use 5.008003;
use strict;
use warnings;

our $VERSION = '0.01';

use Exporter 'import';
our @EXPORT_OK = qw(add subtract multiply divide factorial);

require XSLoader;
XSLoader::load('Abacus', $VERSION);

Now the critical piece that makes header sharing work. A include_dir() method which returns the path to the installed headers so that consumer distributions can find them at build time.

sub include_dir {
    my $dir = $INC{'Abacus.pm'};
    $dir =~ s{Abacus\.pm$}{Abacus/include};
    return $dir;
}

1;

How include_dir() works:

When Perl loads Abacus.pm, it records the full path in %INC (e.g. /usr/lib/perl5/site_perl/Abacus.pm)
include_dir() replaces Abacus.pm with Abacus/include
That directory exists because Makefile.PL installs the headers there (see next step)

Write the Makefile.PL that installs headers

The PM hash is what makes headers available to other distributions after install. It maps source files to their installation destinations.

Abacus/Makefile.PL

use 5.008003;
use strict;
use warnings;
use ExtUtils::MakeMaker;

WriteMakefile(
    NAME             => 'Abacus',
    AUTHOR           => 'Your Name <you@example.com>',
    VERSION_FROM     => 'lib/Abacus.pm',
    ABSTRACT_FROM    => 'lib/Abacus.pm',
    LICENSE          => 'artistic_2',
    MIN_PERL_VERSION => '5.008003',
    CONFIGURE_REQUIRES => {
        'ExtUtils::MakeMaker' => '0',
    },
    TEST_REQUIRES => {
        'Test::More' => '0',
    },
    PREREQ_PM => {},
    XSMULTI => 1,
    # XS configuration
    INC    => '-I. -Iinclude',
    OBJECT => '$(O_FILES)',

    # *** THIS IS THE KEY PART ***
    # Install headers alongside the module so dependent
    # distributions can find them via Abacus->include_dir()
    PM => {
        'lib/Abacus.pm'           => '$(INST_LIB)/Abacus.pm',
        'include/abacus.h'        => '$(INST_LIB)/Abacus/include/abacus.h',
        'include/abacus_math.h'   => '$(INST_LIB)/Abacus/include/abacus_math.h',
    },

    dist  => { COMPRESS => 'gzip -9f', SUFFIX => 'gz' },
    clean => { FILES => 'Abacus-*' },
);

The PM hash does two things:

Installs Abacus.pm as normal
Copies the header files into Abacus/include/ alongside the module

After make install, the filesystem looks like:

site_perl/
  Abacus.pm
  Abacus/
    include/
      abacus.h
      abacus_math.h

Write a test

Abacus/t/01-basic.t

use strict;
use warnings;
use Test::More;
use Abacus qw(add subtract multiply divide factorial);

is(add(2, 3),       5,   'add');
is(subtract(10, 4), 6,   'subtract');
is(multiply(3, 7),  21,  'multiply');
is(divide(20, 4),   5,   'divide');
is(factorial(5),     120, 'factorial');

eval { divide(1, 0) };
like($@, qr/division by zero/, 'divide by zero croaks');

eval { factorial(-1) };
like($@, qr/negative/, 'negative factorial croaks');

done_testing;

Build and install Abacus

cd Abacus
perl Makefile.PL
make
make test
make install          # installs headers into site_perl

Part 2: The Consumer Distribution (Tally)

Tally is a separate distribution that reuses Abacus's C arithmetic without duplicating any code. It adds its own "running total" functionality on top.

Write the Makefile.PL that finds Abacus headers

This is where the consumer locates the provider's headers. The two-step resolution strategy supports both installed (CPAN) and development (sibling
directory) scenarios.

Tally/Makefile.PL

use 5.008003;
use strict;
use warnings;
use ExtUtils::MakeMaker;

# Resolve Abacus include directory:
#   1. Try installed Abacus module (CPAN / system)
#   2. Fall back to sibling directory (development)
my $abacus_inc;
eval {
    no warnings 'redefine';
    local *XSLoader::load = sub {};  # skip XS bootstrap
    require Abacus;
    my $dir = Abacus->include_dir();
    $abacus_inc = $dir if $dir && -d $dir;
};
if (!$abacus_inc && -d '../Abacus/include') {
    $abacus_inc = '../Abacus/include';
}
die "Cannot find Abacus include directory.\n"
  . "Install Abacus or place it as a sibling directory.\n"
    unless $abacus_inc;

WriteMakefile(
    NAME             => 'Tally',
    AUTHOR           => 'Your Name <you@example.com>',
    VERSION_FROM     => 'lib/Tally.pm',
    ABSTRACT_FROM    => 'lib/Tally.pm',
    LICENSE          => 'artistic_2',
    MIN_PERL_VERSION => '5.008003',
    CONFIGURE_REQUIRES => {
        'ExtUtils::MakeMaker' => '0',
        'Abacus'              => '0.01',
    },
    TEST_REQUIRES => {
        'Test::More' => '0',
    },
    PREREQ_PM => {
        'Abacus' => '0.01',
    },

    # Point the compiler at Abacus's installed headers
    INC    => "-I$abacus_inc",
    OBJECT => '$(O_FILES)',

    dist  => { COMPRESS => 'gzip -9f', SUFFIX => 'gz' },
    clean => { FILES => 'Tally-*' },
);

Let's walk through the header resolution:

Try the installed path first - require Abacus loads the module, then Abacus->include_dir() returns the path where the headers were installed. We stub out XSLoader::load because we only need the pure-Perl include_dir() method, not the XS functions.
Fall back to sibling directory - during development, Abacus and Tally often live side by side. ../Abacus/include handles this case.
Die with a clear message if neither path works.

The resolved path is passed to INC, which adds it to the C compiler's include search path (-I/path/to/Abacus/include).

Abacus is listed in both CONFIGURE_REQUIRES and PREREQ_PM:

CONFIGURE_REQUIRES ensures Abacus is installed before Makefile.PL runs (needed because we require Abacus at configure time)
PREREQ_PM ensures it is available at runtime too

Write the XS file

This is where the reuse happens. Tally includes abacus_math.h directly -
no Perl coupling, just pure C function calls.

Tally/Tally.xs

#define PERL_NO_GET_CONTEXT
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"

/* Hook Abacus errors into Perl's croak() */
#define ABACUS_FATAL(msg) croak("%s", (msg))

/* Include the pure-C header from Abacus - no Perl deps in the header */
#include "abacus_math.h"

/* ── Tally's own C logic, built on top of Abacus ─────────────── */

typedef struct {
    int32_t total;
} tally_state_t;

static inline void
tally_init(tally_state_t *state) {
    state->total = 0;
}

static inline int32_t
tally_add(tally_state_t *state, int32_t value) {
    state->total = abacus_add(state->total, value);
    return state->total;
}

static inline int32_t
tally_subtract(tally_state_t *state, int32_t value) {
    state->total = abacus_subtract(state->total, value);
    return state->total;
}

static inline int32_t
tally_multiply_total(tally_state_t *state, int32_t value) {
    state->total = abacus_multiply(state->total, value);
    return state->total;
}

static inline int32_t
tally_get(tally_state_t *state) {
    return state->total;
}

static inline void
tally_reset(tally_state_t *state) {
    state->total = 0;
}

/* ── XS bindings ─────────────────────────────────────────────── */

MODULE = Tally  PACKAGE = Tally

PROTOTYPES: DISABLE

SV *
new(class)
    const char *class
  CODE:
    tally_state_t *state;
    Newxz(state, 1, tally_state_t);
    tally_init(state);
    RETVAL = newSV(0);
    sv_setref_pv(RETVAL, class, (void *)state);
  OUTPUT:
    RETVAL

int
add(self, value)
    SV *self
    int value
  CODE:
    tally_state_t *state = INT2PTR(tally_state_t *, SvIV(SvRV(self)));
    RETVAL = tally_add(state, value);
  OUTPUT:
    RETVAL

int
subtract(self, value)
    SV *self
    int value
  CODE:
    tally_state_t *state = INT2PTR(tally_state_t *, SvIV(SvRV(self)));
    RETVAL = tally_subtract(state, value);
  OUTPUT:
    RETVAL

int
multiply_total(self, value)
    SV *self
    int value
  CODE:
    tally_state_t *state = INT2PTR(tally_state_t *, SvIV(SvRV(self)));
    RETVAL = tally_multiply_total(state, value);
  OUTPUT:
    RETVAL

int
total(self)
    SV *self
  CODE:
    tally_state_t *state = INT2PTR(tally_state_t *, SvIV(SvRV(self)));
    RETVAL = tally_get(state);
  OUTPUT:
    RETVAL

void
reset(self)
    SV *self
  CODE:
    tally_state_t *state = INT2PTR(tally_state_t *, SvIV(SvRV(self)));
    tally_reset(state);

void
DESTROY(self)
    SV *self
  CODE:
    tally_state_t *state = INT2PTR(tally_state_t *, SvIV(SvRV(self)));
    Safefree(state);

Notice that Tally includes abacus_math.h (the pure C header), not abacus.h (the Perl-facing wrapper). This is intentional - Tally has its own Perl/XS setup and only needs the C functions.

Write the Perl module

Tally/lib/Tally.pm

package Tally;

use 5.008003;
use strict;
use warnings;

our $VERSION = '0.01';

require XSLoader;
XSLoader::load('Tally', $VERSION);

1;

__END__

=head1 NAME

Tally - Running total calculator using Abacus C headers

=head1 SYNOPSIS

    use Tally;

    my $t = Tally->new;
    $t->add(10);        # total is now 10
    $t->add(5);         # total is now 15
    $t->subtract(3);    # total is now 12
    $t->multiply_total(2);  # total is now 24
    say $t->total;      # 24
    $t->reset;          # back to 0

=cut

Write a test

Tally/t/01-basic.t

use strict;
use warnings;
use Test::More;

use_ok('Tally');

my $t = Tally->new;
isa_ok($t, 'Tally');

is($t->total, 0, 'starts at zero');

is($t->add(10), 10, 'add 10');
is($t->add(5),  15, 'add 5');
is($t->subtract(3), 12, 'subtract 3');
is($t->multiply_total(2), 24, 'multiply by 2');
is($t->total, 24, 'total is 24');

$t->reset;
is($t->total, 0, 'reset to zero');

done_testing;

Build Tally (development mode)

cd Tally
perl Makefile.PL     # finds ../Abacus/include automatically
make
make test

I hope you found this tutorial useful! If you have questions about XS, C header reuse, or building modular Perl/C libraries, please leave a message.

Horus, Apophis, and Sekhmet: An C/XS Identifier Stack for Perl

LNATION — Sun, 29 Mar 2026 09:52:09 +0000

Three modules, one goal: fast, correct identifier generation in Perl with zero runtime dependencies. Horus generates UUIDs. Sekhmet generates ULIDs. Apophis uses deterministic UUIDs to build content-addressable storage. All three are implemented in C, exposed through XS, and designed to work together.

Horus: Every UUID Version, One Module

Horus implements all UUID versions defined in RFC 9562 -- v1 through v8, plus NIL and MAX. The entire engine is C, compiled once, called millions of times per second.

use Horus qw(:all);

my $random   = uuid_v4();                         # 122 random bits
my $sortable = uuid_v7();                         # timestamp + random, sortable
my $fixed    = uuid_v5(UUID_NS_DNS, "example.com"); # deterministic, always the same

Why multiple versions matter

Each version solves a different problem:

v4 is the workhorse. 122 bits of randomness, no coordination needed. Use it for session tokens, request IDs, anything where uniqueness is all you need.

v7 embeds a millisecond timestamp in the high bits, making UUIDs lexicographically sortable. Database indexes love this -- new rows append instead of scattering across B-tree pages. Horus guarantees monotonic ordering within the same millisecond.

my @ids = map { uuid_v7() } 1..3;
# 019d38f6-3e9a-765c-ae1c-1cfeb0c30000
# 019d38f6-3e9a-765c-ae1c-1cfeb0c40000
# 019d38f6-3e9a-765c-ae1c-1cfeb0c50000
# String sort == chronological sort

v5 is deterministic. Given the same namespace and name, it always produces the same UUID. This is the foundation Apophis builds on... the same content, the same identifier, every time.

my $a = uuid_v5(UUID_NS_DNS, "example.com");
my $b = uuid_v5(UUID_NS_DNS, "example.com");
# $a eq $b -- always

Ten output formats

Every generator accepts a format parameter. Convert between them freely:

my $id = uuid_v4();

uuid_convert($id, UUID_FMT_STR);      # 550e8400-e29b-41d4-a716-446655440000
uuid_convert($id, UUID_FMT_HEX);      # 550e8400e29b41d4a716446655440000
uuid_convert($id, UUID_FMT_BRACES);   # {550e8400-e29b-41d4-a716-446655440000}
uuid_convert($id, UUID_FMT_URN);      # urn:uuid:550e8400-e29b-41d4-a716-446655440000
uuid_convert($id, UUID_FMT_BASE64);   # VQ6EAOKbQdSnFkRmVUQAAA

Bulk generation

When you need thousands of IDs, crossing the Perl/C boundary once beats crossing it thousands of times:

my @ids = uuid_v4_bulk(10_000);  # single call, 10k UUIDs back

Utilities

uuid_validate($string);          # is this a valid UUID?
uuid_version($string);           # which version? (1-8)
uuid_time($v7_uuid);             # extract epoch seconds from v7/v6
uuid_cmp($a, $b);                # sort comparison (-1, 0, 1)

Sekhmet: ULIDs for When You Need Sortable and Compact

A ULID is 26 characters of Crockford base32 encoding: 10 characters of millisecond timestamp followed by 16 characters of randomness. They sort lexicographically by time, they are URL-safe, and they are shorter than UUIDs.

use Sekhmet qw(:all);

my $ulid = ulid();
# 06EKHXHYKAT25K0YQJHN6A6YJR

Monotonic mode

If you generate multiple ULIDs within the same millisecond, the random component increments to guarantee strict ordering:

my $a = ulid_monotonic();
my $b = ulid_monotonic();
my $c = ulid_monotonic();
# $a lt $b lt $c -- guaranteed, even within the same millisecond

Time extraction

The timestamp is baked into the ULID. Extract it without a database lookup:

my $ulid = ulid();
my $epoch = ulid_time($ulid);       # 1774777155.226
my $ms    = ulid_time_ms($ulid);    # 1774777155226

UUID interoperability

ULIDs and UUID v7 share the same structure -- 48-bit timestamp, random fill. Convert between them losslessly:

my $ulid = ulid();
my $uuid = ulid_to_uuid($ulid);    # standard UUID v7 string
# Useful when your API expects UUIDs but you generate ULIDs internally

When to use Sekhmet vs Horus

Use Sekhmet (ulid()) when you want compact, sortable, human friendly identifiers for log entries, event streams, anything displayed in a UI. Use Horus (uuid_v7()) when you need standard UUID format for compatibility with systems that expect 36-character hyphenated strings. Use Horus
(uuid_v4()) when you need pure randomness with no timestamp leakage.

Apophis: Content-Addressable Storage

Apophis answers the question: "Have I seen this content before?" It hashes content with UUID v5 to produce a deterministic identifier, then stores the content in a sharded directory tree. Same content always maps to the same path. Different content never collides.

use Apophis;

my $store = Apophis->new(
    namespace => 'my-app',
    store_dir => '/var/data/cas',
);

my $id = $store->store(\"Hello, world!");
# 3e856e0f-c7ac-569e-827b-40df723c326f

my $id2 = $store->store(\"Hello, world!");
# 3e856e0f-c7ac-569e-827b-40df723c326f  -- same content, same ID

How storage works

Content is stored in a two-level hex-sharded directory tree derived from the UUID. The first four hex characters become two directory levels:

/var/data/cas/
  3e/85/3e856e0f-c7ac-569e-827b-40df723c326f

This gives 65,536 possible directories -- enough to keep any single directory from growing too large, even with millions of files.

Writes are atomic: content goes to a temporary file first, then is renamed into place. A crash mid-write leaves no partial files.

Identification without storage

Sometimes you just want the identifier:

my $id = $store->identify(\"some content");     # UUID, no write
my $id = $store->identify_file("/path/to/big.iso");  # streams in 64KB chunks

File identification is streaming -- a 10GB file uses the same memory as a 10KB file.

Metadata

Attach arbitrary metadata as a sidecar:

my $id = $store->store(\"image data", meta => {
    mime_type     => 'image/png',
    original_name => 'photo.png',
    uploaded_by   => 'user-42',
});

my $meta = $store->meta($id);
# { mime_type => 'image/png', original_name => 'photo.png', ... }

Namespace isolation

The namespace parameter creates a separate UUID v5 namespace. The same content under different namespaces produces different identifiers:

my $a = Apophis->new(namespace => 'uploads');
my $b = Apophis->new(namespace => 'cache');

$a->identify(\"data") ne $b->identify(\"data");  # different IDs

This lets you run multiple independent stores without collision.

Verification

Content-addressable storage has a built-in integrity check: re-hash the content and compare to the filename.

if ($store->verify($id)) {
    # content matches its identifier -- no corruption
}

How They Fit Together

Horus (Foundation)
  |-- UUID v1-v8, NIL, MAX
  |-- C headers reused by downstream XS modules
  |
  |--- Apophis (Content-addressable storage)
  |      Uses UUID v5 for deterministic content identification
  |
  |--- Sekhmet (ULID generation)
         Uses Horus C primitives for Crockford base32, CSPRNG, timestamps

Horus is the foundation. Its C headers are standalone -- no Perl types, no interpreter context. Apophis and Sekhmet include them at compile time via Horus->include_dir().

A practical example using all three:

use Horus qw(:all);
use Apophis;
use Sekhmet qw(:all);

# Event tracking system
my $event_id  = ulid_monotonic();              # sortable event identifier
my $session   = uuid_v4();                     # random session token
my $store     = Apophis->new(namespace => 'events', store_dir => '/var/events');

# Store event payload, get content-addressable ID
my $payload = encode_json({ action => 'click', target => 'button-1' });
my $content_id = $store->store(\$payload, meta => {
    event_id   => $event_id,
    session_id => $session,
    timestamp  => ulid_time($event_id),
});

# Later: retrieve by content hash
my $data = $store->fetch($content_id);

# Or find when the event happened from the ULID
my $when = ulid_time($event_id);

Each module handles one concern well. Horus generates identifiers. Sekhmet adds time sortable compact identifiers. Apophis maps content to identifiers and manages storage. No module tries to do what another already does.

Performance

All three modules use custom ops on Perl 5.14+ to eliminate subroutine dispatch overhead. The hot paths are pure C with no Perl API calls.

Getting Started

cpanm Horus
cpanm Sekhmet
cpanm Apophis

All three are on CPAN under the
Artistic License 2.0.

Eshu: Indentation Fixer for Eight Languages, Written in C

LNATION — Sun, 29 Mar 2026 07:03:48 +0000

Most code formatters want to own your style. They have opinions about brace placement, line length, trailing commas, and a hundred other things you never asked for. Sometimes you just want the indentation fixed. Tabs consistent, nesting correct, content untouched.

That is was the goal for Eshu and exactly what it does. It reads source code line by line, tracks nesting depth through a state machine, rewrites the leading whitespace, and leaves everything else alone. The entire engine is written in C, exposed to Perl through XS, and ships with a CLI that can check, diff, or fix files in place. The distribution also ships with a vim plugin as that is my editor of choice.

Eight languages, one tool

C | Perl | XS | XML | HTML | CSS | JavaScript | POD

Each language gets its own scanner with its own state machine. They share a common architecture which track depth, emit indentation and advance but each state machine actually handles the constructs that make the specific language awkward to indent.

Perl has heredocs, quoted constructs (qw(), qq{}, s///), and embedded
POD. JavaScript has template literals with nested ${} interpolation. XS
files switch between C and Perl conventions at the MODULE = boundary. HTML
has void elements and verbatim zones inside <pre> and <script>. Each of these needs specific handling, and getting any of them wrong means corrupting content visually.

Why C?

Indentation fixing is embarrassingly linear. You read a line, update state, emit the line with new leading whitespace, repeat. There is no tree to build, no AST to walk, no multipass resolution. A single-pass scanner in C processes a large codebase in milliseconds.

The engine is implemented entirely in standalone C header files -- ten of them, roughly 3,600 lines total. They have no Perl dependencies. No SV*, no croak(), no interpreter context. Just stdlib.h, string.h, and ctype.h. This means they can be reused from any C program or language that can bind them, not just my Perl XS modules.

include/
  eshu.h              Core types, config, buffer, language enum
  eshu_c.h            C scanner
  eshu_pl.h           Perl scanner (heredoc, regex, qw, POD)
  eshu_xs.h           XS dual-mode scanner
  eshu_xml.h          XML/HTML scanner
  eshu_css.h          CSS scanner
  eshu_js.h           JavaScript scanner (template literals)
  eshu_pod.h          POD scanner
  eshu_file.h         File I/O, directory walking, binary detection
  eshu_diff.h         Unified diff generation

How the scanner works

Every language scanner follows the same pattern. For each line of input:

pre-adjust depth --> emit indent --> copy content --> post-adjust depth

A closing brace on a line means you dedent before emitting that line. An opening brace means you indent the next line. The scanner maintains a state enum to know whether it is inside a string, a comment, a heredoc, a regex, or regular code. State transitions happen character by character within the scan function; depth changes happen at line boundaries.

The Perl scanner, for example, tracks 14 distinct states: regular code, double-quoted strings, single-quoted strings, regex, heredoc (both standard and indented), qw, qq, q, POD, line comments, and block comments. It detects heredoc terminators, remembers whether the variant is indented (<<~EOF), buffers the body verbatim, and resumes normal scanning after the terminator.

The hard parts

Perl: Is `/` division or regex?

The classic Perl parsing problem. Eshu tracks whether the previous meaningful token was a value (a variable, a closing bracket, a number) or an operator. If it was a value, / is division. Otherwise, it opens a regex. This is the same heuristic that syntax highlighters use, and it covers real-world code well.

XS: Two languages in one file

An XS file is C code at the top and a Perl/C hybrid below MODULE =. Eshu detects the boundary and switches scanners. Below the boundary, it tracks XSUB blocks (each new function declaration resets depth), labels like CODE:, OUTPUT:, and INIT:, and special cases like BOOT: sections that use shallower indentation.

JavaScript: Template literals with nested interpolation

A backtick string in JavaScript can contain ${expr}, and that expression can contain braces, function calls, even another template literal. Eshu maintains a depth counter for interpolation braces so it knows when the } closes the interpolation versus when it closes a block inside the interpolation.

HTML: Script blocks need JavaScript rules

Content inside <script> tags is JavaScript, not HTML. Eshu collects the entire script block, passes it through the JavaScript scanner for reindentation, then splices it back at the correct HTML depth. The same applies to recognising void elements (<br>, <img>, etc.) that should not increase nesting depth.

The CLI

Eshu ships with a command-line tool that supports the three modes you actually need:

# Preview what would change
eshu --diff lib/

# Check in CI (exit 1 if anything needs fixing)
eshu --check lib/ t/

# Fix in place
eshu --fix lib/

By default, nothing is modified. You have to explicitly ask for --fix. Language is detected from file extensions, but can be overridden with --lang. You can filter files with --exclude and --include (regex patterns), choose tabs or spaces, set the indent width, and even restrict processing to a line range within a file.

Directory processing is recursive by default, skips binary files (detected by sampling the first 8KB for NUL bytes), respects a 1MB size limit, and follows file symlinks but not directory symlinks.

The Perl API

Everything the CLI does is available programmatically:

use Eshu;

# Fix a string
my $fixed = Eshu->indent_pl($source, spaces => 4);

# Auto-detect language and process
my $result = Eshu->indent_file('lib/App.pm',
    fix  => 1,
    diff => 1,
);
say $result->{diff} if $result->{status} eq 'changed';

# Process an entire directory
my $report = Eshu->indent_dir('lib/',
    fix       => 1,
    recursive => 1,
    exclude   => [qr/\.bak$/],
);
say "$report->{files_changed} files fixed";

Each language also has a direct method: indent_c, indent_xs, indent_xml, indent_html, indent_css, indent_js, indent_pod.

Idempotent by design

Running Eshu twice produces the same result as running it once. This is not just a goal, it is tested. The test suite includes real-world Perl example, verifies that processing them does not crash, and asserts that a second pass produces identical output.

This matters for CI integration. If eshu --check passes, you know that running eshu --fix would be a no-op. There is no oscillation, no cascading reformats, no "fix the fix" loops.

What it does not do

Eshu does not reformat code. It does not move braces, break long lines, sort imports, add or remove semicolons, or have opinions about blank lines. It touches leading whitespace and nothing else. Diffs are clean: every changed line shows only the whitespace prefix changing.

This is a deliberate constraint. A tool that only fixes indentation is a tool you can run on any codebase without fear. It will not start a style war. It will not produce a 10,000 line diff that buries your actual changes. It will make the nesting visible and get out of the way.

Vim integration

Eshu ships with a Vim plugin in the distribution. It pipes the current buffer through eshu and replaces the content in place, with automatic language detection and cursor position preservation.

Installation

The easiest way with Vim 8+ native packages:

mkdir -p ~/.vim/pack/eshu/start
ln -s /path/to/Eshu/vim ~/.vim/pack/eshu/start/eshu

For Neovim:

mkdir -p ~/.local/share/nvim/site/pack/eshu/start
ln -s /path/to/Eshu/vim ~/.local/share/nvim/site/pack/eshu/start/eshu

Or if you prefer vim-plug:

Plug '/path/to/Eshu/vim'

Or just source it directly in your .vimrc:

source /path/to/Eshu/vim/plugin/eshu.vim

Usage

Once loaded, you get two commands and a default keybinding:

Command	Mode	What it does
`:EshuFix`	Normal	Fix indentation for the entire file
`:EshuFixRange`	Visual	Fix indentation for the selected lines
`\ef`	Normal	Fix entire file (default `<Leader>ef` mapping)
`\ef`	Visual	Fix selected lines (default `<Leader>ef` mapping)

The plugin detects the language from the file extension -- .pm and .pl map to Perl, .xs to XS, .html to HTML, and so on. If the eshu binary is not in your $PATH, point the plugin at it:

let g:eshu_cmd = '/path/to/Eshu/bin/eshu'

To disable the default mappings and use your own:

let g:eshu_no_mappings = 1
nnoremap <silent> <F6> :EshuFix<CR>
vnoremap <silent> <F6> :EshuFixRange<CR>

Eshu is available on CPAN under the Artistic License 2.0.

Ready, Set, Compile... you slow Camel

LNATION — Sun, 25 Jan 2026 13:47:26 +0000

"Perl is slow."

I've heard this for years, well since I started. You probably have too. And honestly? For a long time, I didn't have a great rebuttal. Sure, Perl's fast enough for most things, it's well known for text processing, glueing code and quick scripts. But when it came to object heavy code, the critics have a point.

We will begin by looking at the myth of perl being slow a little more deeply. Here's a benchmark between Perl and Python using CPU seconds, a fair comparison that measures actual work done:

=== PERL (5 CPU seconds per test) ===
Integer arithmetic             1,072,800/s
Float arithmetic                 398,800/s
String concat                    970,000/s
Array push/iterate               368,800/s
Hash insert/iterate               84,800/s
Function calls                   244,000/s
Regex match                   12,921,200/s

=== PYTHON (5 CPU seconds per test) ===
Integer arithmetic               777,200/s
Float arithmetic                 512,400/s
String concat                    627,200/s
List append/iterate              476,400/s
Dict insert/iterate              140,600/s
Function calls                   331,400/s
Regex match                   10,543,713/s

The results are more nuanced than the "Perl is slow" narrative suggests:

Operation	Winner	Margin
Integer arithmetic	Perl	1.4x faster
Float arithmetic	Python	1.3x faster
String concat	Perl	1.5x faster
Array/List ops	Python	1.3x faster
Hash/Dict ops	Python	1.7x faster
Function calls	Python	1.4x faster
Regex match	Perl	1.2x faster

Perl wins at what it's always been good at: integers, strings, and regex. Python wins at floats, data structures, and function calls areas where I am told Python 3.x has seen heavy optimisation work.

But here's the thing that surprised me: neither language is dramatically faster than the other for basic operations. The differences are measured in fractions, not orders of magnitude. So where does the "Perl is slow" reputation actually come from?

Object-oriented code. Let's run that same fair comparison:

=== Object creation + 2 method calls (5M iterations) ===
Perl bless:    4,155,178/s  (1.20 sec)
Python class:  5,781,818/s  (0.86 sec)

Okay, this is not so bad. Perl's only 40% behind. But now let's look at what people actually use these days: Moo.

=== Object creation + 2 method calls (5M iterations) ===
Perl bless:    4,176,222/s  (1.20 sec)
Moo class:       843,708/s  (5.93 sec)
Python class:  5,590,052/s  (0.89 sec)

Wait, what? Moo is 6.6x slower than Python. And it's 5x slower than plain bless.

This is layered with actual business logic is I guess where "Perl is slow" actually comes from. This all comes down to layers. Every Moo accessor has been optimised but if you have all features you build a call stack, each adding overhead:

$obj->name
  └─> accessor method (generated sub)
        └─> type constraint check
              └─> coercion check
                    └─> trigger check
                          └─> lazy builder check
                                └─> finally: $self->{name}

Each of those subroutine calls means:

Push arguments onto the stack (~3-5 ops)
Create a new scope (localizing variables)
Execute the check (even if it's just "return true")
Pop the stack and return (~3-5 ops)

Even a "simple" Moo accessor with just a type constraint involves roughly 30+ additional operations compared to a plain hash access. The type constraint alone might call:

has_type_constraint() - is there a constraint?
type_constraint() - get the constraint object
check() - call the constraint's check method
The actual validation logic

Multiply that by two accessors per iteration, five million iterations, and suddenly you're spending 5 seconds instead of 1.

This is the trade off Moo makes: flexibility and safety for speed. And for most applications, it's the right trade off and even in python they do this with what they call pydantic that halfs the performance of python objects.

I've spent more time than I'd care to admit thinking about this question. Not in a "let's rewrite everything in Rust" kind of way, but genuinely asking: what would it take to make Perl's object system competitive with languages people actually consider fast?

The answer, it turns out, was inside a CPAN module first released on 'Mon Jul 24 11:23:25 2000'. This was highlighted to me by another works who I am indeed one of the three people who do not only read their blogs but also often finds themselves lost within their interesting coding patterns.

So this is the story of the four modules that changed how I think about Perl performance: Marlin, Meow, Inline and XS::JIT. They're different tools with different philosophies, but together they represent something I never quite expected to see Perl object access that's actually faster than Python's equivalent. Not "almost as fast." Faster.

The Marlin story: A faster fish in the Moose family

If you've written any serious Perl in the last fifteen years, you've probably used Moose. Or Moo. Or Mouse. The naming convention is... well, it's a thing we do now.

Marlin fits right into that tradition, and the name's not accidental. Marlins are among the fastest fish in the ocean. That's the pitch: everything you love about Moose-style OO, but with speed as a first-class concern.

Toby Inkster released Marlin in late 2025, and it caught my attention as I stated before, many of his projects do. I'd previously attempted to write a fast OO system myself (Meow), but was struggling to even compete with Moo despite being entirely XS. Partly ability, partly still learning, mostly not being in the right compile time stage.

With my interest piqued, I installed Marlin, played with the API, and ran some benchmarks:

Benchmark: 1,000,000 iterations
            Rate   Meow    Moo Marlin  Mouse
Meow    606,061/s     --    -1%   -45%   -47%
Moo     609,756/s     1%     --   -45%   -46%
Marlin 1,098,901/s    81%    80%     --    -3%
Mouse  1,136,364/s    87%    86%     3%     --

Marlin performed well. Meow at that point was... not impressive. But I liked Marlin's API and, understanding my own implementation's limitations, I was satisfied enough with the speed to build my Claude modules around it, while also understanding it would likely improve in performance.

A few weeks later, and a lot happened in between, but on Friday evening I randomly decided to revisit my Meow directory. Could I fix some of the flaws based upon my recent learnings? I managed to, and saw a huge improvement in my own benchmarks. So I updated to the latest Marlin for a fair comparison.

I was expecting Meow to be faster now since I'm doing much less in this minimalist approach. But what I actually found surprised me:

Benchmark: 10,000,000 iterations
            Rate    Moo  Mouse   Meow Marlin
Moo     868,810/s     --   -47%   -60%   -81%
Mouse  1,626,016/s    87%     --   -26%   -64%
Meow   2,183,406/s   151%    34%     --   -52%
Marlin 4,504,505/s   418%   177%   106%     --

Marlin had gotten dramatically faster, over 4x improvement from the version I'd first tested. Toby had clearly been busy. And while Meow had improved too, it was still only half of Marlin's speed.

This was the moment that changed everything. I needed to understand how Marlin achieved this. What was I missing?

Just in time optimisation

As I mentioned, I read other people's code. I read Toby's posts on Marlin and how he'd studied Mouse's optimisation strategy: only validate when you absolutely need to. But when I started tracing through Marlin's actual implementation, something clicked.

The key insight is in Marlin::Attribute::install_accessors. Here's what happens when Marlin sets up a reader:

if ( $type eq 'reader' and !$me->has_simple_reader and $me->xs_reader ) {
    $me->{_implementation}{$me->{$type}} = 'CXSR';  # Class::XSReader
}
elsif ( HAS_CXSA and $me->has_simple_reader ) {
    # Use Class::XSAccessor for simple cases
    Class::XSAccessor->import( class => $me->{package}, ... );
}

Marlin makes a compile-time decision: what kind of accessor does this attribute actually need?

Simple getter (no default, no lazy, no type check on read)? → Use Class::XSAccessor, which is pure XS and blindingly fast
Getter with lazy default or type coercion? → Use Class::XSReader, which handles the complexity in optimised C
Something exotic (auto_deref, custom behaviour)? → Fall back to generated Perl

This is the magic. Most Moo-style accessors go through a generic code path that handles every possible feature, even features you're not using. Marlin analyses your attribute definition at compile time and generates the minimal accessor that satisfies your requirements.

Consider a read-only attribute with a type but no default:

# Moo accessor path:
$obj->name
  → check if lazy builder needed     # nope, but we still check
  → check if default needed          # nope, but we still check  
  → check if coercion needed         # nope, but we still check
  → finally: $self->{name}

# Marlin accessor (Class::XSAccessor):
$obj->name
  → $self->{name}                    # that's it. One XS call.

The type constraint? Marlin validates it in the constructor, not the getter. Once an object is built, reading an attribute is just a hash lookup: no validation, no subroutine calls, no stack manipulation.

This is why Marlin went from 1.1M ops/sec to 4.5M ops/sec between versions. Toby wasn't just optimising code. He was eliminating entire categories of runtime work by moving decisions to compile time.

A different approach is used forClass::XSConstructor. This reuses a generic XSUB but passes the class data via a custom pointer. This sub is then optimised to not need to reach back into perl for stash, hv lookups etc.

It's JIT compilation, but done at module load time rather than runtime. By the time your code calls ->new or ->name, all the decisions have been made. All that's left is the actual work.

This was my revelation: the path to fast Perl OO isn't avoiding features, it's avoiding runtime feature detection. Know what you need at compile time, generate optimised code for exactly that, and get out of the way.

Now the question became: could I apply this same principle to Meow? It was already setup to build a simple hash that represented the object, I had what I needed but I wanted to do this in a backwards compatible way.

Enter Inline::C

Armed with the understanding of why Marlin was fast, I had a hypothesis: if I could generate XS accessors at compile time tailored to each attribute's needs, Meow could achieve the same performance.

I needed to generate custom C code and then execute it, well for perl that was written by Ingy döt Net back in 2000 the Inline::C.

The idea was simple: when Meow sees ro name => Str, it should generate C code for an accessor that:

Takes the object
Returns the value at the slot index for name
That's it. No method dispatch, no type checking, no feature checking.

I didn't want to just break everything so I leaned into the Moose catalog and added a make_immutable phase. When this is called it would compile the C code needed to generate an optimised XS package and this was fed into Inline::C. The first run would compile; subsequent runs would use the cached .so.

And it worked. I had to change the benchmark to CPU to get a fair result but I've also included a Cor test here which does not have type checking like Marlin or Meow.

Benchmark: running Cor, Marlin, Meow for at least 5 CPU seconds...
       Cor:  5 wallclock secs ( 5.13 usr +  0.02 sys =  5.15 CPU) @ 2,886,788/s
    Marlin:  5 wallclock secs ( 5.01 usr +  0.11 sys =  5.12 CPU) @ 4,523,074/s
      Meow:  5 wallclock secs ( 5.16 usr +  0.02 sys =  5.18 CPU) @ 4,558,344/s

As you can see Meow had caught Marlin. Actually, it was slightly faster, 4.56M vs 4.52M ops/sec, but this would be expected as Meow does ALOT less than Marlin.

But my bottlekneck was now in Inline::C and well nobody wants to write C/XS let alone concatenate it.

Startup overhead: First compilation was slow, several seconds for a complex class
Dependencies: Inline::C pulls in Parse::RecDescent, adds complexity to the dependency chain
Build process: It generates a full Makefile.PL and runs the ExtUtils::MakeMaker machinery
Caching: The caching mechanism is designed for "write once" scripts, not dynamic code generation

For a proof of concept, Inline::C was perfect. But for a production module, I needed something leaner. That's when I started looking at what Inline::C actually does under the hood, and wondering how much of it I could strip away.

Under the hood: XS::JIT as the secret weapon

Inline::C proved the concept worked, but it came with baggage. Every compile spawned a full Makefile.PL build process. Dependencies bloated the install. And the caching system, designed for write-once scripts, wasn't ideal for dynamic code generation.

So I started picking apart what Inline::C actually does:

Parse C code to find function signatures
Generate XS wrapper code
Generate a Makefile.PL
Run perl Makefile.PL && make
Load the resulting .so

And yes, this happens even when you use bind Inline C => ... instead of the use form. The bind keyword just defers compilation to runtime rather than compile time. It doesn't change what gets done, only when. You still get the full Parse::RecDescent parsing, the xsubpp processing, the MakeMaker dance. The only difference is whether it happens at use time or when bind is called.

Most of this was unnecessary for my use case. I didn't need function parsing, I already knew what functions I was generating. I didn't need XS wrappers, I was writing XS-native code directly. And I definitely didn't need the Makefile.PL dance.

XS::JIT strips all of that away. It's a single-purpose tool: take C code, compile it, load it, install the functions. No parsing. No xsubpp. No make. Direct compiler invocation.

Here's what the C API looks like:

#include "xs_jit.h"

/* Function mapping - where to install what */
XS_JIT_Func funcs[] = {
    { "Cat::new",  "cat_new",  0, 1 },  /* target, source, varargs, xs_native */
    { "Cat::name", "cat_name", 0, 1 },
    { "Cat::age",  "cat_age",  0, 1 },
};

/* Compile and install in one call */
int ok = xs_jit_compile(aTHX_
    c_code,           /* Your generated C code */
    "Meow::JIT::Cat", /* Unique name for caching */
    funcs,            /* Function mapping array */
    3,                /* Number of functions */
    "_CACHED_XS",     /* Cache directory */
    0                 /* Don't force recompile */
);

That's it. One function call. The first time it runs, XS::JIT:

Generates a boot function that registers all the XS functions
Compiles directly with the system compiler (cc -shared -fPIC ...)
Loads the .so with DynaLoader
Installs each function into its target namespace

Subsequent runs? It hashes the C code, finds the cached .so, and just loads it. The compile step vanishes entirely.

The key insight is the is_xs_native flag. When set, XS::JIT creates a simple alias: no wrapper function, no stack manipulation, no overhead. Your C function is the XS function:

XS_EUPXS(cat_name) {
    dVAR; dXSARGS;
    SV *self = ST(0);
    AV *av = (AV*)SvRV(self);
    SV **slot = av_fetch(av, 0, 0);  /* slot 0 = name */
    ST(0) = slot ? *slot : &PL_sv_undef;
    XSRETURN(1);
}

No wrapper. No intermediate calls.

This is exactly what Meow needed. During make_immutable, it:

Analyses each attribute's requirements (type constraint? coercion? trigger?)
Generates minimal XS accessor code for each one
Generates an optimised XS constructor that handles all attributes in one pass
Hands the code to XS::JIT for compilation
Gets back installed functions ready to call

The entire JIT compilation happens once per class, at module load time. By the time your code runs, everything is native XS.

Comparing the approaches

Here's what actually happens at runtime for each framework:

Moo accessor call:

$obj->name
  → Perl method dispatch
    → Generated Perl subroutine
      → has_type_constraint() check
        → type_constraint() fetch
          → check() call
            → finally: $self->{name}

Stack frames: 4-6. Operations: ~30.

Marlin accessor call (Class::XSAccessor):

$obj->name
  → Perl method dispatch
    → XS accessor
      → $self->{name}

Stack frames: 1. Operations: ~5.

Note: Toby has some slot magic also

Meow accessor call (XS::JIT):

$obj->name
  → Perl method dispatch
    → XS accessor
      → $self->[SLOT_INDEX]

Stack frames: 1. Operations: ~4 (arrays are slightly faster than hashes).

The benchmark results

With XS::JIT in place, here's where Meow now landed:

Benchmark: running Cor, Marlin for at least 5 CPU seconds... Marlin and Meow has type constraint checking
       Cor:  5 wallclock secs ( 5.13 usr +  0.02 sys =  5.15 CPU) @ 2886788.16/s (n=14866959)
    Marlin:  5 wallclock secs ( 5.01 usr +  0.11 sys =  5.12 CPU) @ 4523074.80/s (n=23158143)
      Meow:  5 wallclock secs ( 5.16 usr + -0.01 sys =  5.15 CPU) @ 5196218.06/s (n=26760523)
Benchmark: running Marlin, Meow, Moo, Mouse for at least 5 CPU seconds...
    Marlin:  5 wallclock secs ( 5.22 usr +  0.13 sys =  5.35 CPU) @ 4814728.04/s (n=25758795)
      Meow:  5 wallclock secs ( 5.23 usr +  0.01 sys =  5.24 CPU) @ 5203329.96/s (n=27265449)
       Moo:  4 wallclock secs ( 5.28 usr +  0.00 sys =  5.28 CPU) @ 860649.81/s (n=4544231)
     Mouse:  6 wallclock secs ( 5.29 usr +  0.01 sys =  5.30 CPU) @ 1603849.25/s (n=8500401)
            Rate    Moo  Mouse Marlin   Meow
Moo     860650/s     --   -46%   -82%   -83%
Mouse  1603849/s    86%     --   -67%   -69%
Marlin 4814728/s   459%   200%     --    -7%
Meow   5203330/s   505%   224%     8%     --

I must be honest, around this time I had not implemented the full benchmarks against Perl and Python. I didn't fully understand the difference, so I had some thoughts that I was hitting limitations with my own hardware (it was late, or early in the morning). Anyway, I kept pushing and ran a benchmark where I accessed the slot directly as an array reference. This got me excited:

Meow (direct) 7,172,481/s     778%    347%     50%     14%

I was seeing a huge improvement. I spent some time making an API that was a little nicer by exposing constants as slot indexes:

{
    package Cat 
    use Meow;
    ro name => Str;
    ro age => Int;
    make_immutable;  # Creates $Cat::NAME, $Cat::AGE
}

# Direct slot access
my $name = $cat->[$Cat::NAME];

I was now on par with Python, but I wanted more. There had to be a way to get that array access without the ugly syntax.

So I dug deeper into Perl's internals and found the missing magic: cv_set_call_checker and custom ops.

The entersub bypass: Custom ops

Here's what normally happens when you call a method in Perl:

name($cat)
  → OP_ENTERSUB (the "call function" op)
    → Push arguments onto stack
    → Look up the CV (code value)
    → Set up new stack frame
    → Execute the XS function
    → Pop stack frame
    → Return

Even for our minimal XS accessor, there's overhead: the entersub op itself, the stack frame setup, the CV lookup. What if we could eliminate all of that?

Perl provides a hook called cv_set_call_checker. It allows you to register a "call checker" function that runs at compile time when the parser sees a call to your subroutine. The checker can inspect the op tree and crucially replace it with something else entirely.

Here's what Meow does:

static void _register_inline_accessor(pTHX_ CV *cv, IV slot_index, int is_ro) {
    SV *ckobj = newSViv(slot_index);  /* Store slot index for later */
    cv_set_call_checker_flags(cv, S_ck_meow_get, ckobj, 0);
}

When the checker sees name($cat), it:

Extracts the $cat argument from the op tree
Frees the entire entersub operation
Creates a new custom op with the slot index baked in
Returns that instead

The custom op is trivially simple:

static OP *S_pp_meow_get(pTHX) {
    dSP;
    SV *self = TOPs;
    PADOFFSET slot_index = PL_op->op_targ;  /* Baked into the op */

    SV **ary = AvARRAY((AV*)SvRV(self));
    SETs(ary[slot_index] ? ary[slot_index] : &PL_sv_undef);

    return NORMAL;
}

That's the entire accessor. No function call. No stack frame. No CV lookup. The slot index is embedded directly in the op structure. The Perl runloop executes this op directly, it's as close to $cat->[$NAME] as you can get while still looking like name($cat).

This is the same technique that builtin::true and builtin::false use in Perl 5.36+. It's also how List::Util::first can be optimised when given a simple block.

The final benchmark

With custom ops in place via import_accessors, here's how the Perl OO frameworks compare:

Benchmark: running Marlin, Meow, Meow (direct), Meow (op), Moo, Mouse for at least 5 CPU seconds...
    Marlin:  6 wallclock secs ( 5.09 usr +  0.11 sys =  5.20 CPU) @ 4766685.58/s (n=24786765)
      Meow:  5 wallclock secs ( 5.29 usr +  0.01 sys =  5.30 CPU) @ 6289606.79/s (n=33334916)
Meow (direct):  5 wallclock secs ( 5.32 usr +  0.01 sys =  5.33 CPU) @ 7172480.86/s (n=38229323)
 Meow (op):  5 wallclock secs ( 5.16 usr +  0.01 sys =  5.17 CPU) @ 7394453.19/s (n=38229323)
       Moo:  4 wallclock secs ( 5.44 usr +  0.02 sys =  5.46 CPU) @ 816865.93/s (n=4460088)
     Mouse:  4 wallclock secs ( 5.18 usr +  0.01 sys =  5.19 CPU) @ 1605727.55/s (n=8333726)
                   Rate      Moo   Mouse  Marlin    Meow Meow (direct) Meow (op)
Moo            816866/s       --    -49%    -83%    -87%          -89%      -89%
Mouse         1605728/s      97%      --    -66%    -74%          -78%      -78%
Marlin        4766686/s     484%    197%      --    -24%          -34%      -36%
Meow          6289607/s     670%    292%     32%      --          -12%      -15%
Meow (direct) 7172481/s     778%    347%     50%     14%            --       -3%
Meow (op)     7394453/s     805%    361%     55%     18%            3%        --

Now lets test that directly against python:

============================================================
Python Direct Benchmark (slots + property accessors)
============================================================
Python version: 3.9.6 (default, Dec  2 2025, 07:27:58)
[Clang 17.0.0 (clang-1700.6.3.2)]
Iterations: 5,000,000
Runs: 5
------------------------------------------------------------
Run 1: 0.649s (7,704,306/s)
Run 2: 0.647s (7,733,902/s)
Run 3: 0.646s (7,736,307/s)
Run 4: 0.648s (7,720,909/s)
Run 5: 0.649s (7,702,520/s)
------------------------------------------------------------
Median rate: 7,720,909/s
============================================================
============================================================
Perl/Meow Benchmark Comparison
============================================================
Perl version: 5.042000
Iterations: 5000000
Runs: 5
------------------------------------------------------------
Inline Op (one($foo)):
  Run 1: 0.638s (7,841,811/s)
  Run 2: 0.629s (7,954,031/s)
  Run 3: 0.631s (7,929,850/s)
  Run 4: 0.631s (7,926,316/s)
  Run 5: 0.633s (7,901,675/s)
  Median: 7,926,316/s
============================================================
Summary:
------------------------------------------------------------
  Inline Op:    7,926,316/s
============================================================

Conclusion: Why JIT might be the right approach

Looking back at this journey, a pattern emerges. The fastest code isn't the cleverest code. It's the code that does the least work at runtime.

Moo is slow because of the abstraction.

Marlin proved that you could have Moo's features without Moo's overhead by making smart choices at compile time. If an accessor doesn't need lazy building, don't generate code that checks for lazy building.

Meow pushed this further: if you're going to generate code at compile time anyway, why not generate exactly the code you need? Not a generic accessor that handles many cases, but a specific accessor for this specific attribute on this specific class.

And XS::JIT made that practical. Without a lightweight JIT compiler, dynamic XS generation would require shipping a C toolchain with every module, or adding multi-megabyte dependencies. XS::JIT strips the problem down to its essence: take C code, compile it, load it.

The result is object access that competes with, and sometimes beats, languages that have had decades of optimisation work. Not because Perl's interpreter got faster, but because we stopped asking it to do unnecessary work.

Is this approach right for every project? No. Most applications don't need 7 million object accesses per second.

But for the times when performance matters (hot loops, high-frequency trading, real-time systems) it's good to know the ceiling isn't as low as we thought. Perl can be fast. We just needed to get out of its way.

The modules discussed in this post:

Marlin: https://metacpan.org/pod/Marlin
Meow: https://metacpan.org/pod/Meow
XS::JIT: https://metacpan.org/pod/XS::JIT
Inline::C: https://metacpan.org/pod/Inline::C

Doubly: Why Arrays Aren't Always Enough

LNATION — Sun, 18 Jan 2026 18:28:08 +0000

As most of you reading this will know, Perl has three core data types: scalars, arrays, and hashes. They're relatively fast, they're familiar, and for most tasks, they just work. But sometimes you need something more.

A while back, I started at a new company and had a conversation with a colleague. They'd been working on a certain part of the codebase where the bottleneck was a pure Perl implementation of a doubly linked list. I knew of the concept and would usually just use an array for this task. They were adamant this could not work with an array though, so I inspected the code to understand the use case.

My initial gut instinct and others on IRC after I had published the first version, was why not just use an array, well.. if you take this example

my @playlist = ('song_a', 'song_b', 'song_c');
my $current = 1;  # pointing at 'song_b'

# Move forward
$current++ if $current < $#playlist;

# Insert after current position? Now things get messy.
splice(@playlist, $current + 1, 0, 'new_song');

Every insertion is O(n) because Perl has to shift elements. Your index can become stale if you're not careful. And if you're working with threads or nested data? Good luck keeping that index synchronised. Also just wrapping this in a module does not work the moment you add insertion.

This launched me on a journey that would eventually produce four different doubly linked list implementations, each one teaching me something new about Perl, XS, C, and the fascinating trade offs between simplicity, speed, and safety. This journey spanned several months and it was 'Claude' opus who helped me come to the final solution which is thread safe and has no memory leakage.

Cursor Based Navigation vs Indexed Access

Before diving into the advantages, let's establish what a doubly linked list actually is. Unlike an array where elements sit in contiguous memory slots accessed by index, a doubly linked list is a chain of nodes. Each node contains three things: your data, a pointer to the next node, and a pointer to the previous node. This bidirectional linking is what puts the "doubly" in doubly linked list.

┌──────────┐    ┌──────────┐    ┌──────────┐
│   prev ←─┼────┼─ prev ←──┼────┼─ prev    │
│  'intro' │    │  'verse' │    │ 'chorus' │
│   next ──┼────┼→ next ───┼────┼→ next    │
└──────────┘    └──────────┘    └──────────┘

Beyond the basic node structure, a well designed doubly linked list maintains a current position—an internal cursor that tracks where you are in the list. This changes everything about how you interact with your data.

Navigation methods like next(), prev(), start(), and end() move this internal cursor, and data() operates on whatever node you're currently pointing at.

Here's what that looks like in practice:

use Doubly;

my $playlist = Doubly->new();
$playlist->add('intro.mp3')
         ->add('verse.mp3')
         ->add('chorus.mp3')
         ->add('outro.mp3');

# Navigate with the cursor
$playlist->start();           # cursor at 'intro.mp3'
$playlist->next();            # cursor at 'verse.mp3'
print $playlist->data();      # prints 'verse.mp3'

$playlist->next()->next();    # method chaining! cursor at 'outro.mp3'
$playlist->prev();            # back to 'chorus.mp3'

Compare this to the array approach where you're juggling data and position separately:

my @playlist = ('intro.mp3', 'verse.mp3', 'chorus.mp3', 'outro.mp3');
my $pos = 0;

$pos++;
print $playlist[$pos];  # 'verse.mp3'

$pos += 2;
$pos--;

The array version works, but you're managing two things instead of one. The index is disconnected from the data. Pass that array to a function and the position doesn't follow, you'd need to pass both.

But here's where it gets really interesting: nested lists with shared position tracking.

When you store a Doubly list inside another Doubly list, something magical happens. The inner list maintains its own cursor, and every time you access it, you get the same object with its position preserved:

my $outer = Doubly->new();
my $inner = Doubly->new();

$inner->add('a')->add('b')->add('c');
$outer->add($inner);

$outer->start();
my $nested = $outer->data();    # get the inner list
$nested->start()->next();        # move inner cursor to 'b'

# Later...
my $same_nested = $outer->data();  # same object!
print $same_nested->data();         # still at 'b'!

This shared position tracking is incredibly powerful for hierarchical data. Think of a file browser where each directory remembers which file you last selected, or a document editor where each section remembers your scroll position.

The cursor based model also makes certain algorithms beautifully clean. Need to process items around your current position? No index arithmetic required:

# Insert after current position - O(1)!
$list->insert_after('new_item');

# Remove current and move to next
$list->remove();

# Check boundaries naturally
while (!$list->is_end()) {
    process($list->data());
    $list->next();
}

With arrays, insert_after at position n means splice(@arr, $n + 1, 0, $item)—and that's an O(n) operation that shifts every subsequent element. The doubly linked list just adjusts a few pointers.

Also with a doubly linked list when you attain a reference to the data, that reference remains the same. If you were to use a plain perl object to track your state that reference will either change on iteration or become stale if you clone.

This cursor based navigation is what makes doubly linked lists shine for the use cases my colleague was struggling with. It's not just about performance—it's about expressing your intent clearly and letting the data structure handle the bookkeeping.

From Hash Nodes to C Registries

Now let's look under the hood at how each implementation actually works.

Doubly::Linked::PP — Pure Perl Simplicity

The pure Perl implementation is the most transparent. Each node is a blessed hash reference with three keys:

my $node = bless {
    data => 'my_value',
    next => $next_node,  # reference to next node (or undef)
    prev => $prev_node,  # reference to previous node (or undef)
}, 'Doubly::Linked::PP';

The list object itself tracks head, tail, current. Every operation is pure Perl hash manipulation:

sub next {
    my ($self) = @_;
    if ($self->{current} && $self->{current}->{next}) {
        $self->{current} = $self->{current}->{next};
    }
    return $self;
}

This approach has huge advantages: no compilation required, fully debuggable with Data::Dumper, and runs anywhere Perl runs. You can literally inspect the entire structure at any time:

use Data::Dumper;
print Dumper($list);  # see everything!

The downside? At 769 ops/sec, it's not winning any races. Every method call goes through Perl's method dispatch. Every hash access is a hash lookup. And there's a subtler problem: circular references. Node A's next points to Node B, and Node B's prev points back to Node A. Perl's reference counting garbage collector can't automatically clean these up—you need to manually break the cycle or use weak references, otherwise you're leaking memory. The overhead adds up fast.

Doubly::Linked — XS with Perl Structures

The next step was Doubly::Linked, which uses XS (Perl's interface to C) but still constructs Perl hash structures. The C code builds the same {data, next, prev} hashes, just faster:

// Simplified from Doubly::Linked XS
HV* node = newHV();
hv_store(node, "data", 4, newSVpv(data, 0), 0);
hv_store(node, "next", 4, next_sv, 0);
hv_store(node, "prev", 4, prev_sv, 0);

This gives you compiled C speed for node creation and manipulation, whilst keeping the familiar Perl hash structure. You still get that Data::Dumper visibility.

Performance jumped to about 1,020 ops/sec in non threaded Perl roughly 1.3x faster than pure Perl. Respectable, but I knew we could do better. The problem is that we're still paying for Perl's hash overhead on every single operation.

Doubly::Pointer — Raw C Pointers

Here's where things get fast. Doubly::Pointer abandons Perl structures entirely and uses pure C:

typedef struct Node {
    SV* data;           // Perl scalar value
    struct Node* next;  // C pointer
    struct Node* prev;  // C pointer
} Node;

typedef struct {
    Node* head;
    Node* tail;
    Node* current;
    int length;
} DoublyList;

The Perl object is just a thin wrapper around a C pointer. No hashes, no Perl data structure overhead just raw memory and pointer arithmetic.

The result? 12,987 ops/sec. That's nearly 17x faster than pure Perl! Navigation is just pointer dereferencing. Insertion is just reassigning a couple of pointers. This is as fast as it gets in Perl land.

I released these three in quick succession and was happy with the performance boost of the Doubly implementation but it had some flaws. Mainly it was leaking memory, I asked on IRC for help but none was provided so I left the module for a while..

Then: Enter Claude

After letting the module sit for several months with its memory leaks, as we all know llm has gradually been getting better at code generation and debugging so I decided to revisit it with the help of Opus. What followed was an intensive collaboration that transformed my understanding of the problem.

Claude helped me trace through the reference counting issues, understand where my SvREFCNT_inc and SvREFCNT_dec calls were mismatched, and ultimately architect the registry based solution that would become Doubly. The breakthrough wasn't just fixing the leaks, it was realising that the entire pointer in Perl object approach was fundamentally flawed for threaded environments.

So the problem is that raw pointers becomes ticking time bombs in threaded code. When Perl clones an object across threads, it copies the pointer value. But that pointer address is only valid in the original thread's memory space. Access it from another thread and you get crashes, corruption, or worse... silent data mangling.

For single threaded applications, the old Doubly now Doubly::Pointer is fine. But I wanted something that could handle threads safely.

Doubly — The Thread Safe Registry

So with Claude's teachings, this is where it all came together. Doubly achieves thread safety through an architectural shift: instead of storing C pointers in Perl objects, it uses a global registry with integer IDs.

#define MAX_LISTS 65536

static DoublyList* registry[MAX_LISTS];
static int registry_ids[MAX_LISTS];
static perl_mutex registry_mutex;

The Perl object stores just an integer ID and a stable node ID. When you call any method, the XS code:

Locks the mutex
Looks up the list by ID in the registry
Finds the node by its unique _node_id
Performs the operation
Unlocks the mutex

// Lookup is O(1) - just array indexing
static DoublyList* _get_list(int id) {
    if (id < 0 || id >= MAX_LISTS) {
        return NULL;
    }
    return list_registry[id];
}

// Each operation wraps with locking:
SHARED_LOCK();
list = _get_list(id);
// ... perform operation ...
SHARED_UNLOCK();

The _node_id is stored per-object in Perl, so multiple references to the same list can point to different nodes, useful when different parts of your code need to track different positions simultaneously. Unlike position based indices, node IDs are stable: if you save a reference to a node and then insert or delete elsewhere in the list, your reference remains valid. This was a key architectural improvement that solved subtle bugs with an earlier position based approach, the same bugs you find with using just Arrays and a index.

For storing Perl references (hashes, arrays, objects), which is not simple for the same reasons as the pointers, Doubly uses threads::shared::shared_clone to create thread safe copies:

# When you do this in threaded Perl:
$list->add({ name => 'test', values => [1, 2, 3] });

# Doubly internally does:
my $shared = threads::shared::shared_clone($data);
# Then stores a reference ID to retrieve it later

We could not figure out how to do this in directly in the C/XS, so had to implement light callback hooks in the perl namespace, but it works perfectly.

The performance impact? In non threaded Perl, Doubly runs at 14,085 ops/sec, about 8% faster than Doubly::Pointer's 12,987 ops/sec. In threaded Perl, it performs similarly at 14,286 ops/sec versus 14,184 ops/sec. The registry architecture optimises surprisingly well. The integer to pointer lookup is O(1). You get nearly all the speed of raw pointers with none of the thread safety nightmares.

The threads::shared::shared_clone call is made directly from XS using Perl's call_pv() mechanism, but we needed light Perl-side helpers (_xs_store_ref, _xs_get_ref, _xs_clear_ref) for storing and retrieving from the shared hash—shared data structures require Perl-side assignment to work correctly with Perl's threading model.

So after all this, four implementations, countless hours of debugging segfaults, and deep dives into XS and mutex locks. When should you actually reach for a doubly linked list instead of a trusty array?

Choose Doubly Lists When:

You need O(1) insertions and deletions in the middle of your data. Arrays make you pay O(n) for every splice. If you're building an editor buffer, a task queue with priorities, or anything where items frequently enter and leave from arbitrary positions, doubly linked lists win decisively.

Your algorithm thinks in terms of "current position." Undo/redo systems, playlist navigators, browser history, paginated views. These all have an inherent notion of "where am I?" that maps perfectly to cursor based navigation. Fighting against this with index tracking is unnecessary complexity.

You need bidirectional traversal. Moving forward and backward through data is natural with next() and prev(). With arrays, you're doing index arithmetic and bounds checking manually.

Thread safety matters. If there's any chance your code will run in a threaded environment, Doubly's registry architecture handles it transparently. No locks to manage, no race conditions to debug.

You're storing nested structures with independent navigation state. This is Doubly's secret weapon, inner lists maintain their own cursor position. Try implementing that cleanly with arrays.

Give doubly linked lists a try the next time you catch yourself wrestling with array indices and splice operations. You might be surprised how naturally some problems fit the cursor based model. And if you find yourself needing threads? You'll be glad you chose Doubly from the start.

Happy coding and may your pointers always be valid!

https://metacpan.org/pod/Doubly