DEV Community

Benchmarking Perl Core Class in v5.38

John Napiorkowski on August 25, 2023

UPDATE Gist to my test script so you can tell me what I did wrong: https://gist.github.com/jjn1056/6a34517ec1184d0fd7190b090eec1414 ...
Read full post
Collapse
 
tobyink profile image
Toby Inkster • • Edited

Ok so... I was not expecting core class to lose out to Moo.

You're largely testing the speed of the constructor in these tests, because it's a much slower process that reader accessor calls, so any differences in speed of the readers is going to be dwarfed by differences in speed of the constructors. And because there's only one object constructed by each iteration of the test, a lot is also wasted on the overhead of the benchmarking process.

Try tests like:

sub test_case1 {
  for my $i ( 1 .. 5000 ) {
    my $obj = MyClass::Moose->new(
      attribute1 => 'hello',
      attribute2 => 42, 
      attribute3 => $i,
    );
    # Test attribute reads more times than constructor,
    # on the grounds that once you've constructed an
    # object, you'll usually access it many times.
    my $dummy;
    $dummy = $obj->attribute1 for 1 .. 10;
    $dummy = $obj->attribute2 for 1 .. 10;
    $dummy = $obj->attribute3 for 1 .. 10;
  }
}
Enter fullscreen mode Exit fullscreen mode

And obviously the same for the Moo and Core implementations.

I'm actually surprised that the Moo size and Moose size are the same, I did expect Moose to be much fatter.

Moo and Moose objects are both just blessed hashrefs, and you've used the same hash keys (attribute names) and same values, so of course they're going to be the same sized objects. Where they'll differ is the overall size of the thread/process.

Another thing you might want to take into consideration is the presence or absence of the module Class::XSAccessor. Moo will use it to build XS accessors when possible, and this can have a major impact on benchmarking results. I'm not by any means suggesting that having it installed is "cheating", but not mentioning whether it's installed is leaving out some pretty important information.

Collapse
 
jjn1056 profile image
John Napiorkowski •

I have Class::XSA installed, no doubt if we ever add accessors to core class we can test that again

Collapse
 
leonerd profile image
Paul Evans • • Edited

As others have pointed out: You're largely just benchmarking the constructor here. Core class constructors do a lot of work, and also they're very much not-optimised yet. They're the absolute simplest implementation I could get away with technically working.

One thing you've not benchmarked at all here is how fast it is to access or modify fields from within methods of the actual class. These should be much faster than any of the method-based accessor models can provide, because they won't involve a method call just to read or write a field. Any real-world code is likely to contain large methods that have lots of field accesses. In those cases I would imagine things to run a lot faster.

I'll readily accept that if all you're doing with your objects is using them as "dumb structs" accessed entirely from outside with no interesting internal methods, then there's really not much if anything to be gained by using core classes in this manner. But then I already wrote a better way of handling those kinds of things anyway:

metacpan.org/pod/Struct::Dumb

Edit: I've also added a comment on another thread on the same subject; some more detail to be found here: dev.to/leonerd/comment/294n3

Collapse
 
jjn1056 profile image
John Napiorkowski •

The way fields work is core class is very interesting and I wish it could have been busted out and not come with all the baggage that the rest of core class has. In any case code would need to be totally rewritten to work as you are suggesting in which case my boss is going to say "we need to re-write the code? Great let's move away from Perl".

Paul, the point of my post is I'm trying to see an upside to core class and just not seeing it. You keep asking for feedback, this is what feedback looks like.

Collapse
 
leonerd profile image
Paul Evans •

You're trying to treat core's feature class as a fully-finished thing. It's not. It's an early implementation that I haven't spent more than maybe 20 minutes trying to do any optimization for at all. Getting it to work fast has not been at all my focus up tlil now - I've been trying to get it to exist at all. The fact that it performs in the same ballpark as the rest of the systems is already something close to a miracle.

If anyone more than just me was working on it in more than just bits and pieces of spare time, we'd no doubt have an even snappier, faster, more featured-thing by now. In fact, based I suspect in part on this very discussion, a couple of folks have already begun looking into it to do just that. Perhaps soon we'll have an updated version that performs even better against those many systems that have had a lot of folks looking into them for years already by now.

Thread Thread
 
jjn1056 profile image
John Napiorkowski •

I'd love for you to have more help on this because I suspect the main reason we're delivering something that is displeasing to many is that it has too few contributors. One of the main findings of my post IMHO is that maybe not everything needs rot be written in XS. The fact its XS is a HUGE barrier to contribution. I had assumed doing this all in XS was needed in order to get big performance and memory wins. Of course some of this needs to be at the XS/C level but maybe not all of it. Can we figure out together how to make it more possible for part of this to move to Perl land such that we can get more contributors? Maybe for example you could do more work on a MOP with an API that is exposed to Perl? I'm happy to work with you on that if we can do it.

Collapse
 
nrdvana profile image
M Conrad •

Any real-world code is likely to contain large methods that have lots of field accesses. In those cases I would imagine things to run a lot faster.

For some code, but I hope that you see that this will never be the case for Catalyst or DBIx::Class, and those are probably the use cases that jnap is thinking about. An object system for Catalyst or DBIx::Class is all about accessor speed, and in many cases constructor speed (row objects especially). These systems will also likely never be ported to the class system in its current form because they rely heavily on multiple inheritance.

Collapse
 
jjn1056 profile image
John Napiorkowski •

I think that is the issue here, I love the fields idea but in practice with tons of existing legacy code it doesn't bring much to the table. And what I'm saying is why are we doing all this at the XS level, where only a few people in the world can meaningfully contribute and where the change process has to be gate kept by the perl porters mailing list if there's no material benefit?

Collapse
 
chrisarg profile image
chrisarg •

Ha, this answered a question I asked at the FB. Now I am even less sure about what to do for an application I am designing: in a typical usage the app will be creating and destroying 10s of M of objects corresponding to database searches. I guess if the objects will never become part of an external API , one can go for blessed references and use something more secure externally? Or just do Moo ?

Collapse
 
jjn1056 profile image
John Napiorkowski •

I don't think any Perl application is going to be speedy at creating 10s of millions of objects. Either rework your architecture to not need that or if you do need it you will need to look at something like Golang or Rust.

Collapse
 
dkechag profile image
Dimitrios Kechagias •

There was another comparison of more object systems published here.