Hacking the Perl core for faster keys, values and each

Last week, I wrote about hacking push and pop to take array references as well as literal arrays. This week, I’ve added similar functionality for keys, values and each.

Given a hash-of-hashref structure called $data, instead of this (Perl as you know it):

for my $k ( keys %{ $data->{key1} } ) { ... };

You can now do this (new, enhanced Perl):

for my $k ( keys $data->{key1} ) { ... };

Maybe you’re thinking “big deal”. But how often do you write keys %{ ... }? How would you like that 70% faster when written as keys ... by itself?

That’s right. Not only does the new version have less line noise, it’s faster than doing explicit dereferencing. From a simple benchmarking program:

use strict; use warnings; use autodie;
use Benchmark qw( cmpthese :hireswallclock );

my $hash = { 1 .. 1000 };
my %hash = ( 1 .. 1000 );

my $count = -10;
cmpthese( $count, {
    'keys %$hash' => sub { my @array = keys %$hash },
    'keys $hash'  => sub { my @array = keys $hash  },
    'keys %hash'  => sub { my @array = keys %hash  },
  });

I get these results with my new patches [1]:

# bleadperl + smart keys, values, each patches
              Rate keys %$hash  keys %hash  keys $hash
keys %$hash 4835/s          --        -41%        -42%
keys %hash  8219/s         70%          --         -1%
keys $hash  8317/s         72%          1%          --

Of course, it’s not just keys, but values and each as well, and for both hashes and arrays.

while ( my($k,$v) = each $hashref ) { ... }
while ( my($i,$v) = each $arrayref ) { ... }

Have you ever wanted a flatten function for arrays? (Without resorting to autobox::Core, that is.) Here it is:

for my $v ( values $obj->stuff_as_arrayref ) { ... }

It even works on objects that overload dereferencing:

for my $k ( keys $overloaded_object ) { ... }

Everything is heavily tested, but I’m working on writing up the new documentation. I’d also like to explore some possible edge cases, and then I’ll be ready to package it up and submit it to the perl5 porters mailing list for discussion.

If you think this is cool stuff, please let me know. (If you hate it, I’m happy to hear constructive criticism, too.)

- *[1] For comparison, I ran the same benchmark (minus the “keys $hash” case) on bleadperl without the patches. The timing of the base cases were similar. (Repeated runs of bleadperl and my patched perl gave similar results plus or minus a few percent.)

# bleadperl
              Rate keys %$hash  keys %hash
keys %$hash 4850/s          --        -41%
keys %hash  8240/s         70%          --