Performant math operations on large Perl6 CArrays?

Question

I have some large CArrays returned by a native sub that I need to perform basic element-wise math operations on. The CArrays are usually on the order of 10^6 elements. I have found that calling .list on them to them treat them as normal Perl6 types is very expensive. Is there a way to do performant element-wise operations on them while keeping them CArrays?

Short test script to time some methods I've tried:

#!/usr/bin/env perl6
use v6.c;
use NativeCall;
use Terminal::Spinners;

my $list;
my $carray;
my $spinner = Spinner.new;

########## create data stuctures ##########

print "Creating 10e6 element List and CArray  ";
my $create = Promise.start: {
    $list = 42e0 xx 10e6;
    $carray = CArray[num32].new($list);
}
$spinner.await: $create;

########## time List subtractions ##########

my $time = now;
print "Substracting two 10e6 element Lists w/ hyper  ";
$spinner.await( Promise.start: {$list >>-<< $list} );
say "List hyper subtraction took: {now - $time} seconds";

$time = now;
print "Substracting two 10e6 element Lists w/ for loop  ";
my $diff = Promise.start: {
    for ^$list.elems {
        $list[$_] - $list[$_];
    }
}
$spinner.await: $diff;
say "List for loop subtraction took: {now - $time} seconds";

########## time CArray subtractions ##########

$time = now;
print "Substracting two 10e6 element CArrays w/ .list and hyper  ";
$spinner.await( Promise.start: {$carray.list >>-<< $carray.list} );
say "CArray .list and hyper subtraction took: {now - $time} seconds";

$time = now;
print "Substracting two 10e6 element CArrays w/ for loop  ";
$diff = Promise.start: {
    for ^$carray.elems {
        $carray[$_] - $carray[$_];
    }
}
$spinner.await: $diff;
say "CArray for loop subtraction took: {now - $time} seconds";

Output:

Creating 10e6 element List and CArray |
Substracting two 10e6 element Lists w/ hyper -
List hyper subtraction took: 26.1877042 seconds
Substracting two 10e6 element Lists w/ for loop -
List for loop subtraction took: 20.6394032 seconds
Substracting two 10e6 element CArrays w/ .list and hyper /
CArray .list and hyper subtraction took: 164.4888844 seconds
Substracting two 10e6 element CArrays w/ for loop |
CArray for loop subtraction took: 133.00560218 seconds

The for loop method seems fastest, but a CArray still took 6x longer to process than a List.

Any ideas?

Answer 1

As long as you can work with a different native data type, Matrix and Vector, you can use the (also native) port of the Gnu Scientific Library by Fernando Santagata. It's got a Vector.sub function you can use.

#!/usr/bin/env perl6
use v6.c;
use NativeCall;
use Terminal::Spinners;
use Math::Libgsl::Vector;

my $list;
my $carray;
my $spinner = Spinner.new;

########## create data stuctures ##########

print "Creating 10e6 element List and CArray  ";
my $list1 = Math::Libgsl::Vector.new(size => 10e6.Int);
$list1.setall(42);
my $list2 = Math::Libgsl::Vector.new(size => 10e6.Int);
$list2.setall(33);

########## time List subtractions ##########

my $time = now;
print "Substracting two 10e6 element Lists w/ hyper  ";
$spinner.await( Promise.start: { $list1.sub( $list2)} );
say "GSL Vector subtraction took: {now - $time} seconds";

This takes:

GSL Vector subtraction took: 0.08243 seconds

Is that fast enough for ya? :-)