Perl program leaking memory when compiling regex

Question

I have a program that's exhibiting what appears to be a memory leak, but I'm having trouble tracking it down. The test program I wrote to demonstrate this behaves rather strangely, and I want to make sure I understand why.

In the example there are two arrays, one named regexLeak and another named noLeak. The noLeak arrays is simply there as a baseline using two simple regexes. The regexLeak array contains to regexes that use a character set.

The test program takes the contents of an array, presumably filled with regex patterns, passes it to a function where the regexes are compiled and placed into an array that's passed as a reference back.

When the "test" array is set to noLeak, there is no leak. When the "test" array is set to regexLeak, there is a notable memory leak. However, if the regexLeak array only contains one element, there is no leak. It doesn't matter which element either. Put both elements in the array though and memory use shoots through the roof.

#!/usr/bin/perl

use strict;
use warnings;

my @regexLeak;
my @noLeak;

# Leaks only when both elements are added to @regexLeak
push(@regexLeak,'user = ([a-zA-Z0-9-_.@]+)');
push(@regexLeak,'admin = ([a-zA-Z0-9-_.@]+)');

# No leaks
push(@noLeak, 'simpleRegex');
push(@noLeak, 'anotherSimpleOne');

my @test = @regexLeak;

my $compiled = compileRegex(@test);

while (1) {

    $compiled = compileRegex(@test);
    # print scalar @{$compiled}."\n";
    # select(undef, undef, undef, 0.25);
}

sub compileRegex {
    my (@r) = @_;

    my @compiled;

    foreach my $regex (@r) {
        my $c = qr/$regex/;
        push(@compiled,$c);
    }

    return \@compiled;
}

Some elements of this were taken from a production program that's exhibiting this problem. The global $compiled variable, for example. I'd like to keep this in here as explaining through them would help me understand. This is also a long running program so leaks are a concern.

Memory leak was observed using ps -aux and reviewing RSS and VSZ sizes.

Any help or guidance would be appreciated, thanks!

Edit: If you need any other environment details let me know

Perl v5.24.0 built for x86_64-linux-thread-multi

Answer 1

The short answer: install either

• v5.16.3 or v5.18.2 (older versions)
• or 5.25.10 (dev-version)
• the v5.24.0 and 5.24.1 shows growing :) memory

Other versions not tested.

The detailed results:

tested the above script leak.pl just added this line to the top of the script:

print "version: $^V ($^X) PID: $$\n";

On the second terminal me running this bash script

while :; do
    ps -o pid=,vsz=,rss=,command= | grep 'perl.*[l]eak'
    sleep 0.5
done

The results

plenv 5.16.3

$ plenv local 5.16.3
$ perl leak.pl
version: v5.16.3 (/opt/anyenv/envs/plenv/versions/5.16.3/bin/perl5.16.3) PID: 4922
^C

the ps (pid=,vsz=,rss=,command=)

 4922  2437208   3264 perl leak.pl
 4922  2441304   3312 perl leak.pl
 ....
 4922  2441304   3312 perl leak.pl
 # constant

OS-default v5.18.2

$ perl leak.pl
version: v5.18.2 (/usr/bin/perl) PID: 5718
#the ps constant
 5718  2455608   3612 perl leak.pl
 5718  2457656   3636 perl leak.pl
 ...
 5718  2457656   3636 perl leak.pl

plenv 5.24.0

$ perl leak.pl
version: v5.24.0 (/opt/anyenv/envs/plenv/versions/5.24.0/bin/perl5.24.0) PID: 6342
# GROWING
 6342  2458172  10672 perl leak.pl
 6342  2495036  40820 perl leak.pl
 6342  2526784  70984 perl leak.pl
 6342  2547268 101492 perl leak.pl
 6342  2579012 132380 perl leak.pl
 6342  2600516 163320 perl leak.pl
 6342  2639432 193636 perl leak.pl
 6342  2669128 223836 perl leak.pl
 6342  2700872 254012 perl leak.pl
 6342  2738760 284400 perl leak.pl
 6342  2768456 314256 perl leak.pl
 6342  2789960 344668 perl leak.pl

plenv 5.24.1

$ perl leak.pl
version: v5.24.1 (/opt/anyenv/envs/plenv/versions/5.24.1/bin/perl5.24.1) PID: 6518
#GROWING
 6518  2466296  23968 perl leak.pl
 6518  2504188  51404 perl leak.pl
 6518  2523644  78616 perl leak.pl
 6518  2560512 105832 perl leak.pl
 6518  2597376 133460 perl leak.pl
 6518  2617856 160496 perl leak.pl
 6518  2643460 187020 perl leak.pl
 6518  2672132 214760 perl leak.pl
 6518  2707972 241532 perl leak.pl
 6518  2743812 269120 perl leak.pl
 6518  2778628 296564 perl leak.pl
 6518  2798084 323252 perl leak.pl
 6518  2825732 350136 perl leak.pl

plenv 5.25.10

$ perl leak.pl
version: v5.25.10 (/opt/anyenv/envs/plenv/versions/5.25.10/bin/perl5.25.10) PID: 6732
# perl adjusted few times the memory - but steady constant
 6732  2445952   3760 perl leak.pl
 6732  2445952   3760 perl leak.pl
 6732  2446976   3772 perl leak.pl
 6732  2448000   3784 perl leak.pl
 6732  2448000   3784 perl leak.pl
 6732  2448000   3784 perl leak.pl
 6732  2448000   3784 perl leak.pl
 6732  2450048   3808 perl leak.pl
 6732  2450048   3812 perl leak.pl
 6732  2450048   3812 perl leak.pl
 6732  2450048   3812 perl leak.pl
 6732  2450048   3812 perl leak.pl
 6732  2450048   3812 perl leak.pl
 ...
 6732  2450048   3812 perl leak.pl