I have a Perl script that crunches a lot of data. There are a bunch of string variables that start small but grow really long due to the repeated use of the dot (concatentation) operator. Will growing the string in this manner result in repeated reallocations? If yes, is there a way to pre-allocate a string?
Yes, Perl growing a string will result in repeated reallocations. Perl allocates a little bit of extra space to strings, but only a few bytes. You can see this using Devel::Peek. This reallocation is very fast and often does not actually copy the memory. Trust your memory manager, that's why you're programming in Perl and not C. Benchmark it first!
You can preallocate arrays with $#array = $num_entries
and a hash with keys %hash = $num_keys
but length $string = $strlen
doesn't work. Here's a clever trick I dug up on Perlmonks.
my $str = "";
vec($str, $length, 8)=0;
$str = "";
Or if you want to get into XS you can call SvGROW()
.
chaos' suggestion to use an array and then join it all together will use more than double the memory. Memory for the array. Memory for each scalar allocated for each element in the array. Memory for the string held in each scalar element. Memory for the copy when joining. If it results in simpler code, do it, but don't think you're saving any memory.
Alternate suggestion that will be much easier to cope with: push the strings onto an array and join it when you're done.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With