Menu
I have a huge hash containing about 10 years worth of daily statistical data. Is there is a way in Perl 6 to determine how much real memory (in bytes) this hash is using (e.g. showMemoryUsed(%myBigHash)). Even if %myBigHash is empty, it is not zero bytes because of the memory used and the Perl 6 implementation of the hash data type. This info will tell me if I need to re-implement my codes or periodically write out to file to alleviate RAM shortage (my program is running on a virtualized Linux with 2G RAM).
Thanks.
942
A part of the data structure is stored using the Yggdrasil memory. The solution provides all the operations in worst case constant running time using 2M + O(lg M) bits of ordinary memory and M bits of the special Yggdrasil memory.
A Bloom filter [1] is a space-efficient approximate data structure. It can be used if not even a well-loaded hash table fits in memory and we need constant read access.
Data structures are generally based on the ability of a computer to fetch and store data at any place in its memory, specified by a pointer—a bit string, representing a memory address, that can be itself stored in memory and manipulated by the program.
In Python, the most basic function for measuring the size of an object in memory is sys. getsizeof() .
Alas, we don't have a thing like that in Rakudo Perl 6 yet. The only thing I can recommend, is using the Telemetry module:
use Telemetry;
my $before = T<max-rss>;
my %h = ...; # initialize hash
say "Memory usage grew { T<max-rss> - $before } KB";
Check out the Telemetry documentation for more information: https://docs.raku.org/type/Telemetry
137
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
