Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Perl script strange behavior / Reclaim memory

My script does the following

  1. Reads a huge text file and creates a hash from it.(About 24million simple key value-pairs.Takes about 5 minutes and consumes 92% of the 4Gb computer memory)
  2. Runs a simulation using information from hash.(Takes about 30minutes)
  3. Prints the results at the end of the simulation(to a file and stdout)

Then it waits for 10+ minutes after the last print statement and exits. The wait at the end doesn't happen every time. During the wait top command shows the same 92% memory usage but no cpu usage. Why does it wait sometimes after it is done ? If I hit CtrlC, it exits immediately without any change in outcome(results). How do I debug this or is it expected behavior as the hash is huge ?

EDIT

Is it possible to reclaim some memory on the fly by deleting unwanted key - value pairs from the hash ?

like image 452
Jean Avatar asked Nov 21 '13 19:11

Jean


1 Answers

I assume you know 'delete' function (so removed reference to perldoc -f delete :)

For the memory debugging you could use valgrind Also this hint can be helpful: Does Perl v5.10.1 have memory leaks or how to interpret valgrind It suggests using:

use Perl::Destruct::Level level => 1;

like image 190
stimur Avatar answered Sep 20 '22 23:09

stimur