peak memory measurement of long running process in linux

Question

How do I monitor the peak memory consumed by a process in Linux? This is not a program I can internally modify to measure peak memory usage.

I do not really want detailed measurements, nor do I want them to slow down my program excessively.. so valgrind or anything heavyweight is not what I am looking for... And like other posts earlier [Peak memory usage of a linux/unix process, time -v doesn't seem to report memory for my machine...

I can just run top or ps and extract the memory consumed strings for my process id using a simple script. However, my process runs for about 20-30 minutes so I want to be able to log and get the max. I can tolerate coarse grained samples... every 1-minute or so... Specifically how do I-> 1. fork this simple mem-measure script in zsh? 2. kill it when the process-under-test finishes?

Answer 1

Just use top -n to iterate a specified number of times, and -d to delay between updates. Also you can grab only the output relevant to your process by grepping its pid, like:

top -n 30 -d 60 | grep <process-id>

Read the top manual page for more information

man top

Of course, you can also grab the column you need by using awk.

Answer 2

Actually, what I said before:

"""

try

/usr/bin/time -v yourcommand

that should help. if you use only "time", bash will execute the built-in (that does not have "-v")

"""

does not work (returns 0).

I made the following perl script (that I called smaps):

#!/usr/bin/perl
use 5.010;
use strict;
use warnings;
my $max = 0;
while( open my $f, '<', "/proc/$ARGV[0]/smaps" ) {
  local $/; $_ = <$f>;
  $max = $1 if /Rss:\s*(\d+)/ and $1 > $max;
  open my $g, '>', '/tmp/max';
  say $g $max
}

And then I call it (for instance, to watch qgit's memory usage):

bash -c './smaps $$ & exec qgit'

Use single quotes so the "daughter" shell interprets $$ (that will be the same PID of qgit after the exec). this answer, I tested :-D

HTH

Answer 3

Valgrind with massif should not be too heavy, but, I'd recommend using /proc. You can easily write your own monitor script. Here is mine, for your convenience:

#!/bin/bash

ppid=$$
maxmem=0

$@ &
pid=`pgrep -P ${ppid} -n -f $1` # $! may work here but not later
while [[ ${pid} -ne "" ]]; do
    #mem=`ps v | grep "^[ ]*${pid}" | awk '{print $8}'`
        #the previous does not work with MPI
        mem=`cat /proc/${pid}/status | grep VmRSS | awk '{print $2}'`
    if [[ ${mem} -gt ${maxmem} ]]; then
        maxmem=${mem}
    fi
    sleep 1
    savedpid=${pid}
    pid=`pgrep -P ${ppid} -n -f $1`
done
wait ${savedpid} # don't wait, job is finished
exitstatus=$?   # catch the exit status of wait, the same of $@
echo -e "Memory usage for $@ is: ${maxmem} KB. Exit status: ${exitstatus}\n"

Answer 4

You could use a munin-node plugin to do this, but it's a little heavyweight. http://munin.projects.linpro.no/