Find maximum memory used by a PBS job

Question

After a job is finished, how can I know the maximum resident size it required at any given point while running?

(tried /usr/bin/time, but not installed on the server)

Thank you!

Answer 1

PBS MOM reports some statistics back and it gets recorded in the PBS server log.

A handy utility called tracejob parses the logs to extract all entries related to a specific job given a job ID.

For example after the job completion on PBS Pro 12.1 tracejob would return several lines including the following

07/11/2014 16:37:27  S    Exit_status=0 resources_used.cpupercent=98
                          resources_used.cput=01:49:14 resources_used.mem=5368kb
                          resources_used.ncpus=1 resources_used.vmem=38276kb
                          resources_used.walltime=01:49:22

Here 5368 kb would correspond to the maximum rss.

Similarly on Torque 3.0.5

07/15/2014 03:45:12  S    Exit_status=0 resources_used.cput=20:44:10
                          resources_used.mem=704692kb
                          resources_used.vmem=1110224kb
                          resources_used.walltime=20:44:30

Here the maximum rss was 704692 kb