Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Simple way to get current memory usage from Guppy

Tags:

python

guppy

tl/dr: how do I get the current memory usage of my python program using Guppy? Is there a simple command?

I'm trying to track memory usage in a python program using guppy. This is my first usage of guppy, so I'm not very sure of how it behaves. What I want is to be able to plot the total usage as "time" progresses in a simulation. This is a basic bit of code for what I can do:

from guppy import hpy
import networkx as nx

h = hpy()
L=[1,2,3]

h.heap()

> Partition of a set of 89849 objects. Total size = 12530016 bytes.
>  Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
>     0  40337  45  3638400  29   3638400  29 str
>     1  21681  24  1874216  15   5512616  44 tuple
>     2   1435   2  1262344  10   6774960  54 dict (no owner)

But I would like to just know what the current size is (the 12530016 bytes). So I'd like to be able to call something like h.total() to get the total size. I'd be shocked if this doesn't exist as a simple command, but so far, looking through the documentation I haven't found it. It's probably documented, just not where I'm looking.

like image 967
Joel Avatar asked Feb 02 '15 00:02

Joel


People also ask

How do I check memory usage in Python?

You can use it by putting the @profile decorator around any function or method and running python -m memory_profiler myscript. You'll see line-by-line memory usage once your script exits.

What is GuPPy in Python?

The program presents users with a set of graphic user interfaces (GUIs) to load data and provide input parameters. Graphs are produced that can be easily exported for integration into scientific figures. As an open-source tool, GuPPy can be modified by users with knowledge of Python to fit their specific needs. © 2021.

What can you use in Python 2 to profile memory?

Memory Profiler: Memory Profiler is an open-source Python module that uses psutil module internally, to monitor the memory consumption of Python functions. It performs a line-by-line memory consumption analysis of the function.


1 Answers

x = h.heap()
x.size

returns the total size. For example:

from guppy import hpy
import networkx as nx

h = hpy()

num_nodes = 1000  
num_edges = 5000  
G = nx.gnm_random_graph(num_nodes, num_edges)

x = h.heap()
print(x.size)

prints

19820968

which is consistent with the Total size reported by

print(x)
# Partition of a set of 118369 objects. Total size = 19820904 bytes.
#  Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
#      0  51057  43  6905536  35   6905536  35 str
#      1   7726   7  3683536  19  10589072  53 dict (no owner)
#      2  28416  24  2523064  13  13112136  66 tuple
#      3    516   0  1641312   8  14753448  74 dict of module
#      4   7446   6   953088   5  15706536  79 types.CodeType
#      5   6950   6   834000   4  16540536  83 function
#      6    584   0   628160   3  17168696  87 dict of type
#      7    584   0   523144   3  17691840  89 type
#      8    169   0   461696   2  18153536  92 unicode
#      9    174   0   181584   1  18335120  93 dict of class
# <235 more rows. Type e.g. '_.more' to view.>
like image 65
unutbu Avatar answered Oct 13 '22 01:10

unutbu