tl/dr: how do I get the current memory usage of my python program using Guppy? Is there a simple command?
I'm trying to track memory usage in a python program using guppy. This is my first usage of guppy, so I'm not very sure of how it behaves. What I want is to be able to plot the total usage as "time" progresses in a simulation. This is a basic bit of code for what I can do:
from guppy import hpy
import networkx as nx
h = hpy()
L=[1,2,3]
h.heap()
> Partition of a set of 89849 objects. Total size = 12530016 bytes.
> Index Count % Size % Cumulative % Kind (class / dict of class)
> 0 40337 45 3638400 29 3638400 29 str
> 1 21681 24 1874216 15 5512616 44 tuple
> 2 1435 2 1262344 10 6774960 54 dict (no owner)
But I would like to just know what the current size is (the 12530016 bytes). So I'd like to be able to call something like h.total()
to get the total size. I'd be shocked if this doesn't exist as a simple command, but so far, looking through the documentation I haven't found it. It's probably documented, just not where I'm looking.
You can use it by putting the @profile decorator around any function or method and running python -m memory_profiler myscript. You'll see line-by-line memory usage once your script exits.
The program presents users with a set of graphic user interfaces (GUIs) to load data and provide input parameters. Graphs are produced that can be easily exported for integration into scientific figures. As an open-source tool, GuPPy can be modified by users with knowledge of Python to fit their specific needs. © 2021.
Memory Profiler: Memory Profiler is an open-source Python module that uses psutil module internally, to monitor the memory consumption of Python functions. It performs a line-by-line memory consumption analysis of the function.
x = h.heap()
x.size
returns the total size. For example:
from guppy import hpy
import networkx as nx
h = hpy()
num_nodes = 1000
num_edges = 5000
G = nx.gnm_random_graph(num_nodes, num_edges)
x = h.heap()
print(x.size)
prints
19820968
which is consistent with the Total size
reported by
print(x)
# Partition of a set of 118369 objects. Total size = 19820904 bytes.
# Index Count % Size % Cumulative % Kind (class / dict of class)
# 0 51057 43 6905536 35 6905536 35 str
# 1 7726 7 3683536 19 10589072 53 dict (no owner)
# 2 28416 24 2523064 13 13112136 66 tuple
# 3 516 0 1641312 8 14753448 74 dict of module
# 4 7446 6 953088 5 15706536 79 types.CodeType
# 5 6950 6 834000 4 16540536 83 function
# 6 584 0 628160 3 17168696 87 dict of type
# 7 584 0 523144 3 17691840 89 type
# 8 169 0 461696 2 18153536 92 unicode
# 9 174 0 181584 1 18335120 93 dict of class
# <235 more rows. Type e.g. '_.more' to view.>
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With