I want to print the memory size of all variables in my scope simultaneously.
Something similar to:
for obj in locals().values():
    print sys.getsizeof(obj)
But with variable names before each value so I can see which variables I need to delete or split into batches.
Ideas?
getsizeof() can be done to find the storage size of a particular object that occupies some space in the memory. This function returns the size of the object in bytes. It takes at most two arguments i.e Object itself.
dir() is a built-in function to store all the variables inside a program along with the built-in variable functions and methods. It creates a list of all declared and built-in variables.
The easiest way to profile a single method or function is the open source memory-profiler package. It's similar to line_profiler , which I've written about before . You can use it by putting the @profile decorator around any function or method and running python -m memory_profiler myscript.
32-bit: the value will be 2^31 – 1, i.e. 2147483647.
A bit more code, but works in Python 3 and gives a sorted, human readable output:
import sys
def sizeof_fmt(num, suffix='B'):
    ''' by Fred Cirera,  https://stackoverflow.com/a/1094933/1870254, modified'''
    for unit in ['','Ki','Mi','Gi','Ti','Pi','Ei','Zi']:
        if abs(num) < 1024.0:
            return "%3.1f %s%s" % (num, unit, suffix)
        num /= 1024.0
    return "%.1f %s%s" % (num, 'Yi', suffix)
for name, size in sorted(((name, sys.getsizeof(value)) for name, value in locals().items()),
                         key= lambda x: -x[1])[:10]:
    print("{:>30}: {:>8}".format(name, sizeof_fmt(size)))
Example output:
                  umis:   3.6 GiB
       barcodes_sorted:   3.6 GiB
          barcodes_idx:   3.6 GiB
              barcodes:   3.6 GiB
                  cbcs:   3.6 GiB
         reads_per_umi:   1.3 GiB
          umis_per_cbc:  59.1 MiB
         reads_per_cbc:  59.1 MiB
                   _40:  12.1 KiB
                     _:   1.6 KiB
                        If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With