Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python - Find current objects in memory

Is there a way to find the objects that are currently in memory including their names, where they are located and module names etc?

My process Python.exe before the main() method in Task Manager has a memory footprint of 15MB.

After the main method completes its first iteration, the process Python.exe memory size is 250MB.

I want to understand which objects are still in memory so that I can delete them

while True:
 # print current object details
 main() 
 # print current object details
like image 259
InfoLearner Avatar asked Jul 01 '20 23:07

InfoLearner


People also ask

How do you check the memory of an object in Python?

In Python, the most basic function for measuring the size of an object in memory is sys. getsizeof() .

How do you find the memory location of an object in Python?

Method 1: Using id() We can get an address using the id() function. id() function gives the address of the particular object. where the object is the data variables.

How do I get all the objects in Python?

import numpy as np import gc a = np. random. rand(100) objects = get_all_objects() print(any[x is a for x in objects]) # will return True, the np.

How do you check memory in an object?

Just go to Debug->Windows->Memory and open one of the four available or use the shortcut Ctrl+Alt+M, 1-4 . Then while debugging the application just type the name of the variable in the address field to translate it to a memory location and show the memory.

How to find out how much memory is being used in Python?

In this article, we are going to see how to find out how much memory is being used by an object in Python. For this, we will use sys.getsizeof () function can be done to find the storage size of a particular object that occupies some space in the memory. This function returns the size of the object in bytes.

How to get the memory address of an object in Python?

In Python, everything is an object, from variables to lists and dictionaries everything is treated as objects. In this article, we are going to get the memory address of an object in Python. We can get an address using the id () function. id () function gives the address of the particular object.

What is unexpected size of Python objects in memory?

In this article, we will discuss unexpected size of python objects in Memory. Python Objects include List, tuple, Dictionary, etc have different memory sizes and also each object will have a different memory address. Unexpected size means the memory size which we can not expect. But we can get the size by using getsizeof () function.

How does Python manage memory?

Python manages memory using reference counting semantics. Once an object is not referenced anymore, its memory is deallocated. But as long as there is a reference, the object will not be deallocated. Things like cyclical references can bite you pretty hard. CPython manages small objects (less than 256 bytes) in special pools on 8-byte boundaries.


Video Answer


2 Answers

No. There is no way to find all objects in Python. Also, most objects don't have names, and object "locations" don't work the way it sounds like you think they work.

The closest thing to what you're looking for is gc.get_objects, which returns a list of all GC-tracked objects. This is not a list of all objects, and it doesn't tell you why objects are still alive. You can get an object's GC-tracked referrers with gc.get_referrers, but not all references are known to the GC.

Even if you make sure objects you no longer need are unreachable, and even if their memory gets reclaimed, that still doesn't mean Python will actually return memory to the OS. Your memory usage might still be 250 MB at the end of all this.

like image 129
user2357112 supports Monica Avatar answered Sep 24 '22 17:09

user2357112 supports Monica


Getting currently loaded variables

The function dir() will list all loaded environment variables, such as:

a = 2
b = 3
c = 4
print(dir())

will return

['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'a', 'b', 'c']

Find below what the documentation of dir says:

dir(...) dir([object]) -> list of strings

If called without an argument, return the names in the current scope.
Else, return an alphabetized list of names comprising (some of) the attributes
of the given object, and of attributes reachable from it.
If the object supplies a method named __dir__, it will be used; otherwise
the default dir() logic is used and returns:
  for a module object: the module's attributes.
  for a class object:  its attributes, and recursively the attributes
    of its bases.
  for any other object: its attributes, its class's attributes, and
    recursively the attributes of its class's base classes.

Getting variable methods and attributes

You can also use dir() to list the methods and attributes associated with an object, for that you shall use: dir(<name of object>)

Getting size of variables currently loaded

If you wish to evaluate the size of your loaded variables/objects you can use sys.getsizeof(), as such:

sys.getsizef(a)
sys.getsizof(<name of variable>)

sys.getsizeof() gets you the size of an object in bytes (see this post for more on it)

Wrapping up

You could combine this functionality in some sort of loop like such

import sys
a =2
b = 3
c = 4
d = 'John'
e = {'Name': 'Matt', 'Age': 32}

for var in dir():
    print(var, type(eval(var)), eval(var), sys.getsizeof(eval(var)))

Hope that helps!

like image 38
Luis Chaves Rodriguez Avatar answered Sep 21 '22 17:09

Luis Chaves Rodriguez