Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

When are objects garbage collected in python?

When are objects garbage collected in python? When is the memory released and does the collection impact performance? Can one opt out or tune the gc algorithm and if so how?

like image 507
Matt Alcock Avatar asked Feb 26 '12 00:02

Matt Alcock


People also ask

What triggers Python Garbage Collection?

For each generation, the garbage collector module has a threshold number of objects. If the number of objects exceeds that threshold, the garbage collector will trigger a collection process. For any objects that survive that process, they're moved into an older generation.

Is Garbage Collection in Python automatic?

Python has an automated garbage collection. It has an algorithm to deallocate objects which are no longer needed. Python has two ways to delete the unused objects from the memory.

How can you make sure an object is garbage collected?

An object is eligible to be garbage collected if its reference variable is lost from the program during execution. Sometimes they are also called unreachable objects. What is reference of an object? The new operator dynamically allocates memory for an object and returns a reference to it.

Does Python automatically delete objects?

Practical Data Science using PythonPython deletes unneeded objects (built-in types or class instances) automatically to free the memory space. The process by which Python periodically reclaims blocks of memory that no longer are in use is termed Garbage Collection.


2 Answers

When are objects garbage collected in python?

There is a lot of detail in the source code for CPython: http://svn.python.org/view/python/trunk/Modules/gcmodule.c?revision=81029&view=markup

Any time a reference count drops to zero, the object is immediately removed.

293 /* Python's cyclic gc should never see an incoming refcount

294 * of 0: if something decref'ed to 0, it should have been

295 * deallocated immediately at that time.

A full collection is triggered when the number of new objects is greater than 25% of the number of existing objects.

87 In addition to the various configurable thresholds, we only trigger a

88 full collection if the ratio

89 long_lived_pending / long_lived_total

90 is above a given value (hardwired to 25%).

When is the memory released?

I was only able to fish out this information.

781 /* Clear all free lists

782 * All free lists are cleared during the collection of the highest generation.

783 * Allocated items in the free list may keep a pymalloc arena occupied.

784 * Clearing the free lists may give back memory to the OS earlier.

785 */

According to this, Python may be keeping your object in a free list for recycling even if you drop its refcount to zero. I am unable to explicitly find when the free call is made to give memory back to the operating system, but I imagine that this is done whenever a collection is made and the object is not being kept in a free list.

Does the collection impact performance?

Any non-trivial garbage collector I have heard of requires both CPU and memory to operate. Therefore, yes, there is always an impact on performance. You'll have to experiment and get to know your garbage collector.

Programs that require real time responsiveness I have run into issues with, since garbage collectors don't grant me control over when they run or for how long they do. Some peculiar cases can cause excessive memory use as well, an example being Python's knack for keeping free lists.

like image 61
erisco Avatar answered Oct 17 '22 12:10

erisco


Here is an excerpt from the language reference

Objects are never explicitly destroyed; however, when they become unreachable they may be garbage-collected. An implementation is allowed to postpone garbage collection or omit it altogether — it is a matter of implementation quality how garbage collection is implemented, as long as no objects are collected that are still reachable.

CPython implementation detail: CPython currently uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable, but is not guaranteed to collect garbage containing circular references. See the documentation of the gc module for information on controlling the collection of cyclic garbage. Other implementations act differently and CPython may change. Do not depend on immediate finalization of objects when they become unreachable (ex: always close files).

EDIT: About postponing garbage collection .... the gc module allows you to interact with the garbage collector, and disable it if you want to and change collection frequency etc. But I have not used it myself. Also, cycles that contain any objects with __del__ methods are not collected.

like image 21
Praveen Gollakota Avatar answered Oct 17 '22 12:10

Praveen Gollakota