Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does python garbage-collect at the end of an iteration in a loop?

Please observe this simple code:

    import random
    while True:
        L = list( str(random.random()))

Question: if I let this run, will python run out of memory?
reason I am asking:
First iteration of this loop, a list is created, and 'L' is assigned to represent that list. The next iteration of this loop, another list is created, 'L' is yanked from the previous list and and assigned to the new list. The previous list has lost it reference. Is the previous list going to be garbage collected? if not at the end of each iteration, but eventually I hope?

Having said that, just expand the scenario a bit further into multiprocessing:

    import random
    while True:
        l1 = list( str(random.random()))            
        pseudo: multiprocessing.Queue.put(l1)
        # how is l1 handled here?
        # is l1 .copy()-ed to the queue or referenced by the queue?
        # is l1 destoryed in this process (this while loop) at the end of iteration?
like image 778
eliu Avatar asked Apr 19 '16 20:04

eliu


People also ask

How does garbage collection take place in Python?

Garbage collection is implemented in Python in two ways: reference counting and generational. When the reference count of an object reaches 0, reference counting garbage collection algorithm cleans up the object immediately.

How often does Python garbage collect?

Any time a reference count drops to zero, the object is immediately removed. 295 * deallocated immediately at that time. A full collection is triggered when the number of new objects is greater than 25% of the number of existing objects.

Does Python have automatic garbage collection?

Yes, Python garbage collector removes every object not referenced to. The feature is based on reference counting. However it can also deal with cyclic references. Of course when the process is terminated, all its resources are released.

Does Python automatically delete unused objects?

Usually, you do not need to worry about memory management. When objects are no longer needed, Python automatically reclaims memory from them. However, understanding how GC works can help you write better and faster Python programs.


1 Answers

The primary means of garbage collection is reference counting in CPython (the reference implementation of the language). When there are no longer any references to an object, the memory it occupies is freed immediately and can be reused by other Python objects. (It may or may not ever be released back to the operating system.) There are a few exceptions of objects that are never freed: smallish integers, interned strings (including literals), the empty tuple, None.

So to answer your initial question, L is going to be reassigned to a new list on each iteration. At that point, the previous list has no references and its memory will be released immediately.

With regard to your second example, putting something into a multiprocessing queue is, of necessity, a copy operation. The object must be serialized ("pickled" in Python parlance) to be sent to the new process, which has its own memory space and can't see anything from the original process's memory. When, in your loop, you reassign li to the next list, the previous list has no references and, again, will be released.

At the end of your loop, the L or l1 variable still refers to a list: the one you created in the last iteration of the loop. If you want to release this object, just del L or del l1 respectively.

PS -- When objects contain references to themselves (either directly, or indirectly through a chain of other objects), this is referred to a cyclic reference. These aren't collected automatically by reference counting and Python has a separate garbage collector which runs periodically to clean them up.

like image 127
kindall Avatar answered Sep 22 '22 18:09

kindall