Menu
My python code process memory increases dynamically as it stores dynamic data in list, dictionary and tuples wherever necessary. Though all those dynamic data is cleared physically in their variables after then, the memory is not shooting down.
Hence i felt like there is a memory leak and i used gc.collect() method to collect all the unfreed memory. But i could not make the memory to minimum when there is no data in the variables.
660
Python optimizes memory utilization by allocating the same object reference to a new variable if the object already exists with the same value. That is why python is called more memory efficient.
A MemoryError means that the interpreter has run out of memory to allocate to your Python program. This may be due to an issue in the setup of the Python environment or it may be a concern with the code itself loading too much data at the same time.
del and gc. collect() are the two different methods to delete the memory in python. The clear memory method is helpful to prevent the overflow of memory. We can delete that memory whenever we have an unused variable, list, or array using these two methods.
It's very hard, in general, for a process to "give memory back to the OS" (until the process terminates and the OS gets back all the memory, of course) because (in most implementation) what malloc returns is carved out of big blocks for efficiency, but the whole block can't be given back if any part of it is still in use -- so, most C standard libraries don't even try.
For a decent discussion in a Python context, see e.g. here. Evan Jones fixed some Python-specific issues as described here and here, but his patch is in the trunk since Python 2.5, so the problems you're observing are definitely with the system malloc package, not with Python per se. A 2.6-specific explanation is here and here.
A SO thread is here, where Hugh Allen in his answer quotes Firefox programmers to the extend that Mac OS X is a system where it's basically impossible for a process to give memory back to the OS.
So, only by terminating a process can you be sure to release its memory. For example, a long-running server, once in a while, could snapshot its state to disk and shut down (with a tiny watchdog process, system or custom, watching over it and restarting it). If you know that the next operation will take a lot of memory for a short time, often you can os.fork, do the memory-hungry work in the child process, and have results (if any) returned to the parent process via a pipe as the child process terminates. And so on, and so forth.
97
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
