Menu
How can a python script know the amount of system memory it's currently using? (assuming a unix-based OS)
961
We can get this information using the handy psutil library, checking the resident memory of the current process: With this particular measurement, we’re using 3083MB, or 3.08GB, and the difference from the array size is no doubt the memory used by the Python interpreter and the libraries we’ve imported.
Memory Profiler The Memory Profiler is a python package that evaluates each line of Python code written within a function and correspondingly checks the usage of internal memory. We can either use pip or conda package managers to install this package.
>>> resource.getrusage (resource.RUSAGE_SELF).ru_maxrss 2656 # peak memory usage (kilobytes on Linux, bytes on OS X) The Python docs don't make note of the units. Refer to your specific system's man getrusage.2 page to check the unit for the value. On Ubuntu 18.04, the unit is noted as kilobytes. On Mac OS X, it's bytes.
Your Python batch process is using too much memory, and you have no idea which part of your code is responsible. You need a tool that will tell you exactly where to focus your optimization efforts, a tool designed for data scientists and scientists.
If you want to know the total memory that the interpreter uses, on Linux, read /proc/self/statm.
If you want to find out how much memory your objects use, use Pympler.
190
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
