How can I grab the memory usage value (displayed in the output of the funciton DataFrame.info()
and assign to a variable?
The info() method in Pandas tells us how much memory is being taken up by a particular dataframe. To do this, we can assign the memory_usage argument a value = “deep” within the info() method. This will give us the total memory being taken up by the pandas dataframe.
Pandas DataFrame info() Method The info() method prints information about the DataFrame. The information contains the number of columns, column labels, column data types, memory usage, range index, and the number of cells in each column (non-null values). Note: the info() method actually prints the info.
The long answer is the size limit for pandas DataFrames is 100 gigabytes (GB) of memory instead of a set number of cells. In effect, this benchmark is so large that it would take an extraordinarily large data set to reach it.
The describe() method computes and displays summary statistics for a Python dataframe. (It also operates on dataframe columns and Pandas series objects.)
DataFrame.memory_usage().sum()
There's an example on this page:
In [8]: df.memory_usage()
Out[8]:
Index 72
bool 5000
complex128 80000
datetime64[ns] 40000
float64 40000
int64 40000
object 40000
timedelta64[ns] 40000
categorical 5800
dtype: int64
# total memory usage of dataframe
In [9]: df.memory_usage().sum()
Out[9]: 290872
Looking at the source code of df.info() shows that using memory_usage() is how they compute the actual memory usage in df.info():
... <last few lines of def info from pandas/frame.py>
mem_usage = self.memory_usage(index=True, deep=deep).sum()
lines.append("memory usage: %s\n" %
_sizeof_fmt(mem_usage, size_qualifier))
_put_lines(buf, lines)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With