Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Grab the Memory Usage Value from Pandas DataFrame.info()

Tags:

pandas

How can I grab the memory usage value (displayed in the output of the funciton DataFrame.info() and assign to a variable?

like image 916
Rodrigo Teixeira Avatar asked Apr 26 '18 18:04

Rodrigo Teixeira


People also ask

How do I check Pandas memory usage?

The info() method in Pandas tells us how much memory is being taken up by a particular dataframe. To do this, we can assign the memory_usage argument a value = “deep” within the info() method. This will give us the total memory being taken up by the pandas dataframe.

What does DF info () do?

Pandas DataFrame info() Method The info() method prints information about the DataFrame. The information contains the number of columns, column labels, column data types, memory usage, range index, and the number of cells in each column (non-null values). Note: the info() method actually prints the info.

How much memory does a DataFrame take?

The long answer is the size limit for pandas DataFrames is 100 gigabytes (GB) of memory instead of a set number of cells. In effect, this benchmark is so large that it would take an extraordinarily large data set to reach it.

What does describe () do in Python?

The describe() method computes and displays summary statistics for a Python dataframe. (It also operates on dataframe columns and Pandas series objects.)


1 Answers

DataFrame.memory_usage().sum()

There's an example on this page:

In [8]: df.memory_usage()
Out[8]: 
Index                 72
bool                5000
complex128         80000
datetime64[ns]     40000
float64            40000
int64              40000
object             40000
timedelta64[ns]    40000
categorical         5800
dtype: int64

# total memory usage of dataframe
In [9]: df.memory_usage().sum()
Out[9]: 290872

Looking at the source code of df.info() shows that using memory_usage() is how they compute the actual memory usage in df.info():

... <last few lines of def info from pandas/frame.py>
    mem_usage = self.memory_usage(index=True, deep=deep).sum()
    lines.append("memory usage: %s\n" %
                 _sizeof_fmt(mem_usage, size_qualifier))
_put_lines(buf, lines)
like image 101
Jarad Avatar answered Oct 08 '22 01:10

Jarad