Python

When dealing with Pandas, I'm attempting to print analysis of an objects Kinematic and Angular states. My code for doing so is as follows:

def displayData(tList, xList, zList, dxList, dzList, thetaList, dthetaList, Q_sList):
    states = pd.DataFrame({ 't' : tList,
                            'x' : xList,
                            'z' : zList,
                            'dx' : dxList,
                            'dz' : dzList,
                            'theta' : thetaList,
                            'dtheta' : dthetaList,
                            'Q_s' : Q_sList})

    print states[['t', 'x', 'z', 'dx', 'dz', 'theta', 'dtheta', 'Q_s']]

However, when asked to print the data, the output breaks up the columns beyond a certain point:

          t           x           z          dx         dz     theta  \
0     0.000 -500.000000 -100.000000  100.000000  -0.000000  0.000000   
1     0.005 -499.500000 -100.000000   99.999983   0.057692 -0.000577   
2     0.010 -499.000000  -99.999712   99.999933   0.115329 -0.001153
...     ...         ...         ...         ...        ...       ... 

        dtheta       Q_s  
0    -0.115385 -0.038462  
1    -0.115274 -0.038425  
2    -0.115163 -0.038388
...        ...       ...

As I have many thousands of states to print at the time, I would like for pandas to not break the table up like so, allowing me to analyze one given state without having to scroll to pick up the remaining two data fields. Is there any way I can define specific dimensions to be printed out so that this does not occur?

Does pandas have a column limit?

The default number of the max number of columns is 20. If we don't change it and having a data frame with more than 20 columns, we won't be able to view the columns in the middle again.

How many columns can pandas handle?

There isn't a set maximum of columns - the issue is that you've quite simply run out of available memory on your computer, unfortunately. One way to fix it is to get some more memory - but that obviously isn't a solid solution in the long run (might be quite expensive, too).

Is there a size limit for pandas DataFrame?

The short answer is yes, there is a size limit for pandas DataFrames, but it's so large you will likely never have to worry about it. The long answer is the size limit for pandas DataFrames is 100 gigabytes (GB) of memory instead of a set number of cells.

There are two useful settings which can be used in this case: pd.options.display.width and pd.options.display.expand_frame_repr

Here is a small demo:

In [118]: pd.options.display.expand_frame_repr
Out[118]: True

In [119]: pd.options.display.width = 50

In [120]: df
Out[120]:
       t      x           z          dx  \
0  0.000 -500.0 -100.000000  100.000000
1  0.005 -499.5 -100.000000   99.999983
2  0.010 -499.0  -99.999712   99.999933

         dz     theta    dtheta       Q_s
0 -0.000000  0.000000 -0.115385 -0.038462
1  0.057692 -0.000577 -0.115274 -0.038425
2  0.115329 -0.001153 -0.115163 -0.038388

In [121]: pd.options.display.width = 100

In [122]: df
Out[122]:
       t      x           z          dx        dz     theta    dtheta       Q_s
0  0.000 -500.0 -100.000000  100.000000 -0.000000  0.000000 -0.115385 -0.038462
1  0.005 -499.5 -100.000000   99.999983  0.057692 -0.000577 -0.115274 -0.038425
2  0.010 -499.0  -99.999712   99.999933  0.115329 -0.001153 -0.115163 -0.038388

In [131]: pd.options.display.width = 40

In [132]: df
Out[132]:
       t      x           z  \
0  0.000 -500.0 -100.000000
1  0.005 -499.5 -100.000000
2  0.010 -499.0  -99.999712

           dx        dz     theta  \
0  100.000000 -0.000000  0.000000
1   99.999983  0.057692 -0.000577
2   99.999933  0.115329 -0.001153

     dtheta       Q_s
0 -0.115385 -0.038462
1 -0.115274 -0.038425
2 -0.115163 -0.038388


In [125]: pd.options.display.expand_frame_repr = False

In [126]: df
Out[126]:
       t      x           z          dx        dz     theta    dtheta       Q_s
0  0.000 -500.0 -100.000000  100.000000 -0.000000  0.000000 -0.115385 -0.038462
1  0.005 -499.5 -100.000000   99.999983  0.057692 -0.000577 -0.115274 -0.038425
2  0.010 -499.0  -99.999712   99.999933  0.115329 -0.001153 -0.115163 -0.038388

In [127]: pd.options.display.width
Out[127]: 30

alternatively, you can use set_options() method

Here is a list of all diplay options:

In [128]: pd.options.display.
pd.options.display.chop_threshold     pd.options.display.latex              pd.options.display.mpl_style
pd.options.display.colheader_justify  pd.options.display.line_width         pd.options.display.multi_sparse
pd.options.display.column_space       pd.options.display.max_categories     pd.options.display.notebook_repr_html
pd.options.display.date_dayfirst      pd.options.display.max_columns        pd.options.display.pprint_nest_depth
pd.options.display.date_yearfirst     pd.options.display.max_colwidth       pd.options.display.precision
pd.options.display.encoding           pd.options.display.max_info_columns   pd.options.display.show_dimensions
pd.options.display.expand_frame_repr  pd.options.display.max_info_rows      pd.options.display.unicode
pd.options.display.float_format       pd.options.display.max_rows           pd.options.display.width
pd.options.display.height             pd.options.display.max_seq_items
pd.options.display.large_repr         pd.options.display.memory_usage

Python - Pandas Output Limits Columns

Tags:

io

pandas

dataframe

S. Gamgee

People also ask

1 Answers

MaxU - stop WAR against UA

Recent Activity

Donate For Us