ipython notebook pandas max allowable columns

Question

I have a simple csv file with ten columns!

When I set the following option in the notebook and print my csv file (which is in a pandas dataframe) it doesn't print all the columns from left to right, it prints the first two, the next two underneath and so on.

I used this option, why isn't it working?

pd.option_context("display.max_rows",1,"display.max_columns",100)

Even this doesn't seem to work:

pandas.set_option('display.max_columns', None)

Answer 1

I assume you want to display your data in the notebook than the following options work fine for me (IPython 2.3):

import pandas as pd
from IPython.display import display
data = pd.read_csv('yourdata.txt')

Either directly set the option

pd.options.display.max_columns = None
display(data)

Or, use the set_option method you showed actually works fine as well

pd.set_option('display.max_columns', None)
display(data)

If you don't want to set this options for the whole script use the context manager

with pd.option_context('display.max_columns', None):
    display(data)

If this doesn't help, you might give a minimal example to reproduce your issue.

Answer 2

You can also display all the data by asking pandas to return HTML markup, and then having IPython render the HTML table.

import pandas as pd
from IPython.display import HTML
data = pd.read_csv('yourdata.csv')
HTML(data.to_html())

Using IPython 3.0.0 and Python 3.4, I found that display(data) as described by @Jakob will render as a table with up/down and left/right scroll bars, but the table is still wider than the cell and some columns are off-screen to the right. To see all the data, one must collapse the cell - which adds scroll bars. Consequently you have a scrolling box in a scrolling box, which is not ideal as you have to shift focus between the doubled-up scroll bars to navigate all the way through the data.

Using the HTML method, you render the enormous table as-is without any scroll bars. This cell can then be collapsed down to show only a single vertical and horizontal bar, which is more user-friendly.

The caveat to using HTML is the table takes longer to render. I was only using a ~150x50 matrix and the speed difference was noticeable, but not inconvenient. If you have an enormous table, don't use this method to display the entire thing at once. That said, if you do have an enormous table, rendering the whole thing at once is obviously going to be a bad idea however you try to do it.