python, storing dictionaries inside a dataframe

Question

I've built a pandas dataframe which is storing a simple dictionary in each cell. For example:

{'Sales':0,'Revenue':0}

I can retrieve a specific value from the dataframe via:

df[columnA][index100]['Revenue']

But now I'd like to plot a graph of all the Revenue values from the dictionaries in columnA - what is the best way of achieving this?

Would life be easier in the long run if I dropped the dictionaries and instead used two identically sized dataframes? (Am very new to pandas so not sure of best practice).

Answer 1

A simple way to get all the Revenue values from a column A is df[columnA].map(lambda v: v['Revenue']).

Depending on what you're doing, life may indeed be easier if you tweak your structure a bit. For instance, you could use a hierarchical index with "Sales" and "Revenue" as the keys in one level.

Answer 2

For the majority of use cases you it's not a good idea to be storing dictionaries in DataFrame.
Another datastructure worth mentioning is a Panel.

Suppose you have something a DataFrame of dictionaries (with fairly consistent keys):

In [11]: df = pd.DataFrame([[{'a': 1, 'b': 2}, {'a': 3, 'b': 4}], [{'a': 5, 'b': 6}, {'a': 7, 'b': 8}]], columns=list('AB'))

In [12]: df
Out[12]:
                  A                 B
0  {'a': 1, 'b': 2}  {'a': 3, 'b': 4}
1  {'a': 5, 'b': 6}  {'a': 7, 'b': 8}

You can create a Panel (note there are more direct/preferable ways to construct this!):

In [13]: wp = pd.Panel({'A': df['A'].apply(pd.Series), 'B': df['B'].apply(pd.Series)})

In [14]: wp
Out[14]:
<class 'pandas.core.panel.Panel'>
Dimensions: 2 (items) x 2 (major_axis) x 2 (minor_axis)
Items axis: A to B
Major_axis axis: 0 to 1
Minor_axis axis: a to b

Sections of which can be accessed efficiently as DataFrames in a variety of ways, for example:

In [15]: wp.A
Out[15]:
   a  b
0  1  2
1  5  6

In [16]: wp.minor_xs('a')
Out[16]:
   A  B
0  1  3
1  5  7

In [17]: wp.major_xs(0)
Out[17]:
   A  B
a  1  3
b  2  4

So you can do all the pandas DataFrame whizziness:

In [18]: wp.A.plot()  # easy!
Out[18]: <matplotlib.axes.AxesSubplot at 0x1048342d0>

There are also ("experimental") higher dimensional Panels.