Pandas: DataFrame within DataFrame

Question

I need to create a DataFrame that contains columns of DataFrames. The DataFrames that go in the column have different sizes and I am getting a StopIteration exception. This doesn't happen, when the DataFrames are of the same size. I know a Panel is more suitable for this, but I need a DataFrame in this case.

a=pd.DataFrame({'cat1':['one','two','three'],'cat2':['four','five','six']})
b=pd.DataFrame({'cat1':['ten','eleven'],'cat2':['twelve','thirteen']})
pd.DataFrame({'col1':{'row1':a,'row2':b}})

If I remove the 'three' and 'six' items from 'cat1', 'cat2' respectively, then this works fine. Any idea how I can achieve this?

Answer 1

this is not a good idea, you lose all efficiency because things are treated as object dtype and operations will be quite slow (as operations cannot be done via c-level base types, like float/int). Better is to use a multi-level index, which can easily encompass what I think you want

In [20]: a
Out[20]: 
    cat1  cat2
0    one  four
1    two  five
2  three   six

In [21]: b
Out[21]: 
     cat1      cat2
0     ten    twelve
1  eleven  thirteen

In [22]: pd.concat([ a, b ], keys={ 'row1' : a, 'row2' : b })
Out[22]: 
          cat1      cat2
row1 0     one      four
     1     two      five
     2   three       six
row2 0     ten    twelve
     1  eleven  thirteen