Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas: Convert Series of DataFrames to single DataFrame

I have a pandas Series object with each value being a DataFrame. I am trying convert this into a single DataFrame with all of the Series values (individual DataFrame) stacked on top of each other. How can I achieve this without a loop?

A toy example below to generate the test object (results).

import pandas as pd
import numpy as np
numrows = 10000

def toy_function(x):
    silly_sequence = np.random.uniform(10, 100, (x+1))
    toy = pd.DataFrame({'ID':pd.Series(np.random.random_integers(1,20,3)),'VALUE':pd.Series((np.median(silly_sequence),np.mean(silly_sequence), np.max(silly_sequence)))})

    return toy

results = pd.DataFrame({'ID':range(numrows)})['ID'].apply(toy_function)

results is of Series type and each element is a DataFrame like so:

In [1]: results[1]
Out[1]: 
   ID      VALUE
0  17  40.035398
1   8  40.035398
2  20  66.483083

I am looking for a way to stack results[1], results[2] etc. on top of each other to yield a DataFrame like this:

   ID      VALUE
0  17  40.035398
1   8  40.035398
2  20  66.483083
4  12  25.035398
5   1  25.135398
6  19  65.553083
...
like image 834
sriramn Avatar asked May 27 '15 00:05

sriramn


People also ask

How do you convert multiple series to DataFrame?

You can create a DataFrame from multiple Series objects by adding each series as a columns. By using concat() method you can merge multiple series together into DataFrame. This takes several params, for our scenario we use list that takes series to combine and axis=1 to specify merge series as columns instead of rows.


1 Answers

Try using pd.concat. At the very least, pd.concat(series.values.tolist()) should work.

Its default is to take a list of pandas dataframes or series and return them tacked end on end. http://pandas.pydata.org/pandas-docs/stable/merging.html

like image 188
jwilner Avatar answered Oct 29 '22 03:10

jwilner