Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Concat/join/merge multiple dataframes based on row index (number) of each individual dataframes

I want to read every nth row of a list of DataFrames and create a new DataFrames by appending all the Nth rows.

Let's say we have the following DataFrames:

>>> df1
   A    B    C    D
0 -0.8 -2.8 -0.3 -0.1
1 -0.1 -0.9  0.2 -0.7
2  0.7 -3.3 -1.1 -0.4 

>>> df2
   A    B    C    D
0  1.4 -0.7  1.5 -1.3
1  1.6  1.4  1.4  0.2
2 -1.4  0.2 -1.7  0.7 

>>> df3
   A    B    C    D
0  0.3 -0.5 -1.6 -0.8
1  0.2 -0.5 -1.1  1.6
2 -0.3  0.7 -1.0  1.0

I have used the following approach to get the desired df:

df = pd.DataFrame()

df_list = [df1, df2, df3]

for i in range(len(df1)):
    for x in df_list:
        df = df.append(x.loc[i], ignore_index = True)

Here's the result:

>>> df
   A    B    C    D
0 -0.8 -2.8 -0.3 -0.1
1  1.4 -0.7  1.5 -1.3
2  0.3 -0.5 -1.6 -0.8
3 -0.1 -0.9  0.2 -0.7
4  1.6  1.4  1.4  0.2
5  0.2 -0.5 -1.1  1.6
6  0.7 -3.3 -1.1 -0.4
7 -1.4  0.2 -1.7  0.7
8 -0.3  0.7 -1.0  1.0 

I was just wondering if there is a pandas way of rewriting this code which would do the same thing (maybe by using .iterrows, pd.concat, pd.join, or pd.merge)?

Cheers

Update Simply appending one df after another is not what I am looking for here.

The code should do:

df.row1 = df1.row1
df.row2 = df2.row1
df.row3 = df3.row1
df.row4 = df1.row2
df.row5 = df2.row2
df.row6 = df3.row2
...
like image 641
Manny Avatar asked Dec 09 '25 17:12

Manny


2 Answers

For a single output dataframe, you can concatenate and sort by index:

res = pd.concat([df1, df2, df3]).sort_index().reset_index(drop=True)

     A    B    C    D
0 -0.8 -2.8 -0.3 -0.1
1  1.4 -0.7  1.5 -1.3
2  0.3 -0.5 -1.6 -0.8
3 -0.1 -0.9  0.2 -0.7
4  1.6  1.4  1.4  0.2
5  0.2 -0.5 -1.1  1.6
6  0.7 -3.3 -1.1 -0.4
7 -1.4  0.2 -1.7  0.7
8 -0.3  0.7 -1.0  1.0

For a dictionary of dataframes, You can concatenate and then group by index:

res = dict(tuple(pd.concat([df1, df2, df3]).groupby(level=0)))

With the dictionary defined as above, each value represents a row number. For example, res[0] will give the first row from each input dataframe.

like image 59
jpp Avatar answered Dec 12 '25 08:12

jpp


There is pd.concat

df=pd.concat([df1,df2,df3]).reset_index(drop=True)

recommended by Jez

df=pd.concat([df1,df2,df3],ignore_index=True)
like image 28
BENY Avatar answered Dec 12 '25 08:12

BENY



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!