Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multiple pandas.dataframe to one csv file

Tags:

python

pandas

csv

I have multiple pandas dataframes, and hope to write them as one CSV file. What is the most straightforward way?

For example, from following four dataframes,

enter image description here

how can I create below CSV?

enter image description here

Note The dataframes all have the same dimensions.

like image 216
Light Yagmi Avatar asked Jun 14 '15 13:06

Light Yagmi


2 Answers

A very straightforward way would be to concat pairs horizontally, concat the results vertically, and write it all out using to_csv:

 import pandas as pd

 pd.concat([
    pd.concat([df1, df2], axis=1),
    pd.concat([df3, df4], axis=1)]).to_csv('foo.csv')

A possibly more memory-conserving way would be to write it piecemeal:

with open('foo.csv', 'w') as f:
     pd.concat([df1, df2], axis=1).to_csv(f)
with open('foo.csv', 'a') as f:
     pd.concat([df3, df4], axis=1).to_csv(f, header=False)

Omitting headers=False would repeat the headers.

like image 119
Ami Tavory Avatar answered Sep 25 '22 08:09

Ami Tavory


For anyone that needs to append multiple dataframes to same csv vertically, you can use this:

with open('all_dfs.csv','a') as f:
    for df in list_of_dfs:
        df.to_csv(f)
        f.write("\n")
like image 39
Nicolae Stroncea Avatar answered Sep 23 '22 08:09

Nicolae Stroncea