Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to save pandas groups to separate files

Tags:

python

pandas

I've created individual groups of my data using the following statements:

df = pd.read_csv(file_path)

grouped = df.groupby(df.some_parameter)

What I would then like to do (in psuedo-code is):

for name, group in grouped:
    'Some Text' + name = group
    write to csv

The end result being a separate .csv file of each chunk of the original dataset.

like image 312
BMichell Avatar asked Jan 24 '14 13:01

BMichell


2 Answers

This answer was very helpful to me - thanks @mkln.

I just wanted to add something specific to my own use case, which relates to the original point about file-naming ('Some Text' + name = group).

You could add the name and additional text, for example current date, to each csv filename, so I will create a function to return the current date and then use this for the filename.

Therefore:

from datetime import datetime

def cur_date():
    return datetime.now().strftime("%Y-%m-%d")

for name, group in grouped:
    group.to_csv('{}_{}.csv'.format(name, cur_date()))
like image 189
Ricky McMaster Avatar answered Nov 15 '22 20:11

Ricky McMaster


You were almost there

for name, group in grouped:
    group.to_csv(path_to_disk)
like image 16
mkln Avatar answered Nov 15 '22 18:11

mkln