Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Combine dictionary of dataframes into 1 single dataframe

I am looking for a solution to put all my dataframes which are in a dictionary into 1 single giant dataframe. I am relatively new to Python so I am unable to understand how to iterate over a dictionary and put all the dataframes into 1. The code I have implemented so far, is as below:

import sys
from ftplib import FTP
import os
import socket
import time
import pandas as pd
import numpy as np
from glob import glob

path = 'path_to_file'

files = glob(path + '/*Mail*.xlsx')

print files

get_df = lambda f: pd.read_excel(f, sheetname=None)

dodf = {f: get_df(f) for f in files}  ### dictionary of dataframes

Now, I need to put all the different dataframes into 1 dataframe and then do my operations on that. Any advise would be appreciated.

I have tried this,

for df in dodf:
pd.concat(dodf.values(), ignore_index=True)

But it does not seem to work right.

like image 897
Manas Jani Avatar asked Sep 18 '17 13:09

Manas Jani


2 Answers

I think need concat with dict comprehension:

dodf = {f: pd.read_excel(f, sheet_name=None) for f in files}
df = pd.concat([v for k,v in dodf.items()])

Or:

dodf = {f: pd.read_excel(f, sheet_name=None) for f in files}
df = pd.concat([pd.concat(v) for k,v in dodf.items()])
like image 181
jezrael Avatar answered Nov 15 '22 19:11

jezrael


df_list = [ v for k,v in dodf.items()] 
df = pd.concat(df_list ,axis=1)

does this work? It also depends whether the concat is by columns or by rows...

like image 34
00__00__00 Avatar answered Nov 15 '22 21:11

00__00__00