Ignore empty dataframe when merging

Question

I have four df (df1,df2,df3,df4)

Sometimes df1 is null, sometimes df2 is null, sometimes df3 and df4 accordingly.

How can I do an outer merge so that the df which is empty is automatically ignored? I am using the below code to merge as of now:-

df = f1.result().merge(f2.result(), how='left', left_on='time', right_on='time').merge(f3.result(), how='left', left_on='time', right_on='time').merge(f4.result(), how='left', left_on='time', right_on='time')

and

df = reduce(lambda x,y: pd.merge(x,y, on='time', how='outer'), [f1.result(),f2.result(),f3.result(),f4.result()])

Answer 1

You can use df.empty attribute or len(df) > 0 to check whether the dataframe is empty or not.

Try this:

dfs = [df1, df2, df3, df4]
non_empty_dfs = [df for df in dfs if not df.empty]

df_final = reduce(lambda left,right: pd.merge(left,right, on='time', how='outer'), non_empty_dfs)

Or, you could also filter empty dataframe as,

non_empty_dfs = [df for df in dfs if len(df) > 0]