pandas append not working; always return empty dataframe

Question

I am trying to combine two tables row wise (stack on top of each other, like using rbind in R). I've followed steps mentioned in:

Pandas version of rbind

how to combine two data frames in python pandas

But none of the "append" or "concat" are working for me.

About my data

I have two panda dataframe objects (type class 'pandas.core.frame.DataFrame'), both have 19 columns. when i print each dataframe they look fine.

The problem

So I created another panda dataframe using:

query_results = pd.DataFrame(columns=header_cols)

and then in a loop (because sometimes i may be combining more than just 2 tables) I am trying to combine all the tables:

for CCC in CCCList:
    query_results.append(cost_center_query(cccode=CCC))

where cost_center_query is a customized function and returns pandas dataframe objects with same column names as the query_results.

however, with this, whenever i print "query_results" i get empty dataframe.

any idea why this is happening? no error message as well, so i am just confused. Thank you so much for any advice!

Answer 1

Consider the concat method on a list of dataframes which avoids object expansion inside a loop with multiple append calls. Even consider a list comprehension:

query_results = pd.concat([cost_center_query(cccode=CCC) for CCC in CCCList], ignore_index=True)