Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Append data frames together in a for loop

Tags:

r

I have a for loop which produces a data frame after each iteration. I want to append all data frames together but finding it difficult. Following is what I am trying, please suggest how to fix it:

d = NULL for (i in 1:7) {    # vector output   model <- #some processing    # add vector to a dataframe   df <- data.frame(model)  }  df_total <- rbind(d,df) 
like image 591
Ibe Avatar asked Apr 01 '15 23:04

Ibe


People also ask

How do you append data to a DataFrame in a for loop?

It turns out Pandas does have an effective way to append to a dataframe: df. loc( len(df) ) = [new, row, of, data] will "append" to the end of a dataframe in-place.

How do I add a data frame to a loop in R?

Within the for-loop we have performed three steps: First, we have created a vector object containing the values that we wanted to add as column to our data frame. Second, we added the new column ad the end of our data frame. Third, we renamed our new column (this step is optional).

How do I append data from one Dataframe to another?

Pandas dataframe.append() function is used to append rows of other dataframe to the end of the given dataframe, returning a new dataframe object. Columns not in the original dataframes are added as new columns and the new cells are populated with NaN value.

How to add a column to a Dataframe using for loop?

Within the for-loop we have performed three steps: First, we have created a vector object containing the values that we wanted to add as column to our data frame. Second, we added the new column ad the end of our data frame. Third, we renamed our new column (this step is optional).

How to append rows of other Dataframe in pandas?

Pandas dataframe.append () function is used to append rows of other dataframe to the end of the given dataframe, returning a new dataframe object. Columns not in the original dataframes are added as new columns and the new cells are populated with NaN value. Attention geek!

Why is concatenating DataFrames in a loop so inefficient?

This is inefficient as it involves copying data repeatedly. A much better idea is to create a list of dataframes and then concatenate them at the end in a final step outside your loop. Here's some pseudo-code:


2 Answers

Don't do it inside the loop. Make a list, then combine them outside the loop.

datalist = list()  for (i in 1:5) {     # ... make some data     dat <- data.frame(x = rnorm(10), y = runif(10))     dat$i <- i  # maybe you want to keep track of which iteration produced it?     datalist[[i]] <- dat # add it to your list }  big_data = do.call(rbind, datalist) # or big_data <- dplyr::bind_rows(datalist) # or big_data <- data.table::rbindlist(datalist) 

This is a much more R-like way to do things. It can also be substantially faster, especially if you use dplyr::bind_rows or data.table::rbindlist for the final combining of data frames.

like image 122
Gregor Thomas Avatar answered Nov 16 '22 02:11

Gregor Thomas


You should try this:

df_total = data.frame() for (i in 1:7){     # vector output     model <- #some processing      # add vector to a dataframe     df <- data.frame(model)     df_total <- rbind(df_total,df) } 
like image 32
maRtin Avatar answered Nov 16 '22 00:11

maRtin