Add missing values in time series efficiently

Question

I have 500 datasets (panel data). In each I have a time series (week) across different shops (store). Within each shop, I would need to add missing time series observations.

A sample of my data would be:

store   week           value
1           1          50
1           3          52
1           4          10
2           1          4
2           4          84
2           5          2

which I would like to look like:

store   week        value
1           1       50
1           2       0
1           3       52
1           4       10
2           1       4
2           2       0
2           3       0
2           4       84
2           5       2

I currently use the following code (which works, but takes very very long on my data):

  stores<-unique(mydata$store)

  for (i in 1:length(stores)){ 
  mydata <- merge(
    expand.grid(week=min(mydata$week):max(mydata$week)),
    mydata, all=TRUE)
  mydata[is.na(mydata)] <- 0
  }

Are there better and more efficient ways to do so?

Answer 1

Here's a dplyr/tidyr option you could try:

library(dplyr); library(tidyr)
group_by(df, store) %>% 
  complete(week = full_seq(week, 1L), fill = list(value = 0)) 
#Source: local data frame [9 x 3]
#
#  store  week value
#  (int) (int) (dbl)
#1     1     1    50
#2     1     2     0
#3     1     3    52
#4     1     4    10
#5     2     1     4
#6     2     2     0
#7     2     3     0
#8     2     4    84
#9     2     5     2

By default, if you don't specify the fill parameter, new rows will be filled with NA. Since you seem to have many other columns, I would advise to leave out the fill parameter so you end up with NAs, and if required, make another step with mutate_each to turn NAs into 0 (if that's appropriate).

group_by(df, store) %>% 
  complete(week = full_seq(week, 1L)) %>%
  mutate_each(funs(replace(., which(is.na(.)), 0)), -store, -week)