R: Merge data.table and fill in NAs

Question

Suppose 3 data tables:

dt1<-data.table(Type=c("a","b"),x=1:2)
dt2<-data.table(Type=c("a","b"),y=3:4)
dt3<-data.table(Type=c("c","d"),z=3:4)

I want to merge them into 1 data table, so I do this:

dt4<-merge(dt1,dt2,by="Type") # No error, produces what I want
dt5<-merge(dt4,dt3,by="Type") # Produces empty data.table (0 rows) of 4 cols: Type,x,y,z

Is there a way to make dt5 like this instead?:

> dt5
   Type x y z
1:    a 1 3 NA
2:    b 2 4 NA
3:    c NA NA 3
4:    d NA NA 4

Answer 1

While you explore the all argument to merge, I'll also offer you an alternative that might want to consider:

Reduce(function(x, y) merge(x, y, by = "Type", all = TRUE), list(dt1, dt2, dt3))
#    Type  x  y  z
# 1:    a  1  3 NA
# 2:    b  2  4 NA
# 3:    c NA NA  3
# 4:    d NA NA  4

Answer 2

If you know in advance the unique values you have in your Type column you can use J and then join tables the data.table way. You should set the key for each table so data.table knows what to join on, like this...

#  setkeys
setkey( dt1 , Type )
setkey( dt2 , Type )
setkey( dt3 , Type )


#  Join
dt1[ dt2[ dt3[ J( letters[1:4] ) , ] ] ]
#   Type  x  y  z
#1:    a  1  3 NA
#2:    b  2  4 NA
#3:    c NA NA  3
#4:    d NA NA  4

This shows off data.table's compound queries (i.e. dt1[dt2[dt3[...]]] ) which are wicked!

If you don't know in advance the unique values for the key column you can make a list of your tables and use lapply to quickly run through them getting the unique values to make your J expression...

#  A simple way to get the unique values to make 'J',
#  assuming they are in the first column.
ll <- list( dt1 , dt2 , dt3 )
vals <- unique( unlist( lapply( ll , `[` , 1 ) ) )
#[1] "a" "b" "c" "d"

Then use it like before, i.e. dt1[ dt2[ dt3[ J( vals ) , ] ] ].