[.data.table (merge - X[Y]) key got lost in v.1.9.3

Question

I have 2 data.tables dtp and dtab.

require(data.table)
set.seed(1)
dtp <- data.table(pid = gl(3, 3, labels = c("du", "i", "nouana")),
                  year = gl(3, 1, 9, labels = c("2007", "2010", "2012")),
                  val = rnorm(9), key = c("pid", "year"))
dtab <- data.table(pid = factor(c("i", "nouana")),
                  year = factor(c("2010", "2000")),
                  abn = sample(1:5, 2, replace = TRUE), key =
                   c("pid", "year"))
dtp
##       pid year        val
## 1:     du 2007 -0.6264538
## 2:     du 2010  0.1836433
## 3:     du 2012 -0.8356286
## 4:      i 2007  1.5952808
## 5:      i 2010  0.3295078
## 6:      i 2012 -0.8204684
## 7: nouana 2007  0.4874291
## 8: nouana 2010  0.7383247
## 9: nouana 2012  0.5757814

dtab
##       pid year abn
## 1:      i 2010   2
## 2: nouana 2000   4

If i merge them using [.data.table, the key get lost:

dtp[dtab]
##       pid year       val abn
## 1:      i 2010 0.3295078   2
## 2: nouana 2000        NA   4

key(dtp[dtab]) # key got lost
## NULL # v.1.9.3

##### which was in 1.8.10
## [1] "pid"  "year"

Ok, for this case I can set it manually:

res1 <- setkeyv(dtp[dtab], key(dtp))
res1
##       pid year       val abn
## 1:      i 2010 0.3295078   2
## 2: nouana 2000        NA   4

key(res1) # repaired it
## [1] "pid"  "year"

Question:

Is this the wanted behaviour, or is it a bug?

Alternative:

Using the merge syntax does what I expected:

merge(dtp, dtab, all.y = TRUE)
##       pid year       val abn
## 1:      i 2010 0.3295078   2
## 2: nouana 2000        NA   4

key(merge(dtp, dtab, all.y = TRUE)) # everything ok
## [1] "pid"  "year"

Usage: add a column:

If I want to merge the column abn from dtab to dtp there is one easy possibility to write dtab[dtp] loosing the key and the order of the columns:

dtab[dtp]
##       pid year abn        val
## 1:     du 2007  NA -0.6264538
## 2:     du 2010  NA  0.1836433
## 3:     du 2012  NA -0.8356286
## 4:      i 2007  NA  1.5952808
## 5:      i 2010   2  0.3295078
## 6:      i 2012  NA -0.8204684
## 7: nouana 2007  NA  0.4874291
## 8: nouana 2010  NA  0.7383247
## 9: nouana 2012  NA  0.5757814

Example how it could work

If there would be other cols in dtab, but just abn should be merged there is another possibility (my favourite):

##### just show it:
## dtp[dtab[dtp, abn]] # v.1.8.10

dtp[dtab[dtp, abn, by = .EACHI]] # since v.1.9.3
##       pid year        val abn
## 1:     du 2007 -0.6264538  NA
## 2:     du 2010  0.1836433  NA
## 3:     du 2012 -0.8356286  NA
## 4:      i 2007  1.5952808  NA
## 5:      i 2010  0.3295078   2
## 6:      i 2012 -0.8204684  NA
## 7: nouana 2007  0.4874291  NA
## 8: nouana 2010  0.7383247  NA
## 9: nouana 2012  0.5757814  NA

or assign it:

dtp[dtab[dtp], abn := abn] # assign it

dtp
##       pid year        val abn
## 1:     du 2007 -0.6264538  NA
## 2:     du 2010  0.1836433  NA
## 3:     du 2012 -0.8356286  NA
## 4:      i 2007  1.5952808  NA
## 5:      i 2010  0.3295078   2
## 6:      i 2012 -0.8204684  NA
## 7: nouana 2007  0.4874291  NA
## 8: nouana 2010  0.7383247  NA
## 9: nouana 2012  0.5757814  NA

key(dtp) # ok
## [1] "pid"  "year"

In latter cases (show or assign) the key survives.

@Arun: Here is the sessionInfo():

sessionInfo()
## R version 3.1.0 (2014-04-10)
## Platform: powerpc64-unknown-linux-gnu (64-bit)

## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     

## other attached packages:
## [1] data.table_1.9.3

## loaded via a namespace (and not attached):
## [1] plyr_1.8       reshape2_1.2.2 stringr_0.6.2

Answer 1

Now fixed in v1.9.5. Closes #477. From NEWS:

56. Key is retained properly when joining on factor type columns. Closes #477. Thanks to @nachti for the report.

# v1.9.5+
key(dtp[dtab])
# [1] "pid"  "year"