Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Error when using rbind to merge data.tables and one of them is empty

An error is issued when merging three (or more) data.tables with rbind and there is an empty data.table between two non-empty data.tables:

> require(data.table)
Loading required package: data.table
data.table 1.9.2  For help type: help("data.table")

> DT <- data.table(x=c(1,2,3))
> rbind(DT, data.table(), DT)
Error in setcolorder(tt, chmatch(match.names, make.names(original.names[[x]],  (from <text>#1) : 
  Column numbers in neworder out of bounds: NA

This does not happen if the empty data.table is in the end:

> rbind(DT, DT, data.table()) 
   x
1: 1
2: 2
3: 3
4: 1
5: 2
6: 3

But some other error happens when the empty data.table is in the beginning:

> rbind(data.table(), DT, DT) 
Error in data.table::.rbind.data.table(...) (from <text>#1) : 
  Some colnames of argument 2 (x) are not present in colnames of item 1. If an argument has colnames they can be in a different order, but they must all be present. Alternatively, you can supply unnamed lists and they will then be joined by position. Or, set use.names=FALSE.

However, if we use only two arguments everything is fine:

> rbind(data.table(), DT) 
   x
1: 1
2: 2
3: 3

What should be the expected behavior?

like image 531
Rodrigo Avatar asked Apr 22 '14 09:04

Rodrigo


2 Answers

This seems to be due to inconsistent behavior in data.table::.rbind.data.table. Adding the following line would fix this. Please file a bug-report with the data.table developers.

....
original.names = lapply(list(...), names)
allargs = lapply(list(...), as.data.table)
original.names = original.names[sapply(allargs, length) > 0L]  # this is the new line
allargs = allargs[sapply(allargs, length) > 0L]
...
like image 155
shadow Avatar answered Nov 14 '22 05:11

shadow


Bug #5612 is now fixed in commit #1266 (v1.9.3). From NEWS:

o Implementing #5249 closes bug #5612, a case where rbind gave error when binding with empty data.tables as shown here: Error when using rbind to merge data.tables and one of them is empty. Thanks to Roger for reporting on SO.

Now, the line:

> rbind(data.table(), DT, DT) 

gives:

#    x
# 1: 1
# 2: 2
# 3: 3
# 4: 1
# 5: 2
# 6: 3
like image 1
Arun Avatar answered Nov 14 '22 05:11

Arun