Why has data.table defined := rather than overloading <-?

Question

data.table has introduced the := operator. Why not overload <-?

Answer 1

There are two places that <- could be 'overloaded' :

x[i, j] <- value           # 1 x[i, {colname <- value}]   # 2 

The first one copies the whole of x to *tmp*, changes that working copy, and assigns back to x. That's an R thing (src/main/eval.c and subassign.c) discussed recently on r-devel here. It sounded like it might be possible to change R to allow packages, or R itself, to avoid that copy to *tmp*, but isn't currently possible, IIUC.

The second one is what Owen's answer refers to, I think. If you accept that it's ok to do assignment by reference within j like that, then which operator? As per the comment to Owen's answer, <- and <<- are already used by users in j, so we hit upon :=.

Even if [<- didn't copy the whole of x, we still like := in j so we can do things like this :

DT[,{newcol1:=sum(a)      newcol2:=a/newcol1}, by=group] 

Where the new columns are added by reference to the table, and the RHS of each := is evaluated within each group. (When := within group is implemented.)

---

Update Oct 2012

As of 1.8.2 (on CRAN in Jul 2012), := by group was implemented for adding or updating single columns; i.e., single LHS of :=. And now in v1.8.3 (on R-Forge at the time of writing), multiple columns can be added by group; e.g.,

DT[, c("newcol1","newcol2") := .(sum(a),sum(b)), by=group] 

or, perhaps more elegantly :

DT[,`:=`(newcol1=sum(a),          newcol2=sum(b)), by=group] 

But the iterative multiple RHS, envisaged for a while, where the 2nd expression could use the result from the first, isn't implemented yet (FR#1492). So this will still give an error "newcol1 not found" and need to be done in two steps :

DT[,`:=`(newcol1=sum(a),          newcol2=a/newcol1), by=group]