Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Median returning an error when using data.table in R

Tags:

r

data.table

I have the following dataset

> head(DT)
    V1 V2 V3   V4   V5     V6 V7
1:  2  1  2 0.91 0.02 880.00  1
2:  3  2  1 0.02 0.00   2.24  2
3:  1  1  1 0.15 0.01   3.41  3
4:  1  2  1 3.92 0.05 268.67  2
5:  1  1  2 0.10 0.01   1.59  3
6:  0  1  1 1.20 0.04   1.43  3

> sapply(DT, class)
       V1        V2        V3        V4        V5        V6        V7 
"integer" "integer" "integer" "numeric" "numeric" "numeric"  "factor" 

which expands for thousands of rows. I am trying to calculate the median values of V1-V6 within the 8 groups defined by the factor Variable V7

> levels(DT$V7)
[1] "1" "2" "3" "4" "5" "6" "7" "8"

At the moment I am using the following command, which returns an error:

> DT[, lapply(.SD, median), by = V7]
 Error in `[.data.table`(DF, , lapply(.SD, median), by = V7) : 
 Column 1 of result for group 4 is type 'integer' but expecting type 'double'. Column types must be consistent for each group.

I read somewhere that a way around this was using as.double(median(X)). But this works for individual columns: DT[, as.double(median(X)), by = V7], but not for when considering all columns: DT[, lapply(.SD, as.double(median)), by = V7] (as expected, because you have to pass an input to median)

I can get around by using aggregate

> aggregate(DT[,c(1:6), with = FALSE], by = list(DF$V7), FUN = median)
  Group.1 V1 V2 V3     V4   V5      V6
   1       1  0  1  1  1.285 0.04 401.500
   2       2  1  2  1  3.565 0.06   6.400
   3       3  0  1  1  0.360 0.03  11.200
   4       4  1  1  1 74.290 0.26 325.960
   5       5  2  1  0  1.145 0.04   1.415
   6       6  0  1  1 10.100 0.18  93.000
   7       7  1  1  0  0.740 0.04   1.080
   8       8  1  1  0  7.970 0.40   0.050

But I'd like to know if there is a way to solve the error described above and do this calculation using data.table

like image 420
Javier Avatar asked Oct 16 '14 09:10

Javier


1 Answers

median is unusual because it can return different types of return values for the same input type:

The default method returns a length-one object of the same type as x, except when x is integer of even length, when the result will be double.

However, data.table needs a consistent return value type. You have two possibilities:

Convert all columns to numeric:

DT[, paste0("V", 1:6) := lapply(.SD, as.numeric), by = V7]

Or convert the return value of median:

DT[, lapply(.SD, function(x) as.numeric(median(x))), by = V7]
like image 179
Roland Avatar answered Sep 24 '22 08:09

Roland