Combine related nested list element vectors into matrices

Question

I have a list where each list element itself holds another list with several names objects. Each of these named object is a vector of the same length. My goal is to efficiently combine the related objects (those of the same name) into a matrix by concatening vectors.

Here's an example of the type of structure I'm working with. However, in the current application it's coming from mclapply as it is a parallelized multilevel model, and I don't think there's a way around getting a list of lists back.

> test=lapply(1:2,function(x){out = list(); out$t=rnorm(3)+x; out$p =rnorm(3)+ x+.1; return(out)})
> test
[[1]]
[[1]]$t
[1] 0.5950165 0.8827352 0.5614947

[[1]]$p
[1] 2.6144102 1.9688743 0.6241944


[[2]]
[[2]]$t
[1] 2.562030 1.832571 3.018756

[[2]]$p
[1] 1.7431969 0.5305784 2.6935106

Here's a crude way to accomplish what I want

> t.matrix = cbind(test[[1]]$t,test[[2]]$t)
> t.matrix
           [,1]     [,2]
[1,]  2.2094525 2.634907
[2,] -0.2822453 2.440666
[3,]  1.1704518 2.483424

but instead I'd like to be able to do this for a very long list (around 1 million elements), and my current solution doesn't scale.

I suppose I could use a for loop, but it seems like there must be a better way to do it with clever use of reduce or unlist or sapply or something like that.

Answer 1

test <- lapply(1:4, function(x) { 
          out = list(); out$t=rnorm(3)+x; out$p =rnorm(3)+ x+.1; return(out)})

do.call(cbind, lapply(test, function(X) X[["t"]]))
## do.call(cbind, lapply(test, "[[", "t"))          ## Or, equivalently 
#           [,1]      [,2]     [,3]     [,4] 
# [1,] 0.7382887 0.9248296 4.205222 5.847823 
# [2,] 3.0321069 3.6806652 3.324739 3.695195 
# [3,] 2.3611483 1.9305901 1.574586 4.287534 

Or, to process both sets of list elements at once:

elems <- c("t", "p")
sapply(elems, function(E) {
     do.call(cbind,
             lapply(test, function(X) {
                 X[[E]]
             }))
}, simplify=FALSE)
# $t
#           [,1]       [,2]     [,3]     [,4]
# [1,] 1.9226614 0.66463844 2.558517 2.743381
# [2,] 3.0026400 0.03238983 2.195404 3.824127
# [3,] 0.9371057 3.54638107 2.968717 2.434471
# 
# $p
#           [,1]     [,2]     [,3]     [,4]
# [1,] 0.8544413 2.942780 4.693698 4.158212
# [2,] 0.7172070 2.381438 4.869630 3.503361
# [3,] 3.1369674 2.464447 2.484968 3.626174

Answer 2

What about using unlist(test, recursive = FALSE). It needs to be done in more than one step though, if you want "p" and "t" to be separate. Here they are together:

temp <- do.call(cbind, unlist(test, recursive = FALSE))
temp
             t         p        t        p
[1,] 0.3735462 2.6952808 2.487429 1.794612
[2,] 1.1836433 1.4295078 2.738325 3.611781
[3,] 0.1643714 0.2795316 2.575781 2.489843

Separating them out is pretty straightforward:

temp[, colnames(temp) %in% "t"]
#              t        t
# [1,] 0.3735462 2.487429
# [2,] 1.1836433 2.738325
# [3,] 0.1643714 2.575781
temp[, colnames(temp) %in% "p"]
#              p        p
# [1,] 2.6952808 1.794612
# [2,] 1.4295078 3.611781
# [3,] 0.2795316 2.489843

Here's the data I used:

set.seed(1)
test <- lapply(1:2, function(x) {
  out = list()
  out$t=rnorm(3)+x
  out$p =rnorm(3)+ x+.1
  return(out)
})