Structure of lists in foreach package

Question

I have found a feature/bug in the foreach package, which I do not understand. Perhaps someone can explain me this behaviour:

I created a for-loop with the foreach package (I use them together with mutlicore calculations, but here just in a sequentiell example, the bug appears in both variants). This loop runs r times. In every run a list with c entries is returned. So I expect a list with r entries, and every entry consists of c lists.

My code was the following one:

library(foreach)

clusters <- 10
runs <- 100

temp <- foreach(r = 1:runs,
                .combine = 'list',
                .multicombine = TRUE) %do% {

              signal_all <- lapply(1:clusters, function(x){

                return(1)

              })

              return(signal_all)
            } ## end do

With this code, all works as expected, see the following picture:

But when increasing runs <- 101, the output temp is this:

The expected list structure is destroyed. But when commenting out the line .combine = 'list' all works as expected.

library(foreach)

clusters <- 10
runs <- 100

temp <- foreach(r = 1:runs,
                .multicombine = TRUE) %do% {

              signal_all <- lapply(1:clusters, function(x){

                return(1)

              })

              return(signal_all)
            } ## end do

Can someone explain this behaviour? Thanks for any help!

Answer 1

Meanwhile I have found a solution.

The foreach function knows that some comine-functions (e.g. c or cbind) take many arguments, and will call them with up to 100 arguments (by default) in order to improve performance. With the argument .maxcombine you can set them manually.

library(foreach)

clusters <- 10
runs <- 101

temp <- foreach(r = 1:runs,
                .combine = 'list',
                .maxcombine = runs,
                .multicombine = T) %do% {

              signal_all <- lapply(1:clusters, function(x){

                return(1)

              })

              return(signal_all)
            } ## end do