Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Running foreach without returning any value in R

I have a function doSomething() which runs in a foreach loop and as a result saves some calculations as .csv files. Hence I have no need for a return value of foreach, in fact I don't want a return value because it clutters my memory to the point where I cannot run as many iterations as I would want to.

How can I force foreach to not have a return value, or delete the return values of the iterations?

Here is a minimal example that illustrates my problem:

cl <- parallel::makePSOCKcluster(1)
doParallel::registerDoParallel(cl)

"%dopar%" <- foreach::"%dopar%"

doSomething <- function () {
  a <- as.numeric(1L)
}

foreach::foreach (i = 1:4) %dopar% {

  doSomething()

}

The output is:

[[1]]
[1] 1

[[2]]
[1] 1

[[3]]
[1] 1

[[4]]
[1] 1
like image 606
elmo Avatar asked Feb 05 '20 08:02

elmo


People also ask

Can foreach return a value?

The forEach() method is an iterative method. It calls a provided callbackFn function once for each element in an array in ascending-index order. Unlike map() , forEach() always returns undefined and is not chainable.

What is Dopar R?

foreach. Description %do% and %dopar% are binary operators that operate on a foreach object and an R expression. The expression, ex, is evaluated multiple times in an environment that is created by the foreach. object, and that environment is modified for each evaluation as specified by the foreach object.

Does R support a foreach loop?

The foreach package provides a new looping construct for executing R code repeatedly.


2 Answers

Parallel computing in R works (as far as I experienced) such that for each cluster node the memory will be allocated.

That means if you have a big data set which each node needs for calculation, this data will be allocated multiple times. This yields to high RAM consumption. Since you want to write the output in each loop and throw away the result afterwards you can try the rm function and call the garbage collection (for example with gc) in each function call.

This worked for E L M as mention above. Thx for testing!

like image 104
Freakazoid Avatar answered Sep 25 '22 05:09

Freakazoid


From ?foreach:

The foreach and %do%/%dopar% operators provide a looping construct that can be viewed as a hybrid of the standard for loop and lapply function. It looks similar to the for loop, and it evaluates an expression, rather than a function (as in lapply), but it's purpose is to return a value (a list, by default), rather than to cause side-effects.

The line

but it's purpose is to return a value (a list, by default)

Says that this is the intended behaviour of foreach. Not sure how you want to proceed from that...

like image 40
dario Avatar answered Sep 23 '22 05:09

dario