Restart R session without interrupting the for loop

Question

In my for loop, I need to remove RAM. So I delete some objects with rm() command. Then, I do gc() but the RAM still the same

So I use .rs.restartR() instead of gc() and it works: a sufficient part of my RAM is removed after the restart of the R session.

My problem is the for loop which is interrupted after the R restart. Do you have an idea to automatically go on the for loop after the .rs.restartR() command ?

Answer 1

I just stumbled across this post as I'm having a similar problem with rm() not clearing memory as expected. Like you, if I kill the script, remove everything using rm(list=ls(all.names=TRUE)) and restart, the script takes longer than it did initially. However, restarting the session using .rs.restartR() then sourcing again works as expected. As you say, there is no way to 'refresh' the session while inside a loop.

My solution was to write a simple bash script which calls my .r file.

Say you have a loop in R that runs from 1 to 3 and you want to restart the session after each iteration. My bash script 'runR.sh' could read as follows:

  #!/bin/bash        

    for i in {1..3}
    do
      echo "Rscript myRcode.r $i" #check call to r script is as expected
      Rscript myRcode.r $i
    done

Then at the top of 'myRcode.r':

args <- commandArgs()
print(args) #list the command line arguments. 

myvar <- as.numeric(args[6])

and remove your for (myvar in...){}, keeping just the contents of the loop.

You'll see from print(args) that your input from your shell script is the 6th element of the array, hence args[6] in the following line when assigning your variable. If you're passing in a string, e.g. a file name, then you don't need as.numeric of course.

Running ./runR.sh will then call your script and hopefully solve your memory problem. The only minor issue is that you have to reload your packages each time, unlike when using .rs.restartR(), and may have to repeat other bits that ordinarily would only be run once.

It works in my case, I would be interested to hear from other more seasoned R/bash users whether there are any issues with this solution...

Answer 2

By saving the iteration as an external file, and writing an rscript which calls itself, the session can be restarted within a for loop from within rstudio. This example requires the following steps.

#Save an the iteration as a separate .RData file in the working directory. 

iter <- 1

save(iter, file="iter.RData")

Create a script which calls itself for a certain number of iterations. Save the following script as "test_script.R"

###load iteration
library(rstudioapi)

load("iter.RData")

###insert function here.
time_now <- Sys.time()

###save output of function to a file.
save(time_now, file=paste0("time_", iter, ".Rdata"))

###update iteration
iter <- iter+1
save(iter, file="iter.RData")

###restart session calling the script again
if(iter < 5){
restartSession(command='source("test_script.R")')
}

Answer 3

You can make your script recursive by sourcing itself after restarting session.

Make sure the script will take into account the initial status of the loop. So you might have to save the current status of the loop in a .rds file before restarting session. Then call the .rds file from inside the loop after restarting session. This would help you start the loop where it was before restarting r session.

I just found out about this command 'restartSession'. I'm using it because I was also running into memory consumption issues as the garbage collector will not give back the RAM to the OS (Linux).

library(rstudioapi)
restartSession(command = "print('x')")