Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Limiting the time that a function processes in an R for loop

Tags:

loops

for-loop

r

I want to apply a function ("foo" for this explanation) to tranform a vector of data into another value. This function takes the data as an input, and needs submit forms to webpages. Sometimes, this goes quickly, and other times, it can a long time. I would like to run the for loop (or equivalent apply function) in a way that skips over the items that take too long. I have tried to limit the time the loop runs before skipping to the next to 5 seconds using the following:

pb <- txtProgressBar(min = 1, max = 100, style = 3)
storage <- matrix(nrow = sample.length, ncol = 2)

for(i in 1:100){  
     s <- Sys.time()  
     storage[i,] <- try(foo(data.vec[i]), TRUE)  
     if (Sys.time() - s >5) {next}  
     # update progress bar  
     setTxtProgressBar(pb, i)  
}  
close(pb)  

I think that I must not be understanding how to apply the 'next' condition in a for loop. have searched to find a clearer explanation, but not getting any luck here.

like image 343
exl Avatar asked Dec 28 '11 06:12

exl


People also ask

Why shouldnt you use for loops in R?

For loops can be slow if you are incorrectly growing objects or you have a very fast interior of the loop and the entire thing can be replaced with a vectorized operation. Otherwise you're probably not losing too much efficiency, as the apply family of functions are performing for loops on the inside, too.

WHY DO FOR loops take so long in R?

Loops are slower in R than in C++ because R is an interpreted language (not compiled), even if now there is just-in-time (JIT) compilation in R (>= 3.4) that makes R loops faster (yet, still not as fast). Then, R loops are not that bad if you don't use too many iterations (let's say not more than 100,000 iterations).

Are for loops slow in R?

R loops are not too slow compared to other programming languages like python, ruby etc… But Yes they are slower than the vectorized code. You can get a faster code by doing more with vectorization than a simple loop.


1 Answers

withTimeout() from package R.utils, in concert with tryCatch(), might provide a cleaner solution.

For example:

require(R.utils)

for(i in 1:5) {
    tryCatch(
        expr = {
            withTimeout({Sys.sleep(i); cat(i, "\n")}, 
                         timeout = 3.1)
            }, 
        TimeoutException = function(ex) cat("Timeout. Skipping.\n")
    )
}

# 1 
# 2 
# 3 
# Timeout. Skipping.
# Timeout. Skipping.

In the artificial example above:

  • The first argument to withTimeout() contains the code to be evaluated within each loop.

  • The timeout argument to withTimeout() sets the time limit in seconds.

  • The TimeoutException argument to tryCatch() takes a function that is to be executed when an iteration of the loop is timed out.

like image 73
Josh O'Brien Avatar answered Sep 19 '22 17:09

Josh O'Brien