I want to apply a function ("foo" for this explanation) to tranform a vector of data into another value. This function takes the data as an input, and needs submit forms to webpages. Sometimes, this goes quickly, and other times, it can a long time. I would like to run the for loop (or equivalent apply function) in a way that skips over the items that take too long. I have tried to limit the time the loop runs before skipping to the next to 5 seconds using the following:
pb <- txtProgressBar(min = 1, max = 100, style = 3)
storage <- matrix(nrow = sample.length, ncol = 2)
for(i in 1:100){
s <- Sys.time()
storage[i,] <- try(foo(data.vec[i]), TRUE)
if (Sys.time() - s >5) {next}
# update progress bar
setTxtProgressBar(pb, i)
}
close(pb)
I think that I must not be understanding how to apply the 'next' condition in a for loop. have searched to find a clearer explanation, but not getting any luck here.
For loops can be slow if you are incorrectly growing objects or you have a very fast interior of the loop and the entire thing can be replaced with a vectorized operation. Otherwise you're probably not losing too much efficiency, as the apply family of functions are performing for loops on the inside, too.
Loops are slower in R than in C++ because R is an interpreted language (not compiled), even if now there is just-in-time (JIT) compilation in R (>= 3.4) that makes R loops faster (yet, still not as fast). Then, R loops are not that bad if you don't use too many iterations (let's say not more than 100,000 iterations).
R loops are not too slow compared to other programming languages like python, ruby etc… But Yes they are slower than the vectorized code. You can get a faster code by doing more with vectorization than a simple loop.
withTimeout()
from package R.utils
, in concert with tryCatch()
, might provide a cleaner solution.
For example:
require(R.utils)
for(i in 1:5) {
tryCatch(
expr = {
withTimeout({Sys.sleep(i); cat(i, "\n")},
timeout = 3.1)
},
TimeoutException = function(ex) cat("Timeout. Skipping.\n")
)
}
# 1
# 2
# 3
# Timeout. Skipping.
# Timeout. Skipping.
In the artificial example above:
The first argument to withTimeout()
contains the code to be evaluated within each loop.
The timeout
argument to withTimeout()
sets the time limit in seconds.
The TimeoutException
argument to tryCatch()
takes a function that is to be executed when an iteration of the loop is timed out.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With