R doParallel Progress bar to monitor finished jobs

Question

I am trying to write a CRAN package with multithreaded capabilities. I achieved a perfect solution with doSNOW, but the package has been flagged as "superseded" by the CRAN team and they asked me to switch to a doParallel solution. This is fine, however I could not find a way to monitor the number of jobs completed using doParallel in the same way I did with doSNOW. Here is my doSNOW solution:

# Set up parameters
nthreads<-2
nreps<-100
funrep<-function(i){
    Sys.sleep(0.1)
    res<-c(log2(i),log10(i))
    return(res)
}
# doSNOW solution
library(doSNOW)
cl<-makeCluster(nthreads)
registerDoSNOW(cl)
pb<-txtProgressBar(0,nreps,style=3)
progress<-function(n){
    setTxtProgressBar(pb,n)
}
opts<-list(progress=progress)
i<-0
output<-foreach(i=icount(nreps),.combine=c,.options.snow=opts) %dopar% {
    s<-funrep(i)
    return(s)
}
close(pb)
stopCluster(cl)

And here's a doParallel solution as suggested in a previous Stack Overflow post. However, as you can see, it doesn't print progress as the jobs are done, it just does it when the results are combined, at the very end.

# doParallel solution
library(doParallel)
progcombine<-function(){
  count<-0
  function(...) {
    count<<-count+length(list(...))
    setTxtProgressBar(pb,count)
    utils::flush.console()
    c(...)
  }
}
cl <- makeCluster(nthreads)
registerDoParallel(cl)
output<-foreach(i = icount(nreps),.combine=progcombine()) %dopar% {
    funrep(i)
}
stopCluster(cl)

Can you suggest me a solution to monitor job status completion using doParallel or at least without using the superseded doSNOW? Possibly with a progress bar and also possibly with multi-OS capability. Thanks a lot!

Answer 1

I could not find a solution with doParallel (I don't think it supports progress bars for job completion), but maybe you can try the new package pbapply:

# pbapply solution
library(pbapply)
cl<-parallel::makeCluster(nthreads)
invisible(parallel::clusterExport(cl=cl,varlist=c("nreps")))
invisible(parallel::clusterEvalQ(cl=cl,library(utils)))
result<-pblapply(cl=cl,
                 X=1:nreps,
                 FUN=funrep)
parallel::stopCluster(cl)