Use tqdm with concurrent.futures?

Question

I have a multithreaded function that I would like a status bar for using tqdm. Is there an easy way to show a status bar with ThreadPoolExecutor? It is the parallelization part that is confusing me.

import concurrent.futures  def f(x):     return f**2  my_iter = range(1000000)  def run(f,my_iter):     with concurrent.futures.ThreadPoolExecutor() as executor:         function = list(executor.map(f, my_iter))     return results  run(f, my_iter) # wrap tqdr around this function? 

Answer 1

You can wrap tqdm around the executor as the following to track the progress:

list(tqdm(executor.map(f, iter), total=len(iter))

Here is your example:

import time   import concurrent.futures from tqdm import tqdm  def f(x):     time.sleep(0.001)  # to visualize the progress     return x**2  def run(f, my_iter):     with concurrent.futures.ThreadPoolExecutor() as executor:         results = list(tqdm(executor.map(f, my_iter), total=len(my_iter)))     return results  my_iter = range(100000) run(f, my_iter) 

And the result is like this:

16%|██▏           | 15707/100000 [00:00<00:02, 31312.54it/s]