I am using a np.vectorize
-ed function and would like to see the progress of the function with tqdm
. However, I have not been able to figure out how to do this.
All the suggestions I have found relate to converting the calculation into a for-loop, or into a pd.DataFrame.
Define a vectorized function which takes a nested sequence of objects or numpy arrays as inputs and returns a single numpy array or a tuple of numpy arrays. The vectorized function evaluates pyfunc over successive tuples of the input arrays like the python map function, except it uses the broadcasting rules of numpy.
Again, some have observed vectorize to be faster than normal for loops, but even the NumPy documentation states: “The vectorize function is provided primarily for convenience, not for performance.
I finally found a method that works to get the tqdm progress bar to update with a np.vectorize function. I wrap the vectorize function using a with
with tqdm(total=len(my_inputs)) as pbar:
my_output = np.vectorize(my_function)(my_inputs)
in my_function() I then add the following lines
global pbar
pbar.update(1)
and voila! I now have a progress bar that updates with each iteration. Only slight performance dip on my code.
Note: when you instantiate the function it might complain that pbar is not yet defined. Simply put a pbar = 0 before you instantiate, and then the function will call the pbar defined by the with
Hope it helps everyone reading here.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With