To make my code more "pythonic" and faster, I use multiprocessing
and a map function to send it a) the function and b) the range of iterations.
The implanted solution (i.e., calling tqdm
directly on the range tqdm.tqdm(range(0, 30))
) does not work with multiprocessing (as formulated in the code below).
The progress bar is displayed from 0 to 100% (when python reads the code?) but it does not indicate the actual progress of the map function.
How can one display a progress bar that indicates at which step the 'map' function is ?
from multiprocessing import Pool
import tqdm
import time
def _foo(my_number):
square = my_number * my_number
time.sleep(1)
return square
if __name__ == '__main__':
p = Pool(2)
r = p.map(_foo, tqdm.tqdm(range(0, 30)))
p.close()
p.join()
Any help or suggestions are welcome...
Solution found. Be careful! Due to multiprocessing, the estimation time (iteration per loop, total time, etc.) could be unstable, but the progress bar works perfectly.
Note: Context manager for Pool
is only available in Python 3.3+.
import time
from multiprocessing import Pool
from random import randint
from tqdm import tqdm
def _foo(my_number):
square = my_number * my_number
time.sleep(randint(1, 2) / 2)
return square
if __name__ == "__main__":
max_ = 30
with Pool(processes=2) as p, tqdm(total=max_) as pbar:
for result in p.imap(_foo, range(0, max_)):
pbar.update()
pbar.refresh()
# do something with `result`