Python: Parallel processing of N files in list [closed]

Question

There is list of files which I am processing one by one with some function:

list_of_files = [file_1, file_2, file_3, .... file_n]

# each file we are processing in function function_x(file)

for file in list_of_files:
   function_x(file)

But it takes too long to process one file so I would like to process 4 files in paralel, when any of them is finished, to continue with next form list_of_files

Answer 1

Try to use parallel map:

import multiprocessing
pool = multiprocessing.Pool()
pool.map(function_x, list_of_files)