Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python multiprocessing throws error with argparse and pyinstaller

In my project, I'm using argprse to pass arguments and somewhere in script I'm using multiprocessing to do rest of the calculations. Script is working fine if I call it from command prompt for ex.

"python complete_script.py --arg1=xy --arg2=yz" .

But after converting it to exe using Pyinstaller using command "pyinstaller --onefile complete_script.py" it throws

error

" error: unrecognized arguments: --multiprocessing-fork 1448"

Any suggestions how could I make this work. Or any other alternative. My goal is to create an exe application which I can call in other system where Python is not installed.

Here are the details of my workstation:

Platform: Windows 10
Python : 2.7.13 <installed using Anaconda>
multiprocessing : 0.70a1
argparse:   1.1

Copied from comment:

def main():
     main_parser = argparse.ArgumentParser()
     < added up arguments here>
    all_inputs = main_parser.parse_args()
    wrap_function(all_inputs)


def wrap_function(all_inputs):
    <Some calculation here >
   distribute_function(<input array for multiprocessing>)

def distribute_function(<input array>):
    pool = Pool(process = cpu_count)
    jobs = [pool.apply_async(target_functions, args = (i,) for i in input_array)]
    pool.close()
like image 947
Prabhakar Mishra Avatar asked Sep 21 '17 05:09

Prabhakar Mishra


1 Answers

(A bit late but it can be useful for someone else in the future...)

I had the same problem, after some research I found this multiprocessing pyInstaller recipe that states:

When using the multiprocessing module, you must call

multiprocessing.freeze_support()

straight after the if __name__ == '__main__': line of the main module.

Please read the Python library manual about multiprocessing.freeze_support for more information.

Adding that line of code solved the problem for me.

like image 178
sanzoghenzo Avatar answered Nov 16 '22 10:11

sanzoghenzo