Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multiprocessing program has AttributeError in Anaconda notebook

I am running a simple "Hello World" program on Windows 7 64 bit with the following specifications:

Python 3.4.3 | Anaconda 2.3.0 (64-bit) | [MSC v.1600 64 bit (AMD64)] IPython 4.0.0

The program:

from multiprocessing import Process, freeze_support

def f():
    print ('hello world!')

if __name__ == '__main__':
    #freeze_support()
    Process(target=f).start()

gives the following error:

[I 15:02:23.855 NotebookApp] Saving file at /uhc/FeatureContributionToK-meansClu
sterWithPC.ipynb
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 106, in spawn_main
    exitcode = _main(fd)
  File "C:\Anaconda3\lib\multiprocessing\spawn.py", line 116, in _main
    self = pickle.load(from_parent)
AttributeError: Can't get attribute 'f' on module '__main__' (built-in)  
like image 365
Sanoj Avatar asked Sep 09 '15 21:09

Sanoj


1 Answers

This is because of the fact that multiprocessing does not work well in the interactive interpreter. The main reason is that there is no fork() function applicable in windows. It is explained on their web page itself.

"Functionality within this package requires that the main module must be importable by the children. This is covered in Programming guidelines however it is worth pointing out here. This means that some examples, such as the multiprocessing.Pool examples will not work in the interactive interpreter."

https://docs.python.org/2/library/multiprocessing.html#windows

This same problem will come if you are using pool function in multiprocessing. It is solved in this post . You can hence use that method for executing your idea of parallel processing.

Python multiprocessing apply_async never returns result on Windows 7

Hope it is a useful information to you.

like image 73
Arun Sooraj Avatar answered Oct 01 '22 05:10

Arun Sooraj