I have the following code
from multiprocessing import Process, Queue
from queue import Empty
from time import sleep
def f(q):
n = 100000000
while n != 100000000 // 2:
n -= 1
q.put("the awkening!")
print("my work here is done")
def main():
q = Queue()
p = Process(target=f, args=(q,))
p.start()
while True:
try:
print(q.get(block=False))
raise systemexit
except Empty:
print("i found nothing :(")
sleep(2)
p.join()
If I add
if __name__ == '__main__':
main()
To the end then use python script_name.py
to run it, everything works fine. However, if I just run the scirpt using python -i script_name.py
then run main()
Python complains:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python34\lib\multiprocessing\spawn.py", line 98, in spawn_main
exitcode = _main(fd)
File "C:\Python34\lib\multiprocessing\spawn.py", line 108, in _main
self = pickle.load(from_parent)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>
The error comes from the child process, the main process runs fine.
This is not a big deal, but I wonder why this happens, also it would be nice if it works in interactive mode
The chunksize argument is the same as the one used by the map() method. For very long iterables using a large value for chunksize can make the job complete much faster than using the default value of 1. — multiprocessing — Process-based parallelism. In the case of Pool.
multiprocessing. freeze_support() This function will allow a frozen program to create and start new processes via the multiprocessing. Process class when the program is frozen for distribution on Windows. If the function is called and the program is not frozen for distribution, then it has no effect.
If we are using the context manager to create the process pool so that it is automatically shutdown, then you can configure the number of processes in the same manner. The number of workers must be less than or equal to 61 if Windows is your operating system.
The multiprocessing documentation discusses this:
Note
Functionality within this package requires that the __main__ module be importable by the children. This is covered in Programming guidelines however it is worth pointing out here. This means that some examples, such as the multiprocessing.Pool examples will not work in the interactive interpreter.
My understanding is that __main__
is defined very differently in the context of an interactive session (as it is associated with the shell, not the file that is running).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With