Python multiprocessing daemon vs non-daemon vs main

Question

I'm learning multiprocessing in Python while I found this odd behaviour between daemon and non-daemon process with respect to the main process. My Code:

import multiprocessing
import time

def worker(name,num):
    print name, 'Starting'
    time.sleep(num)
    print name, 'Exiting'

if __name__ == '__main__':
    print 'main starting'
    p1=multiprocessing.Process(target=worker, args=("p1",7,))
    p2=multiprocessing.Process(target=worker, args=("p2",5,))
    p2.daemon=True
    p1.start()
    p2.start()
    time.sleep(4)
    print 'main exiting'

The output I'm getting is:

main starting
p1 Starting
p2 Starting
main exiting
p1 Exiting

Expected Output:

main starting
p1 Starting
p2 Starting
main exiting
p2 Exiting
p1 Exiting

After few searches, I found this answer and inserted the following line to my code.

logger = multiprocessing.log_to_stderr(logging.INFO)

And the output I got is,

main starting
[INFO/p1] child process calling self.run()
p1 Starting
[INFO/p2] child process calling self.run()
p2 Starting
main exiting
[INFO/MainProcess] process shutting down
[INFO/MainProcess] calling terminate() for daemon p2
[INFO/MainProcess] calling join() for process p2
[INFO/MainProcess] calling join() for process p1
p1 Exiting
[INFO/p1] process shutting down
[INFO/p1] process exiting with exitcode 0

As per my understanding,

• When a main process exits, it terminates all of its child daemon processes. (Which can be seen here)
• A main process can not exit before all of its child non-daemon processes exit.

But here, why the main process is trying to shut down before p1 exits?

• p1 starts and sleeps for 7 seconds.
• p2 starts and sleeps for 5 seconds.
• main process sleeps for 4 seconds.
• main process wakes up after 4 seconds and waits for p1 to exit.
• p2 wakes up after 5 seconds and exits.
• p1 wakes up after 7 seconds and exits.
• main process exits.

Wouldn't be the above a normal time-line for the above program?

Can anyone please explain what's happening here and why?

EDIT

After adding the line p1.join() at the end of the code, I'm getting the following output:

main starting
[INFO/Process-1] child process calling self.run()
p1 Starting
[INFO/Process-2] child process calling self.run()
p2 Starting
main exiting
p2 Exiting
[INFO/Process-2] process shutting down
[INFO/Process-2] process exiting with exitcode 0
p1 Exiting
[INFO/Process-1] process shutting down
[INFO/Process-1] process exiting with exitcode 0
[INFO/MainProcess] process shutting down

Answer 1

When you see

[INFO/MainProcess] calling join() for process p1

it means the main process has not exited yet -- it's in the process of shutting down, but of course will not be done shutting down until that join returns... which will happen only once the joined process is done.

So the timeline is indeed as you expect it to be -- but since that join to p1 is the very last think the main process ever does, you don't see that in the output or logging from it. (main has taken all the termination-triggered action, but as a process it's still alive until then).

To verify, use (on a Unixy system) ps from another terminal while running this (perhaps with slightly longer delays to help you check): you will never see a single Python process running out of this complex -- there will be two (main and p1) until the end.