Join a group of python processes with a timeout

Question

I'm using python's multiprocessing library to create several processes.

from multiprocessing import Process
processes = [Process(target=function) for function in FUNCTIONS]
for p in processes:
    p.start()

I want them to run for some duration and then if they have not completed, terminate them.

DURATION = 3600

A bad way to do it is as follows (bad because if the processes finish faster than DURATION, it still waits for all of DURATION):

from time import sleep
sleep(duration)
for p in processes:
    p.join(0)
    p.terminate()

Another bad way to do it (bad because it can possibly take N * DURATION to finish, where N is the number of processes):

for p in processes:
    p.join(DURATION)
    p.terminate()

What is a good way to do this?

Answer 1

I believe this does what you want without any polling required, and it will only wait up to your specified DURATION.

time_waited = 0
then = time.time()
for p in processes:
    if time_waited >= DURATION:
        p.join(0)
        p.terminate()
    p.join(DURATION - time_waited)
    time_waited = time.time() - then