Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Timing out a multiprocessing function

Tags:

python

timeout

I need to set a time limit on a python function which use some multiprocessing stuff (I don't know if it matters). Something like this:

function(a_list):

     p1 = Process(a_list[0:len(a_list/2)])
     p2 = Process(a_list[len(a_list)/2: len(a_list)])

     //start and join p1, p2

I look around the net and I found a time out decorator but it looks quite tricky and verbose (I'm newbie on decorators). What I'd want is a simple thing.

EDIT:

I think I made it too simple. My program iterates over the above function and store result in a list something like this:

while(something):

     retval = function(some_list)  # here I need the time out thing

     # if function timed out then skip

     ris_list.append(retval)
like image 305
user1017220 Avatar asked Dec 22 '11 09:12

user1017220


People also ask

How do you stop a multiprocessing process?

A process can be killed by calling the Process. kill() function. The call will only terminate the target process, not child processes. The method is called on the multiprocessing.

How do you make a timeout in Python?

Then it's as simple as this to timeout a test or any function you like: @timeout(5.0) # if execution takes longer than 5 seconds, raise a TimeoutError def test_base_regression(self): ... Be careful since this does not terminate the function after timeout is reached!

How do I close a multiprocessing queue?

Close a Python Multiprocessing Queue If you want no process should write into a multiprocessing queue, you can close the queue using the close() method. The close() method, when invoked on a multiprocessing queue in any of the processes, closes the queue.

What does multiprocessing Freeze_support () do?

freeze_support() function. Add support for when a program which uses multiprocessing has been frozen to produce a Windows executable. (Has been tested with py2exe, PyInstaller and cx_Freeze.) This function will allow a frozen program to create and start new processes via the multiprocessing.


1 Answers

You should be able to do that with this code:

process.join(timeout)
if process.is_alive():
    process.terminate()

So instead of setting a timeout in the function, you can join with a timeout the process and if the process hasn't finished after that timeout, then terminate it.

like image 122
jcollado Avatar answered Oct 14 '22 15:10

jcollado