python debug tools for multiprocessing [closed]

Question

I have a python script that works with threads, processes, and connections to a database. When I run my script, python crashes.

I cannot explicitly detect the case in which this happens.

Now I am looking for tools to get more information when python crashes, or a viewer to see all my created processes/connections.

Answer 1

I created a module RemoteException.py that shows the full traceback of a exception in a process. Python2. Download it and add this to your code:

import RemoteException

@RemoteException.showError
def go():
    raise Exception('Error!')

if __name__ == '__main__':
    import multiprocessing
    p = multiprocessing.Pool(processes = 1)
    r = p.apply(go) # full traceback is shown here

OLD ANSWER

I had the problem, too.

this is what i did... a RemoteException to debug multiprocessing calls

RemoteException.py

copy the source and remove line 19: file.write('\nin %s ' % (Process.thisProcess,)) and the line import Process

The problem is: multiprocessing only transfers the exception but looses the traceback. The code below creates an Exception object that saves the traceback. And prints it in the calling process.

In your script you can do something like this:

import RemoteException

def f():
    try:
        # here is code that fails but you do know not where
        pass
    except:
        ty, err, tb = RemoteException.exc_info() # like sys.exc_info but with better message
        raise ty, err, tb

# here follows your multiprocessing call to f