Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to debug when using multiprocessing in pycharm

I am debugging a multiprocess program with anaconda2 in pycharm community edition. It has several background worker processes. The worker process will check the input Queue to retrieve the task without sleep until a task received. In fact, I'm only interested in the main process. But the pycharm debugger always step into the subprocess, it seems that the main process hasn't been working, and the task never sent out. How can I make the debugger out of the subprocess? The worker subprocess looks like this:

class ILSVRC_worker:

...

def run(self):
    cfg_parser = ConfigParser.ConfigParser()
    cfg_parser.read(self.cfg_path)
    data_factory = ILSVRC_DataFactory(cfg_parser)
    logger = mp.log_to_stderr(logging.INFO)
    while True:
        try:
            annotation_path = self.que_in.get(True,0.1)
        except Queue.Empty:
            continue
        if annotation_path is None:
            # to exit the subprocess
            logger.info('exit the worker process')
            break
        ...
like image 877
Kai Chen Avatar asked Dec 02 '15 02:12

Kai Chen


1 Answers

I could think of two ways to achieve this but unfortunately I think it won't be possible with the community edition.

  • If you have the PID of the process you could try attaching to it by using the Tools>Attach to Process.. functionality (I don't know if that is available in the community edition). This is difficult if you use a Pool because you don't know which process the job is assigned to.
  • Another way would be to use a remote debugger and connect to it in the dispatched python process. This is only available in the professional edition

I ended up testing my code without any multiprocessing

like image 152
Elias Avatar answered Oct 30 '22 05:10

Elias