Python HTTP debuger

Question

I would like to set some debugging command (like import ipdb; ipdb.set_trace()) that would run debugger in jupyter (I would have to run a HTTP server). Does anybody know about something like this?

Context: I have a long running tasks that are processed by a scheduler (not interactive mode). I would like to be able to debug such a task while running it the same way.

Answer 1

I need to run code in "detached" (not interactive). And when some error is detected I would like to run debugger. That's why I've been thinking about remote debugger/jupyter notebook or whatever. So - by default there is no debugging session - so I think that PyCharm remote debugger is not a case.

Contrary to what you might seem to think here, you do not really need to run the code in a "debugging session" to use remote debugging.

Try the following:

• Install pydevd in the Python environment for your "detached" code:

  pip install pydevd
  

• Within the places in that code, where you would have otherwise used pdb.set_trace, write

  import pydevd; pydevd.settrace('your-debugger-hostname-or-ip')
  

Now whenever your code hits the pydevd.settrace instruction, it will attempt to connect to your debugger server.

You may then launch the debugger server from within Eclipse PyDev or Pycharm, and have the "traced" process connect to you ready for debugging. Read here for more details.

It is, of course, up to you to decide what to do in case of a connection timeout - you can either have your process wait for the debugger forever in a loop, or give up at some point. Here is an example which seems to work for me (ran the service on a remote Linux machine, connected to it via SSH with remote port forwarding, launched the local debug server via Eclipse PyDev under Windows)

import pydevd
import socket
from socket import error

def wait_for_debugger(ex, retries=10):
    print("Bam. Connecting to debugger now...")
    while True:
        try:
            pydevd.settrace()
            break
        except SystemExit:
            # pydevd raises a SystemExit on connection failure somewhy
            retries -= 1
            if not retries: raise ex
            print(".. waiting ..")

def main():
    print("Hello")
    world = 1
    try:
        raise Exception
    except Exception as ex:
        wait_for_debugger(ex)

main()

It seems you should start the local debug server before enabling port forwarding, though. Otherwise settrace hangs infinitely, apparently believing it has "connected" when it really hasn't.

There also seems to be a small project named rpcpdb with a similar purpose, however I couldn't get it to work right out of the box so can't comment much (I am convinced that stepping through code in an IDE is way more convenient anyway).