Unable to debug dev_appserver in Google App Engine

Question

I don't think this issue is unique to PyDev, but to any python debugger.

Using Eclipse and pydev, I'm unable to break on my WSGI handler, in a dev_appserver (The Google app engine development server) process. I'm not 100% sure, but I think this is a regression in GAE 1.7.6 or 1.7.7, as I'm almost certain I was able to debug my code before I upgrade to 1.7.7

It appears that GAE creates a new process ('_python_runtime.py') which isn't controlled by PyDev. I couldn't find any evidence that 'debugging subprocesses' feature is available in PyDev, so right now I'm a little bit lost.

Looking at the GAE code (1.7.7) it appears the subprocess is created in tools/devappserver2/http_runtime.py which calls safe_subprocess.py/start_process.

Goofing around I didn't see any obvious method to either: 1. Tell GAE to server to user handler from the same process. 2. Tell GAE to change the command line of the new project from python _python_runtime.py to python pydev.py ... --file _python_runtime.py (and even then, not sure PyDev will be able to pick it up).

Any suggestion? Is that really a regression?

EDIT (partial answer):
Here is a partial answer. IN SDK 1.7.6 Google App Engine has a completely new server. The server is now a multi-process. The main process spawns sub-processes (called runtime) to run the WSGI handlers. These sub-processes aren't being debugged.

This change caused a lot of reaction in the community, but apparently the GAE community mainly lives in the Google Groups and not in SO (something I wasn't aware of until yesterday). Here is a link to the discussion:

https://groups.google.com/forum/?fromgroups=#!topic/google-appengine/ep5BWYKpQpU

There are basically two solutions:

1. The simple thing to do would be to use the old server, which as of 1.7.7 is still available. Instead of dev_appserver.py, simply launch old_dev_appserver.py. In Eclipse PyDev, go to Debug Configuration..., and replace the 'Main Module' to $(GOOGLE_APP_ENGINE)/old_dev_appserver.py, and launch as if the new server never happened. This solution has obviously the drawback of running an older server, and it is unknown how long this setting is going to be maintained.

2. The second solution is a little bit more involving, and I wasn't able to crack it all the way yet. It is based on remote debugging feature of PyDev, and the ability to tell GAE to run a script at the beginning of the runtime process. So this is how to do it:

   1. Create a script and name it: gae_runtime_startup.py. Put is somewhere (below).
   2. In global PyDev preferences (Window menu -> Preferences -> PyDev -> Interpreter Python -> String Substitution Variables, add a new PYDEV variable, and set the value to the PyDev plugin of eclipse (In my computer this is c:\eclipse\plugins\org.python.pydev_2.7.1.2012100913).
   3. In the project property, add ${PYDEV}/pysrc to the PYTHONPATH. This way, you'll be able to impord pydevd
   4. You need to tell GAE to run gae_runtime_startup.py. Go to the Launcher, and add the following options to the command line (Debug Configurations -> Arguments): --python_startup_script=<full path>/gae_runtime_startup.py --max_server_instances=1
   5. Start the PyDev remote server.
   So, after doing all that, I get a break point within runtime_startup.py running on the runtime process. If I go up the stack, I'll step within the runtime.py sources - so I think I'm in the right direction. And yet, the breakpoints I set in my handlers to not break - so this route is still blocked for me. Any help will be appreciated.

# gae_runtime_startup.py
import pydevd; 
pydevd.settrace()

Some related links:

Google Group discussion: https://code.google.com/p/appengine-devappserver2-experiment/issues/detail?id=28

A document from Google explaining how to debug (my second method): https://docs.google.com/a/london.org.il/document/d/1CCSaRiIWCLgbD3OwmuKsRoHHDfBffbROWyVWWL0ZXN4/edit

A document from PyDev explaining how to set remote debugger. http://pydev.org/manual_adv_remote_debugger.html

See also excellent comment from @Tim Hoffman below.

Answer 1

That was indeed a regression that was fixed in 1.8.3: https://code.google.com/p/googleappengine/wiki/SdkReleaseNotes.