"ImportError: No module named _ssl" with dev_appserver.py from Google App Engine

Question

---

Background

"In the Python runtime, we've added support for the Python SSL Library, so you can now open secure connections to remote services such as Apple's Push Notification service."

This quote is taken from a recent post on the Google App Engine blog.

---

Implementation

If you want to use native python ssl, you must enable it using the libraries configuration in your application's app.yaml file where you specify the library name "ssl" . . .

These instructions are provided for developers through the Google App Engine documentation.

The following lines have been added to the app.yaml file:

libraries:
- name: ssl
  version: latest

This much is in line with the advice provided through the Google App Engine documentation.

---

Problem

I have tried running my project in three different configurations. Two are working, and one is not.

Working ...

After I upload my application to Google App Engine, and run my project through the live server, everything works fine.

Working ...

When I run my project with manage.py runserver and include the Google App Engine SKD in my PYTHONPATH, everything works fine.

Not Working ...

However, when I run my project with dev_appserver.py, I get the following error:

ImportError at /
No module named _ssl
Request Method: GET
Request URL:    http://localhost:8080/
Django Version: 1.4.3
Exception Type: ImportError
Exception Value:    
No module named _ssl
Exception Location: /usr/local/lib/google_appengine_1.7.7/google/appengine/tools/devappserver2/python/sandbox.py in load_module, line 856
Python Executable:  /home/rbose85/Code/venvs/appserver/bin/python
Python Version: 2.7.3
Python Path:    
['/home/rbose85/Code/product/site',
 '/usr/local/lib/google_appengine_1.7.7',
 '/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
 '/usr/local/lib/google_appengine_1.7.7',
 '/usr/local/lib/google_appengine_1.7.7',
 '/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
 '/usr/local/lib/google_appengine_1.7.7',
 '/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
 '/home/rbose85/Code/venvs/appserver/lib/python2.7',
 '/home/rbose85/Code/venvs/appserver/lib/python2.7/lib-dynload',
 '/usr/lib/python2.7',
 '/usr/local/lib/google_appengine',
 u'/usr/local/lib/google_appengine_1.7.7/lib/django-1.4',
 u'/usr/local/lib/google_appengine_1.7.7/lib/ssl-2.7',
 u'/usr/local/lib/google_appengine_1.7.7/lib/webapp2-2.3',
 u'/usr/local/lib/google_appengine_1.7.7/lib/webob-1.1.1',
 u'/usr/local/lib/google_appengine_1.7.7/lib/yaml-3.10']
Server time:    Wed, 24 Apr 2013 11:23:49 +0000

Answer 1

For the current GAE version (1.8.0 at least until 1.8.3), if you want to be able to debug SSL connections in your development environment, you will need to tweak a little bit the gae sandbox:

• add "_ssl" and "_socket" keys to the dictionary _WHITE_LIST_C_MODULES in /path-to-gae-sdk/google/appengine/tools/devappserver2/python/sandbox.py
• Replace the socket.py file provided by google in /path-to-gae-sdk/google/appengine/dis27 from the socket.py file from your Python framework.

IMPORTANT: Tweaking the sandbox environment might end up with functionality working on your local machine but not in production (for example, GAE only supports outbound sockets in production). I will recommend you to restore your sandbox when you are done developing that specific part of your app.

Answer 2

The solution by jmg works, but instead of changing the sdk files, you could monkey patch the relevant modules.

Just put something like this on the beginning of your project setup.

# Just taking flask as an example
app = Flask('myapp')

if environment == 'DEV':
    import sys

    from google.appengine.tools.devappserver2.python import sandbox
    sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']

    from lib import copy_of_stdlib_socket.py as patched_socket

    sys.modules['socket'] = patched_socket
    socket = patched_socket

Answer 3

I had to use a slightly different approach to get this working in CircleCI (unsure what peculiarity about their venv config caused this):

appengine_config.py

import os

if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    import imp
    import os.path
    import inspect
    from google.appengine.tools.devappserver2.python import sandbox

    sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']
    # Use the system socket.

    real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
    psocket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
    imp.load_source('socket', psocket)