ImportError: Importing the devappserver sandbox module from user application code is not permitted

Question

I'm working on a Google App Engine application written in Python (Standard Environment) and I need to whitelist some additional modules in the development server environment.

I have been doing it using this code in the appengine_config file for a long time, and it worked very well:

from google.appengine.tools.devappserver2.python import sandbox
sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']

As answered in this question, some time ago the Google Cloud SDK was updated and the previous import caused an ImportError. The import just needed to be changed to:

from google.appengine.tools.devappserver2.python.runtime import sandbox

With this change, everything worked well again, until I updated the Cloud SDK to version: 186.0.0.

Now it seems a "SandboxAccessPreventionImportHook" class has been added to the sandbox module so it can't be imported from the App Engine application. This is the error the application is raising:

ImportError: Importing the devappserver sandbox module (google.appengine.tools.devappserver2.python.runtime.sandbox) from user application code is not permitted.

Does anyone have an idea on how to bypass this? Or is there another way to whitelist modules in the development server environment?

Thanks!!!

Answer 1

We had the exact same issue as OP but unfortunately Alex's solution didn't work for us.

While a really really hacky solution this is what worked for us. Please be aware that the modification of the sandbox must be redone after an update.

Modify the sandbox.py file located at {appengine_sdk_dir}/google/appengine/tools/devappserver2/python/runtime/sandbox.py and add _ssl and _socket like in Alex's first example to the _WHITE_LIST_C_MODULES list.

_WHITE_LIST_C_MODULES = [
    // keep existing items
    '_ssl',
    '_socket'
]

Then we removed the imports and sandbox overrides from appengine_config.py.

vendor.add('lib')

if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    import imp
    import os.path
    import inspect

    # Use the system socket.
    real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
    psocket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
    imp.load_source('socket', psocket)

# handle requests_toolbelt's monkeypatch as you see fit.

Let's hope there is a day when this is no longer necessary!

Answer 2

Context

So for me the root of this issue was the dev_appserver.py's inability to send outbound https requests / sockets.

The solution then was to put this in my appengine_config.py:

vendor.add('lib')

if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    import imp
    import os.path
    import inspect
    try:
        from google.appengine.tools.devappserver2.python import sandbox
    except ImportError:
        from google.appengine.tools.devappserver2.python.runtime import sandbox

    sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']
    # Use the system socket.

    real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
    psocket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
    imp.load_source('socket', psocket)
else:
    # Doing this on dev_appserver/localhost seems to cause outbound https requests to fail
    import requests
    from requests_toolbelt.adapters import appengine as requests_toolbelt_appengine

    # Use the App Engine Requests adapter. This makes sure that Requests uses
    # URLFetch.
    requests_toolbelt_appengine.monkeypatch() 

Today I upgraded my cloud sdk and started getting

ImportError: Importing the devappserver sandbox module (google.appengine.tools.devappserver2.python.runtime.sandbox) from user application code is not permitted.

If I remove the _WHITE_LIST_C_MODULES stuff, I then get this error when using the python requests library to make outbound https requests:

  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/urlfetch.py", line 293, in fetch
    return rpc.get_result()
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/urlfetch.py", line 413, in _get_fetch_result
    rpc.check_success()
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success
    self.__rpc.CheckSuccess()
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_rpc.py", line 157, in _WaitImpl
    self.request, self.response)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 222, in MakeSyncCall
    self._MakeRealSyncCall(service, call, request, response)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 241, in _MakeRealSyncCall
    request_pb.set_request(request.Encode())
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/net/proto/ProtocolBuffer.py", line 103, in Encode
    self.Output(e)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/net/proto/ProtocolBuffer.py", line 347, in Output
    self.OutputUnchecked(e)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/urlfetch_service_pb.py", line 481, in OutputUnchecked
    out.putDouble(self.deadline_)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/net/proto/ProtocolBuffer.py", line 592, in putDouble
    a.fromstring(struct.pack("<d", v))
error: required argument is not a float 

I then found that stack trace in this issue https://github.com/requests/requests/issues/4078 which seemed to indicate that this only started happening after python-requests version 2.16.0

Solution

All my third party libs were installed to a folder in my project root called lib using

pip install -t lib

Now, I have lib & localhost_libs and I did:

pip install -t localhost_libs requests==2.16

My appengine_config.py now has this instead:

vendor.add('lib')

if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    vendor.add('localhost_libs')
    import pkg_resources
    pkg_resources.require("requests==2.16.0")
import requests
print "requests.__version__", requests.__version__

from requests_toolbelt.adapters import appengine as requests_toolbelt_appengine

# Use the App Engine Requests adapter. This makes sure that Requests uses
# URLFetch.
requests_toolbelt_appengine.monkeypatch()
print "Appengine config done"

Edit: Modified solution to use pkg_resources and not require a prod_libs folder