Using Google Drive API with service account in Google Apps Engine python project with local debugging

Question

for the last couple of days (during the odd hour that I can spend on this project), I've been struggling to start using the Google Drive API with a service account in GAE python, for local debugging.

My setup:

• Eclipse 4.4
• Google App Engine SDK for Python 1.8.0
• Google API client Python GAE 1.1

I've activated (among others) these 3rd party libraries in app.yaml:

- name: pycrypto
  version: latest
- name: ssl
  version: latest

This is what I understand about the setup sofar, in a couple of statements:

• because my app doesn't need to access user files, but a file which is specific to the app, the app should use a 'service account' to own and access the file on Google Drive
• service accounts can authenticate in two ways: (1) by API key and (2) by private key credentials
• when developing a GAE app using the SDK, there are two environments to take into account: the local system (for debugging) and the GAE server (for deployment)
• API key authentication doesn't work (and will never work) when running on the local system, because of the two-legged authentication (don't completely grasp this..., but it seems true)

I really want the local debugging facility, because I'm learning python and learning the google drive interface, so debugging on the server is a great burden.

So I need to get the private key credentials working on the local system. But then I bump into the issue "ImportError: cannot import name SignedJwtAssertionCredentials". Tried almost everything that I found on the web:

• using the python 2.7 runtime and enable the pycrypto library
• upgrading google-api-python-client-gae to 1.1 (which includes this fix)
• installed OpenSSL on my system (but probably didn't succeed in setting the right paths)
• read instructions to install pycrypto locally, but assumed they were outdated

=> my first question, just to understand, is: is it possible at all to authenticate on your local system from within the GAE SDK to the Google Drive API using Python? Maybe the answer is very simply 'no'?

=> if the answer would be 'yes', then would there be an example setup and code sample around for showing the way to achieve this local authentication?

=> the error log (below) seems to suggest that there's still a problem with pycrypto not being available, but the docs quite explicitly say it's included in the Python 2.7 GAE runtime environment.

=> maybe (please confirm) I'm confused by the difference between local and server python setup. When I look in Eclipse into the 'Run Local' PYTHONPATH, it includes (1) my project folders, (2) the google-api-client-python-gae folders (which don't seem to include pycrypto!!), while the GAE runtime does --- what's the difference? and (3) my local Python 2.7 deployment. So what's missing in this local configuration that I need to mimic the server configuration in order to start debugging?

This is my code for authenticating using the private key credentials:

from oauth2client.client import SignedJwtAssertionCredentials
f = file(SERVICE_ACCOUNT_PKCS12_FILE_PATH, 'rb')
key = f.read()
f.close()
credentials = SignedJwtAssertionCredentials(SERVICE_ACCOUNT_EMAIL, key, scope=OAUTH_SCOPE)
http = httplib2.Http()
http = credentials.authorize(http)
return build('drive', 'v2', http=http)

This my error log:

ERROR    2013-06-18 00:59:57,562 dev_appserver_import_hook.py:1251] Third party package Crypto was enabled in app.yaml but not found on import. You may have to download and install it.
ERROR    2013-06-18 00:59:59,255 dev_appserver_import_hook.py:1251] Third party package Crypto was enabled in app.yaml but not found on import. You may have to download and install it.
ERROR    2013-06-18 00:59:59,289 webapp2.py:1552] import_string() failed for 'illustrations.SyncHandler'. Possible reasons are:

- missing __init__.py in a package;
- package or module path not included in sys.path;
- duplicated package or module name taking precedence in sys.path;
- missing module, class, function or variable;

Original exception:

ImportError: cannot import name SignedJwtAssertionCredentials

Debugged import:

- 'illustrations' not found.
Traceback (most recent call last):
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1272, in default_dispatcher
    self.handlers[handler] = handler = import_string(handler)
  File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1850, in import_string
    return getattr(__import__(module, None, None, [obj]), obj)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 1766, in load_module
    return self.FindAndLoadModule(submodule, fullname, search_path)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 1630, in FindAndLoadModule
    description)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 1577, in LoadModuleRestricted
    description)
  File "C:\Users\vic\Dropbox\Development\Eclipse-juno-workspace\Missale\src\illustrations.py", line 6, in <module>
    import drive
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 1766, in load_module
    return self.FindAndLoadModule(submodule, fullname, search_path)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 1630, in FindAndLoadModule
    description)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py", line 1577, in LoadModuleRestricted
    description)
  File "C:\Users\vic\Dropbox\Development\Eclipse-juno-workspace\Missale\src\drive.py", line 6, in <module>
    from oauth2client.client import SignedJwtAssertionCredentials
ImportStringError: import_string() failed for 'illustrations.SyncHandler'. Possible reasons are:

- missing __init__.py in a package;
- package or module path not included in sys.path;
- duplicated package or module name taking precedence in sys.path;
- missing module, class, function or variable;

Original exception:

ImportError: cannot import name SignedJwtAssertionCredentials

Debugged import:

- 'illustrations' not found.

[update] reviewing my question, I think I'll need to to have a closer look in to installing pycrypto locally. If this is the fix, I'm going to give a feedback on this article to request adding a note about discrepancies between the GAE server runtime libraries and the local SDK libraries. And I'll add the installation instructions here as well.

[update2] the SignedJwtAssertionCredentials import problem had gone, but another import problem came up on the tlslite package. I had no clue how to fix this, because the import looked perfectly sane, and I resorted to reconfiguring the whole IDE from scratch. I have now installed another precompiled pycrypto library and I followed the hint in the error message, and converted my .p12 private key file to a .pem file. Note that the .pem file created by openssl contained 4 text lines before the "-----BEGIN", that I had to remove manually in order for the .pem file to be recognized by oauth2client!

[update3] when reconfiguring the IDE from scratch, I overlooked to use the 'old_dev_appserver.py' for running the app locally, instead of the 'dev_appserver.py'. The latter will not enable breakpoints! But it looks like it has something to do with the SignedJwtAssertionCredentials import problem. Using 'dev_appserver.py', I do not have the import problem (but no breakpoints), and using 'old_dev_appserver.py', I can reproduce the import problem. So the 'old_dev_appserver.py' may have been part of the problem all along!

Answer 1

Yes - you can use SignedJwtAssertionCredentials to connect to Google Services. I wrote code which do that at one time and it work great. Using that class is one problem. On GAE is provided PyCrypto library like you wrote but on local you must install it manually. GAE have his own changed version of this library so they didn't provide her source code. Second problem with that library is that she is compiled for specified machine which will use her. If you look in source code of that class you will see that she needs some dependencies to work. If they didn't meet the requirements of that class she will be not accessible from outside code.

About problems with import in old and new dev_server i can't write anything because i don't know anything about that.