Using Two Python Libraries with Conflicting Names

Question

I want to use two Python libraries (Google's Cloud Library, and their Cloud SDK) in a single application, but they have conflicting names (they both use google in their base import names and do not use relative imports internally). How can I use them in a single app?

Changing the library's code to use proper relative imports is not practical. Also, I know I can use virtualenv to access these libraries from separate python applications, but how do I access them from within the same python app?

Details of the Naming Conflict

Here are some of the details on the import. When I import a module from the Cloud Library (I run import google.cloud.datastore), there is an exception about another import within that library:

>>> import libs.google.cloud.datastore
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "C:\[ProjectDIR]\libs\google\cloud\datastore\__init__.py", line 52, in <module>
    from google.cloud.datastore.batch import Batch
ImportError: No module named cloud.datastore.batch

The library is trying to do an absolute import, rather than a relative one. The reason that the Google Cloud Library cannot import google.cloud.datastore.batch is because google is already defined in the SDK, there is a naming conflict:

>>> print google.__path__
['C:\\Program Files (x86)\\Google\\Cloud SDK\\google-cloud-sdk\\platform\\google_appengine\\google']

Because the Cloud Library uses absolute imports, and the name google is already defined in the SDK, then the import fails.

Answer 1

The google packages take care to register themselves as a namespace package. With a properly set up sys.path there is no conflict here.

You need to set up your library environment correctly. Add a appengine_config.py file in the root of your project with:

from google.appengine.ext import vendor

# Add any libraries installed in the "lib" folder.
vendor.add('lib')

This adds the lib subdirectory in the right location of sys.path. See the Installing a third-party library section in the Developing Python Apps on App Engine How-To.

From here on out imports of google.cloud just work:

$ ls -1d lib *.py *.yaml
app.yaml
appengine_config.py
lib
main.py
$ pip install -t lib google-cloud
# installing into the lib subdirectory
$ cat main.py
import google
from google.cloud import datastore
from google.appengine.api import memcache
import os.path

here = os.path.dirname(os.path.abspath(__file__))

def app(*args, **kwargs):
    return '''
google: {}<br />
google.cloud.datastore: {}<br />
google.appengine.api.memcache: {}'''.format(
        os.path.relpath(google.__file__, here),
        os.path.relpath(datastore.__file__, here),
        os.path.relpath(memcache.__file__, here))

and in the browser I am served:

google: ../google-cloud-sdk/platform/google_appengine/google/__init__.py
google.cloud.datastore: lib/google/cloud/datastore/__init__.pyc
google.appengine.api.memcache: ../google-cloud-sdk/platform/google_appengine/google/appengine/api/memcache/__init__.pyc