Running GAE GCS on PyCharm under OS X, runtime error "No module named cloudstorage"

Question

I am trying to add Google Cloud Storage functionality to a Python GAE app that is already running with significant functionality. I work entirely within PyCharm on my development computer, which is a Mac running OS X 10.9.5.

I have created a new Python module that contains this statement:

import cloudstorage as gcs

as shown in the sample code at https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/getstarted

When I first added that line, PyCharm said "No module named cloudstorage" in the editor.

I then followed both the "pip" and the "svn" instructions at https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/download to download the GCS Client Library.

In trying to follow those instructions, taking into account my prior experience with this programming environment, I actually tried using "pip" three times:

• Once without the "-t" option, since I've never needed that option with "pip" before
• Once using the "-t" option to specify my application directory's "lib" subdirectory
• Once using: pip install GoogleAppEngineCloudStorageClient -t /Applications/GoogleAppEngineLauncher.app//Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib, since I wasn't sure what the instructions meant by "<your_app_directory/lib>"

As mentioned, I also executed the "svn" command. Then, as mentioned in install python google cloud storage client on Ubuntu 14.04, I ran "cd gcs-client/src" and "sudo python setup.py install". I ran these commands in my user root directory.

After each of those successful but different installations of the GCS Client Library, I looked at the PyCharm editor window for my module, and it always had the same "No module named cloudstorage" error. But as an experiment, I would also try restarting PyCharm, and also try running my app.

At some point, the editor window stopped showing the error. It was not immediately after one of those steps above, but after I would go away to read various webpages and then come back to look at the error again. I don't know which of the installations was the one that got rid of the error message in the PyCharm editor.

In any case, whenever I try to run the app (again, inside PyCharm), I always get the runtime error "ImportError: No module named cloudstorage" on the same import statement.

The Run/Debug Configuration page for this app has both "Add content roots to PYTHONPATH" and "Add source roots to PYTHONPATH" checked.

Of course the main help I want is how to get past the "No module named cloudstorage" runtime error, even though the import statement no longer shows an error.

I think I also have as many as three spurious versions of the GCS Client Library. I'm much more concerned with getting past "Module not found", since it's a show-stopper, but if you have any idea how I can delete the spurious versions so that they're not just lying around, I'd be most grateful for that help as well.

Answer 1

If the "cloudstorage" directory is at <app>/lib/cloudstorage, then the import statement has to specify "lib":

import lib.cloudstorage

In my case, it's:

import lib.cloudstorage as gcs

By the way, the <app>/lib/GoogleAppEngineCloudStorageClient-1.9.5.0-py2.7.egg-info directory does not seem to be needed and can be deleted.

Answer 2

Actually, you also need to

touch __init__.py

in the lib directory. This will make the cloudstorage module visible to the "import lib.cloudstorage" command.

Dear Google: The distributions should include this file (or the procedure should account for it), and the demo script should be changed to reflect the expected usage. But more importantly why are you distributing/PROLIFERATING library code like this??!!! Why is this not distributed via gcloud? How am I ever going to pick up a patch for this library?

Answer 3

The accepted answer's solution

import lib.module_name

definitely can solve the problem. But I don't like add lib in front of every single module and happened to see how Google suggest import third party libs like this.

appengine will automatically run a file called appengine_config.py. So you can create such a file and put

from google.appengine.ext import vendor
vendor.add('lib')

inside that file. This will help you tell appengine to find dependencies in that folder, so you can simply write

import cloudstorage as gcs

Answer 4

I solve the missing module issue by adding the following to my main application file (main.py):

import os
import sys
sys.path.append(os.path.join(os.path.dirname(__file__), "lib"))

I think this is the way Guido intended. Now my code simply says import cloudstorage as gcs. None of the lib.cloudstorage or lib/__init__.py business.

From https://stackoverflow.com/a/37645984/1740008