pylint can't find google.cloud

Question

I've installed the Google Cloud SDK and want the code I'm writing to pass pylint. Unfortunately any time I import anything from google.* I get an error:

E: 10, 0: No name 'cloud' in module 'path/to/my/current/module.google' (no-name-in-module)
E: 10, 0: Unable to import 'google.cloud' (import-error)

Versions:

$: pylint --version
pylint 1.7.0, 
astroid 1.5.0
Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4]

If I put a hook in pylint to print out the sys path I get nothing interesting. The google-cloud-sdk is in /usr/local/lib/python2.7/dist-packages so it should be able to find it.

['/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/dist-packages/pylint-1.7.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/backports.functools_lru_cache-1.3-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/configparser-3.5.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/singledispatch-3.4.0.3-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/editdistance-0.3.1-py2.7-linux-x86_64.egg', '/usr/local/lib/python2.7/dist-packages/mccabe-0.6.1-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/astroid-1.5.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/wrapt-1.10.10-py2.7-linux-x86_64.egg', '/usr/local/lib/python2.7/dist-packages/lazy_object_proxy-1.2.2-py2.7-linux-x86_64.egg', '/usr/lib/python2.7/dist-packages']

Does anyone know why it is looking in my local path for the "google" module and how I can fix it?

Updates with more detail about my environment:

The Google Cloud SDK modules in question are located at:

/usr/local/lib/python2.7/dist-packages/google

if I ls that directory it shows:

api auth cloud gapic gax iam ...

and if I follow down those paths all the modules are in the places I'd expect them to be given the import statements.

However, there are no __init__.py files, which makes me think that they're using implicit namespace packages. So it seems like the relevant question here is: how to make pylint recognize an implicit namespace package? The docs say it should "just work":

https://docs.pylint.org/en/latest/user_guide/run.html?highlight=re

For the record, the same problem shows up when using mypy.

Answer 1

Update (2):

It turns out I may have a solution in a Google App Engine project I am working on:

def fixup_paths(path):
    """Adds GAE SDK path to system path and appends it to the google path
    if that already exists."""
    # Not all Google packages are inside namespace packages, which means
    # there might be another non-namespace package named `google` already on
    # the path and simply appending the App Engine SDK to the path will not
    # work since the other package will get discovered and used first.
    # This emulates namespace packages by first searching if a `google` package
    # exists by importing it, and if so appending to its module search path.
    try:
        import google
        google.__path__.append("{0}/google".format(path))
    except ImportError:
        pass

    sys.path.insert(0, path)

# and then call later in your code:
fixup_paths(path_to_google_sdk)

I really hope that helps!

Update (1):

Here is a possible solution I found: No module named cloud while using google.cloud import bigquery

In that post, the problem is a python 2.7 script is trying to import from google.cloud, and is failing due to the implicit namespace issue. The solution is:

Fortunately in Python 2.7 you can fix this easily by avoiding implicit imports. Just add this to the top of your file:

from __future__ import absolute_import

Original Answer:

Could you please post your import statement, the full path to the google/cloud.py or whatever you are trying to import, the contents of the google module from which you are trying to import, and the contents of your current module please?

Without that information it is hard to explore this issue. My initial hunch is that you are missing an __init__.py, either in your current module or in the google module. As shown in this post, your code can run fine without __init__.py in your module, but if pylint runs from outside your module then it may not see your module properly without that.

The reason I suspect a missing __init__.py in the google module is because in PEP 420 in python 3, they dropped the requirement for __init__.py, and I have seen questions resulting from people trying to import from google modules which are designed for python 3, and which omit the __init__.py that is required in python 2.

If none of those are the case, then maybe the module you are trying to import is located a few directories into the google-cloud-sdk directory that is located at /usr/local/lib/python2.7/dist-packages, as opposed to being an immediate child of the google-cloud-sdk directory, and the python path your code normally sees accounts for this, while the python path that pylint sees does not.