ImportError: No module named google.protobuf

Question

I am following this guide and using the exact sample of addressbook.proto.

Post the content of compiler generated addressbook_pb2.py file as well. When I run the following simple program, there is error saying, cannot find google.protobuf, any ideas how to resolve this issue? Thanks.

BTW, using Python 2.7 on Mac OSX.

from addressbook_pb2 import Person

p = Person()
p.email = "abc"

print p.email

Here is the automated generated file addressbook_pb2.py,

# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: addressbook.proto

import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
from google.protobuf import descriptor_pb2
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor.FileDescriptor(
  name='addressbook.proto',
  package='tutorial',
  syntax='proto2',
  serialized_pb=_b('\n\x11\x61\x64\x64ressbook.proto\x12\x08tutorial\"\xda\x01\n\x06Person\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\n\n\x02id\x18\x02 \x02(\x05\x12\r\n\x05\x65mail\x18\x03 \x01(\t\x12+\n\x05phone\x18\x04 \x03(\x0b\x32\x1c.tutorial.Person.PhoneNumber\x1aM\n\x0bPhoneNumber\x12\x0e\n\x06number\x18\x01 \x02(\t\x12.\n\x04type\x18\x02 \x01(\x0e\x32\x1a.tutorial.Person.PhoneType:\x04HOME\"+\n\tPhoneType\x12\n\n\x06MOBILE\x10\x00\x12\x08\n\x04HOME\x10\x01\x12\x08\n\x04WORK\x10\x02\"/\n\x0b\x41\x64\x64ressBook\x12 \n\x06person\x18\x01 \x03(\x0b\x32\x10.tutorial.Person')
)
_sym_db.RegisterFileDescriptor(DESCRIPTOR)



_PERSON_PHONETYPE = _descriptor.EnumDescriptor(
  name='PhoneType',
  full_name='tutorial.Person.PhoneType',
  filename=None,
  file=DESCRIPTOR,
  values=[
    _descriptor.EnumValueDescriptor(
      name='MOBILE', index=0, number=0,
      options=None,
      type=None),
    _descriptor.EnumValueDescriptor(
      name='HOME', index=1, number=1,
      options=None,
      type=None),
    _descriptor.EnumValueDescriptor(
      name='WORK', index=2, number=2,
      options=None,
      type=None),
  ],
  containing_type=None,
  options=None,
  serialized_start=207,
  serialized_end=250,
)
_sym_db.RegisterEnumDescriptor(_PERSON_PHONETYPE)


_PERSON_PHONENUMBER = _descriptor.Descriptor(
  name='PhoneNumber',
  full_name='tutorial.Person.PhoneNumber',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='number', full_name='tutorial.Person.PhoneNumber.number', index=0,
      number=1, type=9, cpp_type=9, label=2,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='type', full_name='tutorial.Person.PhoneNumber.type', index=1,
      number=2, type=14, cpp_type=8, label=1,
      has_default_value=True, default_value=1,
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[],
  enum_types=[
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=128,
  serialized_end=205,
)

_PERSON = _descriptor.Descriptor(
  name='Person',
  full_name='tutorial.Person',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='name', full_name='tutorial.Person.name', index=0,
      number=1, type=9, cpp_type=9, label=2,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='id', full_name='tutorial.Person.id', index=1,
      number=2, type=5, cpp_type=1, label=2,
      has_default_value=False, default_value=0,
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='email', full_name='tutorial.Person.email', index=2,
      number=3, type=9, cpp_type=9, label=1,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='phone', full_name='tutorial.Person.phone', index=3,
      number=4, type=11, cpp_type=10, label=3,
      has_default_value=False, default_value=[],
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[_PERSON_PHONENUMBER, ],
  enum_types=[
    _PERSON_PHONETYPE,
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=32,
  serialized_end=250,
)


_ADDRESSBOOK = _descriptor.Descriptor(
  name='AddressBook',
  full_name='tutorial.AddressBook',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='person', full_name='tutorial.AddressBook.person', index=0,
      number=1, type=11, cpp_type=10, label=3,
      has_default_value=False, default_value=[],
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[],
  enum_types=[
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=252,
  serialized_end=299,
)

_PERSON_PHONENUMBER.fields_by_name['type'].enum_type = _PERSON_PHONETYPE
_PERSON_PHONENUMBER.containing_type = _PERSON
_PERSON.fields_by_name['phone'].message_type = _PERSON_PHONENUMBER
_PERSON_PHONETYPE.containing_type = _PERSON
_ADDRESSBOOK.fields_by_name['person'].message_type = _PERSON
DESCRIPTOR.message_types_by_name['Person'] = _PERSON
DESCRIPTOR.message_types_by_name['AddressBook'] = _ADDRESSBOOK

Person = _reflection.GeneratedProtocolMessageType('Person', (_message.Message,), dict(

  PhoneNumber = _reflection.GeneratedProtocolMessageType('PhoneNumber', (_message.Message,), dict(
    DESCRIPTOR = _PERSON_PHONENUMBER,
    __module__ = 'addressbook_pb2'
    # @@protoc_insertion_point(class_scope:tutorial.Person.PhoneNumber)
    ))
  ,
  DESCRIPTOR = _PERSON,
  __module__ = 'addressbook_pb2'
  # @@protoc_insertion_point(class_scope:tutorial.Person)
  ))
_sym_db.RegisterMessage(Person)
_sym_db.RegisterMessage(Person.PhoneNumber)

AddressBook = _reflection.GeneratedProtocolMessageType('AddressBook', (_message.Message,), dict(
  DESCRIPTOR = _ADDRESSBOOK,
  __module__ = 'addressbook_pb2'
  # @@protoc_insertion_point(class_scope:tutorial.AddressBook)
  ))
_sym_db.RegisterMessage(AddressBook)


# @@protoc_insertion_point(module_scope)

Edit 1

Tried pip install protobuf, met with the following error,

Requirement already satisfied (use --upgrade to upgrade): protobuf in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /Users/foo/miniconda2/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from protobuf)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Users/foo/miniconda2/lib/python2.7/site-packages (from protobuf)

Here is the output of python version,

python -V
Python 2.7.11 :: Continuum Analytics, Inc.

Edit 2

Post exact error message,

Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1531, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 938, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Users/foo/personal/featureExtraction/protobuf_test.py", line 1, in <module>
    from addressbook_pb2 import Person
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_bundle/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/foo/personal/featureExtraction/addressbook_pb2.py", line 6, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_bundle/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
ImportError: No module named google.protobuf

Edit 3

error message when import google,

Edit 4

Output of which pip,

which pip
/Users/foo/miniconda2/bin/pip

Output of sys.executable,

/Users/foo/anaconda/bin/python

Edit 5

foo-mn1:featureExtraction foo$ sudo /Users/foo/miniconda2/bin/pip install protobuf
Password:
The directory '/Users/foo/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/foo/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): protobuf in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /Users/foo/miniconda2/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from protobuf)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Users/foo/miniconda2/lib/python2.7/site-packages (from protobuf)
foo-mn1:featureExtraction foo$ sudo /Users/foo/miniconda2/bin/pip install google
The directory '/Users/foo/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/foo/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): google in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in /Users/foo/miniconda2/lib/python2.7/site-packages (from google)

Answer 1

Had the same issue and I resolved it by using :

conda install protobuf

Answer 2

You should run:

pip install protobuf

That will install Google protobuf and after that you can run that Python script.

As per this link.

Answer 3

I also have this issue and have been looking into it for a long time. It seems that there is no such problem on python 3+. The problem is actually on google.protobuf

Solution 1:

pip uninstall protobuf
pip uninstall google
pip install google
pip install protobuf
pip install google-cloud

Solution 2:

create an __init__.py in "google" folder.

cd /path/to/your/env/lib/python2.7/site-packages/google
touch __init__.py

Hopefully it will work.

Answer 4

The reason for this would be mostly due to the evil command pip install google. I was facing a similar issue for google-cloud, but the same steps are true for protobuf as well. Both of our issues deal with a namespace conflict over the 'google' namespace.

If you executed the pip install google command like I did then you are in the correct place. The google package is actually not owned by Google which can be confirmed by the command pip show google which outputs:

 Name: google
 Version: 1.9.3
 Summary: Python bindings to the Google search engine.
 Home-page: http://breakingcode.wordpress.com/
 Author: Mario Vilas
 Author-email: [email protected]
 License: UNKNOWN
 Location: <Path where this package is installed>
 Requires: beautifulsoup4

Because of this package, the google namespace is reserved and coincidentally google-cloud also expects namespace google > cloud and it results in a namespace collision for these two packages.

See in below screenshot namespace of google-protobuf as google > protobuf

Solution :- Unofficial google package need to be uninstalled which can be done by using pip uninstall google after this you can reinstall google-cloud using pip install google-cloud or protobuf using pip install protobuf

FootNotes :- Assuming you have installed the unofficial google package by mistake and you don't actually need it along with google-cloud package. If you need both unofficial google and google-cloud above solution won't work.

Furthermore, the unofficial 'google' package installs with it 'soupsieve' and 'beautifulsoup4'. You may want to also uninstall those packages.

Let me know if this solves your particular issue.

Answer 5

When pip tells you that you already have protobuf, but PyCharm (or other) tells you that you don't have it, it means that pip and PyCharm are using a different Python interpreter. This is a very common issue, especially on a Mac, with no standard Python package management.

The best way to completely eliminate such issues is using a virtualenv per Python project, which is essentially a directory of Python packages and environment variable settings to isolate the Python environment of the project from everything else.

Create a virtualenv for your project like this:

cd project
virtualenv --distribute virtualenv -p /path/to/python/executable

This creates a directory called virtualenv inside your project. (Make sure to configure your VCS (for example Git) to ignore this directory.)

To install packages in this virtualenv, you need to activate the environment variable settings:

. virtualenv/bin/activate

Verify that pip will use the right Python executable inside the virtualenv, by running pip -V. It should tell you the Python library path used, which should be inside the virtualenv.

Now you can use pip to install protobuf as you did.

And finally, you need to make PyCharm use this virtualenv instead of the system libraries. Somewhere in the project settings you can configure an interpreter for the project, select the Python executable inside the virtualenv.