Google Cloud Platform: ImportError: No module named 'google.api.core' on deploy

Question

This is the error I receive while I try to deploy using gcloud app deploy. I have previously successfully deployed the same app. I am able to run the app in local machine, but receives the error on deploy

the traceback:

Updating service [default]...failed.                                           
ERROR: (gcloud.app.deploy) Error Response: [9] 
Application startup error:
[2017-08-25 10:50:23 +0000] [1] [INFO] Starting gunicorn 19.7.1
[2017-08-25 10:50:23 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2017-08-25 10:50:23 +0000] [1] [INFO] Using worker: sync
[2017-08-25 10:50:23 +0000] [7] [INFO] Booting worker with pid: 7
[2017-08-25 10:50:23 +0000] [7] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/env/lib/python3.5/site-packages/gunicorn/arbiter.py", line 578, in spawn_worker
    worker.init_process()
  File "/env/lib/python3.5/site-packages/gunicorn/workers/base.py", line 126, in init_process
    self.load_wsgi()
  File "/env/lib/python3.5/site-packages/gunicorn/workers/base.py", line 135, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/env/lib/python3.5/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/env/lib/python3.5/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/env/lib/python3.5/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/env/lib/python3.5/site-packages/gunicorn/util.py", line 352, in import_app
    __import__(module)
  File "/home/vmagent/app/main.py", line 19, in <module>
    app = bookshelf.create_app(config)
  File "/home/vmagent/app/bookshelf/__init__.py", line 49, in create_app
    model = get_model()
  File "/home/vmagent/app/bookshelf/__init__.py", line 107, in get_model
    from . import model_datastore
  File "/home/vmagent/app/bookshelf/model_datastore.py", line 16, in <module>
    from google.cloud import datastore
  File "/env/lib/python3.5/site-packages/google/cloud/datastore/__init__.py", line 61, in <module>
    from google.cloud.datastore.client import Client
  File "/env/lib/python3.5/site-packages/google/cloud/datastore/client.py", line 33, in <module>
    from google.cloud.datastore.query import Query
  File "/env/lib/python3.5/site-packages/google/cloud/datastore/query.py", line 19, in <module>
    from google.api.core import page_iterator
ImportError: No module named 'google.api.core'
[2017-08-25 10:50:23 +0000] [7] [INFO] Worker exiting (pid: 7)
[2017-08-25 10:50:24 +0000] [1] [INFO] Shutting down: Master
[2017-08-25 10:50:24 +0000] [1] [INFO] Reason: Worker failed to boot.

Answer 1

tl;dr: Upgrade your google-cloud to 0.27, and it should fix things.

---

I believe this is a bug with the new google-cloud dependencies. In my case, google-cloud==0.25 was pulling in these dependencies in its setup.py:

 'google-cloud-core >= 0.24.0, < 0.25dev',
 'google-cloud-datastore >= 1.0.0, < 2.0dev',

Just recently on 8/24 (a day before this issue was filed), the google-cloud-datastore package was updated to 1.3.0.

Unfortunately, google-cloud-datastore 1.3.0 is depending on a newer version of google-cloud-core:

 'google-cloud-core >= 0.27.0, < 0.28dev',

But it seems this versioning conflict is unresolved/unwarned by pip, which uses the older version. But google-cloud-datastore wants to from google.api.core import page_iterator, even though google.api.core, which wasn't added until 0.27.0, and then everything breaks.

I believe the "bug" is in the overload broad dependency in google-cloud===0.25 (or possibly whatever version you are using).

I believe the "fix" for us is to upgrade to the latest version of google-cloud=0.27.

Though the "proper fix" is for google-cloud to improve their versioning dependencies, and not be so broad, or risk breaking backwards compatibility with already-published modules.