Google App Engine - Big Query - Python can't find libraries google.cloud

Question

I'm building an App Engine project using Python on Ubuntu Linux 16.4 using Eclipse and PyDev.

I'm using the Flask Framework which works fine, what I want to do is connect to Big Query and that's where I am having trouble.

So I have used:

pip install --upgrade google-cloud-bigquery

To install the Big Query Library and I can see the packages in /usr/local/lib/python2.7/dist-packages

For some reason Eclipse doesn't seem to think the library is registered, I can live with that but when I run my project dev_appserver.py App Engine can't find the library:

I have this in my Python PyDev:

It is weird as if I add from lib.google.cloud import bigquery PyDev resolves, the dist-packages is in the PyDev path so I don't get why it doesn't pick it up.

You'll also notice I've commented out the apiclient.discovery and oauth2client.client, I was going to try another way but App Engine complained it couldn't find the SignedJwtAssertionCredentials library so I am sure this is all linked and caused by the same problem.

In my appengine_config.py I have:

Answer 1

(Jun 2021): There seems to be some confusion in the OP and various responses, so let's make it super clear for Python 2 users:

1. Add google-cloud-bigquery to requirements.txt
2. Create appengine_config.py to match that of Step 4 on this page (which you seem to have)
3. Delete the lines with apiclient or googleapiclient and oauth2client (wrong client libraries) from main.py
4. Run pip2 install -t lib -r requirements.txt (or just pip if you don't have Python 3 installed)
5. NOTE: Python 3 users don't have to play these games... step 1 above is all you need to do... no appengine_config.py file, no running pip install, and no lib folder.

For a broader audience, see my answer to this more general question.