Google app engine deployment fails- Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')

Question

We are using command prompt c:\gcloud app deploy app.yaml, but get the following error:

Running "python3 -m pip install --requirement requirements.txt --upgrade --upgrade-strategy only-if-needed --no-warn-script-location --no-warn-conflicts --force-reinstall --no-compile (PIP_CACHE_DIR=/layers/google.python.pip/pipcache PIP_DISABLE_PIP_VERSION_CHECK=1)"
Step #2 - "build": /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
Step #2 - "build": Done "python3 -m pip install --requirement requirements.txt --upgr..." (34.49892ms)
Step #2 - "build": Failure: (ID: 0ea8a540) /layers/google.python.pip/pip/bin/python3: Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__')
Step #2 - "build": --------------------------------------------------------------------------------
Step #2 - "build": Running "mv -f /builder/outputs/output-5577006791947779410 /builder/outputs/output"
Step #2 - "build": Done "mv -f /builder/outputs/output-5577006791947779410 /builder/o..." (12.758866ms)
Step #2 - "build": ERROR: failed to build: exit status 1
Finished Step #2 - "build"
ERROR
ERROR: build step 2 "us.gcr.io/gae-runtimes/buildpacks/python37/builder:python37_20211201_3_7_12_RC00" failed: step exited with non-zero status: 145

Our Requirements.txt is as below. We are currently on Python 3.7 standard app engine

firebase_admin==3.0.0
sendgrid==6.9.3
google-auth==1.35.0
google-auth-httplib2==0.1.0
jinja2==3.0.3
MarkupSafe==2.0.1
pytz==2021.3
Flask==2.0.2
twilio==6.46.0
httplib2==0.20.2
requests==2.24.0
requests_toolbelt==0.9.1
google-cloud-tasks==2.7.1
google-cloud-logging==1.15.1
googleapis-common-protos==1.54.0

Please help.The above code was working well before updating the requirements.txt file. We tried to remove gunicorn to allow the system pickup the latest according to documentation here.

We have a subdirectory structure that stores all the .py files in controllers and db definitions in models. Our main.py has the following -

sys.path.append(os.path.join(os.path.dirname(__file__), '../controllers'))
sys.path.append(os.path.join(os.path.dirname(__file__), '../models'))

Does anyone know how to debug this error - Error while finding module specification for 'pip' (AttributeError: module '__main__' has no attribute '__file__'). What does this mean?

Answer 1

The release 60.3.0 of setuptools has a bug causing the above AttributeError, see : https://github.com/pypa/setuptools/issues/3002

It has been yanked by Pypi since then:

Yanked files are always ignored, unless they are the only file that matches a version specifier that "pins" to an exact version using either == (without any modifiers that make it a range, such as .*) or ===. Matching this version specifier should otherwise be done as per PEP 440 for things like local versions, zero padding, etc.

Source: https://www.python.org/dev/peps/pep-0592/

What you can do as of now is pass the version of the setuptools package explicitly, to use for example version 60.2.0 or 60.3.1.