Pipenv install fails on cryptography package: "Disabling PEP 517 processing is invalid" error

Question

Without being aware of it, I've been using the cryptography package for many months now without issue, but suddenly it won't build and it's brought development on my project to a halt.

It turns out cryptography is a dependency of one of the other packages in my Pipfile: apns2. As I said, it's been working fine, but something must have changed in my environment. Here is my Pipfile with just that package isolated:

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[requires]
python_version = "3.7.9"

[packages]

apns2 = "*"

[dev-packages]

This fails. I use pipenv (the latest version 2020.11.15), so when I run it now this is the output:

# pipenv install
Creating a virtualenv for this project...
Pipfile: /Users/dylan/Dev/RideHare/server/Pipfile
Using /Users/dylan/.pyenv/versions/3.7.9/bin/python3.7m (3.7.9) to create virtualenv...
⠏ Creating virtual environment...created virtual environment CPython3.7.9.final.0-64 in 502ms
  creator CPython3Posix(dest=/Users/dylan/.local/share/virtualenvs/server-2uc2X-TS, clear=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/dylan/Library/Application Support/virtualenv)
    added seed packages: pip==20.2.2, setuptools==51.0.0, wheel==0.35.1
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /Users/dylan/.local/share/virtualenvs/server-2uc2X-TS
Installing dependencies from Pipfile.lock (aa4cd9)...
An error occurred while installing cryptography==3.3.1; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5' --hash=sha256:84ef7a0c10c24a7773163f917f1cb6b4444597efd505a8aed0a22e8c4780f27e --hash=sha256:788a3c9942df5e4371c199d10383f44a105d67d401fb4304178020142f020244 --hash=sha256:9f6b0492d111b43de5f70052e24c1f0951cb9e6022188ebcb1cc3a3d301469b0 --hash=sha256:a69bd3c68b98298f490e84519b954335154917eaab52cf582fa2c5c7efc6e812 --hash=sha256:b4890d5fb9b7a23e3bf8abf5a8a7da8e228f1e97dc96b30b95685df840b6914a --hash=sha256:0003a52a123602e1acee177dc90dd201f9bb1e73f24a070db7d36c588e8f5c7d --hash=sha256:7e177e4bea2de937a584b13645cab32f25e3d96fc0bc4a4cf99c27dc77682be6 --hash=sha256:dc42f645f8f3a489c3dd416730a514e7a91a59510ddaadc09d04224c098d3302 --hash=sha256:69e836c9e5ff4373ce6d3ab311c1a2eed274793083858d3cd4c7d12ce20d5f9c --hash=sha256:9e21301f7a1e7c03dbea73e8602905a4ebba641547a462b26dd03451e5769e7c --hash=sha256:0e85aaae861d0485eb5a79d33226dd6248d2a9f133b81532c8f5aae37de10ff7 --hash=sha256:c366df0401d1ec4e548bebe8f91d55ebcc0ec3137900d214dd7aac8427ef3030 --hash=sha256:594a1db4511bc4d960571536abe21b4e5c3003e8750ab8365fafce71c5d86901 --hash=sha256:83d9d2dfec70364a74f4e7c70ad04d3ca2e6a08b703606993407bf46b97868c5! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 10/10 — 00:00:08
Installing initially failed dependencies...
[InstallError]:   File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/cli/command.py", line 233, in install
[InstallError]:       retcode = do_install(
[InstallError]:   File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py", line 2052, in do_install
[InstallError]:       do_init(
[InstallError]:   File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]:       do_install_dependencies(
[InstallError]:   File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]:       batch_install(
[InstallError]:   File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]:   File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Looking in indexes: https://pypi.python.org/simple
[pipenv.exceptions.InstallError]: Collecting cryptography==3.3.1
[pipenv.exceptions.InstallError]:   Using cached cryptography-3.3.1.tar.gz (539 kB)
[pipenv.exceptions.InstallError]: ERROR: Disabling PEP 517 processing is invalid: project specifies a build backend of setuptools.build_meta in pyproject.toml
ERROR: Couldn't install package: cryptography
 Package installation failed...
  ☤  ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/1 — 00:00:01

If instead of the apns2 line in the Pipfile I substitute cryptography explicitly (crytography = "*"), it fails too in the same way.

Answer 1

The confusing error messages are caused by the fact that if/when pipenv fails to build cryptography for some reason, it tries a second time with --no-use-pep517 (see my comment below) and then we see those PEP 517 disabling errors instead of the root cause!

To track down your actual issue, try pip install cryptography (not pipenv) so you can see the root cause.

It is likely that you need to install rust and/or cargo, or openssl on your platform using apt-get install, apk add, brew install or whatever, then try again.

Once you get the issue resolved that way, pipenv installation should work.

See also cryptography package github issue #5773

Answer 2

If you are on a recent macOS version, the default pip version in your pipenv virtualenv incorrectly detects your OS version which makes looking up a compatible wheel fail (and it will start compiling cryptography dependencies).

To avoid the issue at it's root you can upgrade pip (in the virtualenv) to a version that detects mac os correctly:

pipenv shell
pip install -U pip

The bug was fixed in pip version 20.3 (2020-11-30) https://github.com/pypa/pip/issues/9138

Answer 3

It seems the cryptography package can only be installed with pip, not pipenv, so by doing this first...

brew install pkg-config libffi openssl
env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install cryptography

(Commands taken from this question.)

...I could then run pipenv install which built and installed everything fine.