SSL: CERTIFICATE_VERIFY_FAILED error with python3 on macOS 10.15

Question

/usr/bin/python3 from Xcode/CLT on macOS 10.15 (DB6/PB5 at the moment, with Xcode 11 beta 6) fails with SSL: CERTIFICATE_VERIFY_FAILED for all HTTPS requests originating from PSL, e.g. from urllib.request:

$ /usr/bin/python3 -c 'import urllib.request; urllib.request.urlopen("https://www.apple.com/")'
...
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>

How to solve this problem?

(I know the answer, will post shortly; just sharing it in case other people run into it.)

Answer 1

According to this GitHub issue, Apple refused to fix this:

The problem behaves as intended.

certifi is a third-party module, not part of Python itself.

urllib is a low-level library. It can handle SSL, but you must explicitly set up the SSL context with a cafile.

Try the following instead:

pip3 install requests
python3 -c 'import requests; print(requests.get("https://apple.com").text)'

If you only want to get cacert.pem, you can use pip3 install certifi, but you must still explicitly pass cafile to urllib.

So my solution is simply using Requests instead. This is supported and future proof.

Answer 2

Supplemental to @4ae1e1's answer, you can create a symlink to the SSL folder instead of rsyncing it. This will give the added benefit of keeping any changes in /etc/ssl up-to-date at /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/ssl/.

/usr/bin/sudo /bin/mkdir /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc
/usr/bin/sudo /bin/ln -s /etc/ssl/ /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/

Should do it.

Answer 3

The problem is that /usr/bin/python3 (from either Xcode or CLT) fails to correctly locate the trust store in /etc/ssl, as we can see using ssl.get_default_verify_paths():

$ /usr/bin/python3 -c 'import ssl; print(ssl.get_default_verify_paths())'
DefaultVerifyPaths(cafile=None, capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/ssl/certs')

It's looking into /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/ssl, which doesn't exist.

Knowing this, we can use the following hack:

$ sudo rsync -avzP /etc/ssl/ /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/ssl/

I've submitted a bug report to Apple (btw, just realized bugreport.apple.com is now gone, and I had to use the Feedback Assistant website). Open radar https://openradar.appspot.com/7111585 (that radar number is unfortunately wrong — since bugreport.apple.com is gone, I don't have a radar number anymore, only a feedback number FB7111585).