Certificate Verification Failure for youtube-dl

Question

I am trying to get the audio from a YouTube video, using the command:

youtube-dl --extract-audio --audio-format mp3 [video link]

Each time I try to run this command in Terminal on macOS (High Sierra v10.13.2), it gives me this error:

ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)> (caused by URLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)'),))

Note: I have youtube-dl installed, and also have Python 2.7 and Python 3.6 (if that helps)

If anyone can help me resolve that issue, that would be great.

Answer 1

I've had this issue for a while now and was never able to solve it. So I did the temporary fix of

[ Terminal ]

--no-check-certificate

Sadly, that just turns your HTTPS youtube-dl request into a plain text HTTP request.

After some digging I found that it was a Python v3.6 issue.

https://bugs.python.org/issue29065#msg283984

If you use VIM or any edit to check

vim '/Applications/Python 3.6/ReadMe.rtf'

If you look for ( Certificate verification and OpenSSL\ ) You'll see that ...

During installation of Python v3.6 They decide to use their own private version of OpenSSL, unfortunately that doesn't natively work with the default root certificates. It does however, come with a command script to install a curated bundle of default root certificates.

The bug recommended me to the python certifi module. The modules just seems to be good for finding where your certificate is.

[ Python v3.6 ]

import certifi

certifi.where()

'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/certifi/cacert.pem'

That really just told where it was, but I did it using the python terminal interpreter just in case.

[ Terminal ]

open '/Applications/Python 3.6/Install Certificates.command'

Me personally this was my terminal response...

[ Terminal ]

-- pip install --upgrade certifi

Requirement already up-to-date: certifi in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages

-- removing any existing file or link

-- creating symlink to certifi certificate bundle

-- setting permissions

Traceback (most recent call last):

File "", line 44, in

File "", line 40, in main

PermissionError: [Errno 1] Operation not permitted: 'cert.pem'

logout

Saving session...

...copying shared history...

...saving history...truncating history files...

...completed.

[Process completed]

BUT IT WORKED AND I DON'T HAVE TO DEAL WITH THAT SSL VERIFICATION FAILED ANYMORE! (-.-)\ /(-.-)/

Answer 2

I will try to make a proper approach to the problem rather than just listing what works for me. The following does not require the Install Certificates.command script, which may or may not have been installed along with Python.

---

Source of the problem

It needs to be pointed out that the source of the error is not Python itself. Looking closely to the error message, you can see that Python just communicates the error produced by the OpenSSL library (called by _ssl.c). If the Python you are using has been installed using homebrew, chances are that more network utilities (e.g. wget) installed using homebrew have similar issues.

This means that in order to fix the problem, we need to make sure that the OpenSSL library used by Python has access to a valid, up-to-date certificates bundle.

---

Locating the right OpenSSL library

As many OpenSSL libraries may be installed on your system, you need to find the one used by your Python interpreter. The OpenSSL library is loaded by the ssl Python module, so we need to locate that first:

pyssld=$(python3 -c 'import ssl, pathlib; print(pathlib.Path(ssl.__file__).parent)')
echo "$pyssld"

This will print out the directory where we should look for the C library used by the ssl to load OpenSSL. This is done with the following command:

pyssl=$(find "$pyssld" -iname '*ssl*.so')
echo "$pyssl"

Finally, we can check where the OpenSSL library loaded by the Python ssl module is located:

pyopenssl=$(otool -L "$pyssl" | grep libssl | awk '{print $1}')
echo "$pyopenssl"

This shall print something like:

/opt/homebrew/opt/[email protected]/lib/libssl.1.1.dylib

This points where the OpenSSL library used by Python is located.

---

Fixing the problem

Acquiring a certificate bundle

To acquire an up to date certificate bundle to use with OpenSSL library, you can install the Python certifi package.

pip3 install --upgrade certifi
cabundle=$(python3 -c 'import certifi; print(certifi.where())')
echo $cabundle

Adding the new certificate bundle to OpenSSL

Finally, we need to place the certificate bundle where OpenSSL can find it. We know the location of the library. But the bundles are stored in a different directory. To jump to the right directory and link the bundle, use:

cd $(echo "$pyopenssl" | sed -E 's%/opt/(openssl[^/]*).*%/etc/\1%')
ln -sf "$cabundle" cert.pem

Cleanup

After checking that everything works, you can now clean up the environment variables we have used.

unset cabundle pyopenssl pyssl pyssld