undefined symbol: OPENSSL_sk_num

Question

I'm trying to renew Let's Encrypt certificate with Certbot. It stopped working and i don't know why. Here is the error:

ImportError: /root/.local/share/letsencrypt/local/lib/python2.7/site-packages/cryptography/
hazmat/bindings/_openssl.so: undefined symbol: OPENSSL_sk_num

I have newest OpenSSL version installed

OpenSSL 1.1.0d  26 Jan 2017

I tried debugging this problem by doing the following. First i just tried adding import OpenSSL in python console. It worked perfectly, no errors. But when i try

. ~/.local/share/letsencrypt/bin/activate

Then >>> import OpenSSl I get error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import rand, crypto, SSL
  File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/OpenSSL/rand.py", line 12, in <module>
    from OpenSSL._util import (
  File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: /root/.local/share/letsencrypt/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/_openssl.so: undefined symbol: OPENSSL_sk_num

I tried removing the /root/.local/share/letsencrypt/ path then tried to run certbot-auto again. Still i get the same error. Is there anyone who faced this problem and know the solution? Please help me out here. Need to renew few certificates.

UPDATE:

I'v found the problem source that in /lib/x86_64-linux-gnu directory there is an old version of libssl.so.1.0.0 and it doesn't have OPENSSL_sk_num. When i try to replace with newer version libssl1.1 (it does have OPENSSL_sk_num) then i get an error that it requires OPENSSL_VERSION 1.0.1. Then after some struggle deleting libraries from /usr directories and local directories i get error ImportError: libssl.so.1.0.0: cannot open shared object file: No such file or directory. How can i change it so letsencrypt uses newer library?

SOLUTION

After some struggle. I just reinstalled openssl version 1.1.0c. Copied letsencrypt library from another project and it worked. I think some upgrade ruined it. So i suggest everyone when you are running letsencrypt just use --no-self-upgrade option.

SOLUTION UPDATE

After encountered this problem one more time i decided to resolve it the correct way. So basically you need to recompile openssl 1.1.0c with command:

./config -Wl,--enable-new-dtags,-rpath,'$(LIBRPATH)' and make

Copy the compiled libcrypto.so.1.1 and libssl.so.1.1 to /usr/lib/x86_64-linux-gnu

Then you need to redo or just modify libcrypto and libssl symlinks. By being in /usr/lib/x86_64-linux-gnu folder enter commands ln -s libssl.so.1.1 libssl and ln -s libcrypto.so.1.1 libcrypto.

Then enter following commands:

cd ~/.local/share/letsencrypt/bin/
./pip uninstall cryptography pyopenssl -y
./pip install --upgrade pip
rm -rf ~/.cache/
./pip install cryptography pyopenssl

And your'e done, everything should work correctly.

Answer 1

It looks like in version 1.1.0f of openssl the symbol OPENSSL_sk_num has moved to libcrypto.a. The build of python 3 didn't seem to link that in, hence the missing symbol. However, I was mistaken. When the file Modules/Setup.dist is modified to pick up your own version of openssl, you need to copy it to Modules/Setup, otherwise it will use the already installed ssl.