./python: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

Question

I need to try python 3.7 with openssl-1.1.1 in Ubuntu 16.04. Both python and openssl versions are pre-release. Following instructions on how to statistically link openssl to python in a previous post, I downloaded the source for opnssl-1.1.1. Then navigate to the source code for openssl and execute:

./config
sudo make
sudo make install

Then, edit Modules/Setup.dist to uncomment the following lines:

SSL=/usr/local/ssl
_ssl _ssl.c \
    -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
    -L$(SSL)/lib -lssl -lcrypto

Then download python 3.7 source code. Then, navigate inside the source code and execute:

./configure
make
make install

After I execute make install I got this error at the end of the terminal output:

./python: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
generate-posix-vars failed
Makefile:596: recipe for target 'pybuilddir.txt' failed
make: *** [pybuilddir.txt] Error 1

I could not figure out what is the problem and what I need to do.

Answer 1

What I have done to fix this :

./configure --with-ssl=./libssl --prefix=/subsystem
sed -i 's!^RUNSHARED=!RUNSHARED=LD_LIBRARY_PATH=/path/to/own/libssl/lib!' Makefile
make
make install

Setting LD_LIBRARY_PATH with export was not sufficient

Answer 2

This has (should have) nothing to do with Python or OpenSSL versions.

Python build process, includes some steps when the newly built interpreter is launched, and attempts to load some of the newly built modules - including extension modules (which are written in C and are actually shared objects (.sos)).

When an .so is loaded, the loader must find (recursively) all the .so files that the .so needs (depends on), otherwise it won't be able to load it.

Python has some modules (e.g. _ssl*.so, _hashlib*.so) that depend on OpenSSL libs. Since you built yours against OpenSSL1.1.1 (the lib names differ from what comes by default on the system: typically 1.0.*), the loader won't be able to use the default ones.

What you need to do, is instruct the loader where to look for "your" OpenSSL libs (which are located under /usr/local/ssl/lib). One way of doing that is adding their path in ${LD_LIBRARY_PATH} env var (before building Python):

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/ssl/lib
./configure
make
make install

Check [SO]: How to enable FIPS mode for libcrypto and libssl packaged with Python? (@CristiFati's answer) for details on a wider problem remotely related to yours.