where to get and install crypto.dll on 64 bit Windows

Question

Note: This is not an issue with Visual Studio, but rather with incompatible dll versions. The steps below replicate the problem since running in Visual Studio in debug mode breaks on exceptions being thrown. If you just run, the thrown exception is handled elsewhere and the program works fine. But since I spend a lot of time in debug mode, I would prefer to fix this problem.

When debugging, I want to be able to step into modules I have added to my Virtual Environment in Visual Studio. I get a 'library not found' error that I am not able to fix. Here are the steps:

1. In Visual Studio create a new Python Application.
2. Create a virtual environment for that application (Python 3.6 64 bit).
3. pip install twilio into your virtual environment. You get the following output.

...

   ----- Installing 'twilio' -----
Collecting twilio
  Using cached twilio-6.10.5-py2.py3-none-any.whl
Collecting pytz (from twilio)
  Using cached pytz-2018.3-py2.py3-none-any.whl
Collecting six (from twilio)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting PyJWT>=1.4.2 (from twilio)
  Using cached PyJWT-1.6.0-py2.py3-none-any.whl
Collecting requests>=2.0.0; python_version >= "3.0" (from twilio)
  Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting pysocks; python_version >= "3.0" (from twilio)
  Using cached PySocks-1.6.8.tar.gz
Collecting certifi>=2017.4.17 (from requests>=2.0.0; python_version >= "3.0"->twilio)
  Using cached certifi-2018.1.18-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.0.0; python_version >= "3.0"->twilio)
  Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests>=2.0.0; python_version >= "3.0"->twilio)
  Using cached urllib3-1.22-py2.py3-none-any.whl
Collecting idna<2.7,>=2.5 (from requests>=2.0.0; python_version >= "3.0"->twilio)
  Using cached idna-2.6-py2.py3-none-any.whl
Installing collected packages: pytz, six, PyJWT, certifi, chardet, urllib3, idna, requests, pysocks, twilio
  Running setup.py install for pysocks: started
    Running setup.py install for pysocks: finished with status 'done'
Successfully installed PyJWT-1.6.0 certifi-2018.1.18 chardet-3.0.4 idna-2.6 pysocks-1.6.8 pytz-2018.3 requests-2.18.4 six-1.11.0 twilio-6.10.5 urllib3-1.22
----- Successfully installed 'twilio' -----

4. Add the following line to the top of your .py file:

   from twilio.rest import Client

5. In Visual Studio go to tools > options > python > debugging. Make sure 'Enable debugging of Python standard library' is checked

6. Run the application. You get the following error:

ModuleNotFoundError: No module named 'OpenSSL'

7. pip install pyopenssl You get the following output:

...

----- Installing 'pyopenssl' -----
Collecting pyopenssl
  Using cached pyOpenSSL-17.5.0-py2.py3-none-any.whl
Requirement already satisfied: six>=1.5.2 in c:\users\x\source\repos\pythonapplication9\pythonapplication9\env\lib\site-packages (from pyopenssl)
Collecting cryptography>=2.1.4 (from pyopenssl)
  Using cached cryptography-2.1.4-cp36-cp36m-win_amd64.whl
Requirement already satisfied: idna>=2.1 in c:\users\x\source\repos\pythonapplication9\pythonapplication9\env\lib\site-packages (from cryptography>=2.1.4->pyopenssl)
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=2.1.4->pyopenssl)
  Using cached cffi-1.11.5-cp36-cp36m-win_amd64.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=2.1.4->pyopenssl)
  Using cached asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=2.1.4->pyopenssl)
  Using cached pycparser-2.18.tar.gz
Installing collected packages: pycparser, cffi, asn1crypto, cryptography, pyopenssl
  Running setup.py install for pycparser: started
    Running setup.py install for pycparser: finished with status 'done'
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.1.4 pycparser-2.18 pyopenssl-17.5.0
----- Successfully installed 'pyopenssl' -----

8. Run the application. You get the following error:

   asn1crypto._ffi.LibraryNotFoundError: The library libcrypto could not be found

The error is thrown in the file named _big_num_ctypes.py in asn1crypto. The code line where this is thrown is:

libcrypto_path = find_library(b'crypto' if sys.version_info < (3,) else 'crypto')
if not libcrypto_path:
    raise LibraryNotFoundError('The library libcrypto could not be found')

Update: I was asked to provide the full backtrace. I modified the code in this way to print it:

import unittest
import traceback

class Test_test1(unittest.TestCase):
    def test_A(self):
        try:
            from twilio.rest import Client
        except Exception as e:
            print('foo')
            foo = traceback.extract_stack()
            traceback.print_exc(e)

if __name__ == '__main__':
    unittest.main()

As before the import line throws the exception but the exception is not caught and the lines in 'except' clause are never executed from twilio.rest import Client

update 2: I somehow had gotten this to work following @Prateek and @user8212173. But now it is not working again. As both suggested, the problem is that crypto.dll is not there. So I went thru the steps below to add it with no success:

1. I installed Win64 OpenSSL v1.1.0j from https://slproweb.com/products/Win32OpenSSL.html (pointed to from https://wiki.openssl.org/index.php/Binaries). It does not contain crypto.dll.
2. I then installed crypto.dll from http://www.dlldownloader.com/crypto-dll/ (as @user8212173 suggested) (there is only a 32 bit version) and followed the instructions. I then got a new error message "ImportError: DLL load failed: %1 is not a valid Win32 application" which means that the crypto.dll I installed has a version conflict (I am running 64bit python on a 64bit computer). I remember installing it from Unofficial Windows Binaries for Python Extension Packages I can't find it there. So where do I get a working 64bit version of crypto.dll?

Answer 1

I searched a lot and could find that you are missing crypto.dll file. Your code is looking for this dll file and it is unable to find it.

Please note this wont be installed by pip install crypto as this is python library and the code is looking for a dll file.

ctypes.util.find_library searches for dll file from windows environment path variable.

Reference : find_library() in ctypes

To verify I checked.

find_library('l2gpstore')
>>'C:\\WINDOWS\\system32\\l2gpstore.dll'
find_library('java')
>>'C:\\Program Files\\Java\\jdk-9.0.4\\bin\\java.dll'

Furthermore you should install OpenSSL with libcrypto module from here

OpenSSL

OpenSSL installation instructions

The master sources are maintained in our git repository, which is accessible over the network and cloned on GitHub, at https://github.com/openssl/openssl. Bugs and pull patches (issues and pull requests) should be file on the GitHub repo. Please familiarize yourself with the license.

libcrypto with respect to OpenSSL

reference : GitHub

libcrypto (with platform specific naming): Provides general cryptographic and X.509 support needed by SSL/TLS but not logically part of it.

Once you install binaries and check crypto.dll is available in one of the path strings in your environment variables this issue should be resolved.

If not add it into path variable and check.

Update:

Update since the question has been updated and the issue has recurred.

There are architectural changes with OpenSSL 1.1.0 as compared to 1.0.2

September 13, 2018 - OpenSSL 1.1.0 and later are quite different from previous releases. Users should install BOTH the 1.0.2 series (LTS) and the 1.1.1 (LTS) series for maximum application compatibility. Developers need to recompile their software to support 1.1.1. See the official OpenSSL release strategy document for more details. – Prateek yesterday

If you open 1.0.2 from Github you can see crypto.h file , the same file is missing in latest version. Also OpenSSL there is change in DLL names , they renamed libeay32 to libcrypto

You need to post code which makes use of asn1crypto library in the post. There is no code that explicitly uses asn1crypto in your post. So, not able to reproduce your issue using pipenv.

Make sure you are using updated libraries too.

I would not recommend downloading DLL source from unreliable source like DLLdownloader

---

If you are having issues with latest version of OpenSSL and asn1crypto its better to downgrade OpenSSL to 1.0.2 ,I think that would work considering it ships with crypto.h file.

Good luck!

Answer 2

I tried to reproduce the error on my computer and was successful when I ran the "error-producing" file _big_num_ctypes.py. Although, I do not have Visual Studio, the error stems from the missing crypto.dll file. We will deduce this step-by-step. Let's first examine the error causing code snippet in the file _big_num_ctypes.py:

#imports
from ctypes.util import find_library
.
.
from .._ffi import LibraryNotFoundError, FFIEngineError

try:
# On Python 2, the unicode string here may raise a UnicodeDecodeError as it
# tries to join a bytestring path to the unicode name "crypto"
    libcrypto_path = find_library(b'crypto' if sys.version_info < (3,) else 'crypto')
    if not libcrypto_path:
        raise LibraryNotFoundError('The library libcrypto could not be found')
.
.
except (AttributeError):
    raise FFIEngineError('Error initializing ctypes')

I ran the file:

C:\>cd "C:\ProgramData\Anaconda3\Lib\site-packages\asn1crypto\_perf"
C:\ProgramData\Anaconda3\Lib\site-packages\asn1crypto\_perf>python _big_num_ctypes.py

and had a Traceback for the library import:

Traceback (most recent call last):
  File "_big_num_ctypes.py", line 27, in <module>
    from .._ffi import LibraryNotFoundError, FFIEngineError
ValueError: attempted relative import beyond top-level package

So, I changed the import path for.ffito:

from asn1crypto._ffi import LibraryNotFoundError, FFIEngineError

On the second run, the missing libcrypto library error appeared:

asn1crypto._ffi.LibraryNotFoundError: The library libcrypto could not be found

The exception is raised when the dll library named crypto could not be found at C:\Windows\System32 and/or SYSWOW64(for 64-bit)

libcrypto_path = find_library(b'crypto' if sys.version_info < (3,) else 'crypto')

The purpose of find_library is to find a specified library and return a pathname. The behavior of this method varies with OS as described in the docs. If this method cannot find any packages, it returns None.

>>> from ctypes.util import find_library
>>> print(find_library("c"))
None

In our case, the search is for crypto.dll and I couldn't find this file on my computer. So, I downloaded and installed it exactly according to the instructions here. When I checked again:

 >>> find_library('crypto')
'C:\\windows\\system32\\crypto.dll'

Now I ran python _big_num_ctypes.py again and got a different Traceback:

    Traceback (most recent call last):
  File "_big_num_ctypes.py", line 37, in <module>
    libcrypto = CDLL(libcrypto_path)
  File "C:\ProgramData\Anaconda3\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 is not a valid Win32 application

A further investigation into the above error revealed that if I'm using a 32bit DLL with 64bit Python, or vice-versa, then I may get such errors as explained here. So, I installed Python 3.6 32-bit and tried again with py -3.6-32 _big_num_ctypes.py. I also installed all the required packages, but this error persisted.

Could it be possible that we may require 32-bit binaries for the Crypto package? This answer and this give more information.

I realized that Pycryptodome is a regularly maintained package and is preferred over the old Crypto package but can still be installed under Crypto. Another point to notice is that one of the requirements for this package is MS Visual Studio 2015 (Community Edition) and the C/C++ compilers and the redistributable only. It could be possible that some C++ compiler files or MS Visual Studio files are missing at present and causing these issues to happen.

If you install all the above prerequisites, the crypto.dll file and the Pycryptodomepackage, I believe this error will be Resolved. You have already installed other required packages OpenSSL & Twilio. Unfortunately, I am restricted to install MS Visual Studio on my computer and so I couldn't test this further.

I also ran the unittest code and it ran successfully for me:

#Output

.
----------------------------------------------------------------------
Ran 1 test in 0.771s

OK