How to package a linked DLL and a pyd file into one self contained pyd file?

Question

I am building a python module with Cython that links against a DLL file. In order to succesfully import my module I need to have the DLL in the Windows search path. Otherwise, the typical error message is:

ImportError: DLL load failed: The specified module could not be found.

Is there a way to packaged the DLL directly into the produced pyd file to make the distribution easier?

One example of this is with the OpenCV distribution, where a (huge) pyd file is distributed and is the only file needed for the Python bindings to work.

Answer 1

Python's packaging & deployment is still a pain point for many of us. There is just not a silver bullet. Here are several methods:

---

1. OpenCV build method

The method is decribed here : https://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_bindings/py_bindings_basics/py_bindings_basics.html#bindings-basics

OpenCV generates these wrapper functions automatically from the C++ headers using some Python scripts which are located in modules/python/src2.

Basically it parse the header files and generate the static PyObject keywords whenever it's needed. Once the header are created appropriately, it just call python setup. Honestly, it might work, but I would not advise this method.

2. Makefiles

If you already use Makefile, just create a rule to place your lib accordinlgy. Example, from my own code :

setup.py

from distutils.core import setup, Extension
setup(name='sha1_hmac', version='1.0',  \
      ext_modules=[Extension('sha1_hmac',
                             library_dirs=['C:\MinGW\lib'],
                             sources= ['../tools/sha1.c','sha1_hmac.c'])])

Makefile

# The hmac generation used by the webserver is done
# using the sha1.c implementation. There is a binding needed to
# glue the C code with the python script
libsha1_hmac:
ifeq ($(OS), Windows_NT)
    $(PYTHON) setup.py build --compiler=mingw32
else
    $(PYTHON) setup.py install --home=$(CURDIR)
endif

.PHONY: webserver
webserver:  libsha1_hmac
ifeq ($(OS), Windows_NT)
    mv $(shell find build -type f -name "sha1*.pyd") $(LIB)
else
    mv -f $(shell find $(LIB)/python -type f -name "sha1*.so") $(LIB)
endif
    $(PYTHON) hmac_server.py

3. Modern deployement tools

There are several new tools to deploy python applications, namely wheels which seem to gain traction. I don't use it, but it look like it can ease up your bundling problem :

• How can I make a Python Wheel from an existing native library?

Once it wheeled, you can install it like this : pip install some-package.whl