I'm building a python package using a C library with ctypes. I want to make my package portable (Windows, Mac and Linux).
I found a strategy, using build_ext
with pip
to build the library during the installation of my package. It creates libfoo.dll
or libfoo.dylib
or libfoo.so
depending on the target's platform.
The problem with this is that my user needs CMake installed.
Does exist another strategy to avoid building during the installation? Do I have to bundle built libraries in my package?
I want to keep my users doing pip install mylib
.
Edit: thank to @Dawid comment, I'm trying to make a python wheel
with the command python setup.py bdist_wheel
without any success.
How can I create my python wheel for different platform with the embedded library ?
Edit 2: I'm using python 3.4 and working on Mac OS X, but I have access to Windows computer, and Linux computer
By and large, the official third-party library repositories for languages run as open source projects, like Python, are safe. But malicious versions of a library can spread quickly if unchecked.
You're certainly heading down the right path according to my research... As Daniel says, the only option you have is to build and distribute the binaries yourself.
In general, the recommended way to install packages is covered well in the packaging user guide. I won't repeat advice there as you have clearly already found it. However the key point in there is that the Python community, specifically PyPA are trying to standardize on using platform wheels to package binary extensions. Sadly, there are a few issues at this point:
I think you are hitting this last issue. A workaround is to force the Distribution to build a platform wheel by overriding is_pure() to always return False. However you could just keep your original build instructions and bdist_wheel should handle it.
Once you've built the wheel, though, you still need to distribute it and maybe other binary packages that it uses or use it. At this point, you probably need to use one of the recommended tools like conda or a PyPI proxy like devpi to serve up your wheels.
EDIT: To answer the extra question about cross-compiling
As covered here Python 2.6 and later allows cross-compilation for Windows 32/64-bit builds. There is no formal support for other packages on other platforms and people have had limited success trying to do it. You are really best off building natively on each of your Linux/Mac/Windows environments.
@rth and @PeterBrittain help me a lot. Here the solution I use:
structure folder :
setup.py
python_package/
lib/
libfoo.dylib
libfoo.dll
__init__.py
main.py
setup.py :
from setuptools import setup, dist
class BinaryDistribution(dist.Distribution):
def is_pure(self):
return False
setup(
name='python_package',
package_data={'python_package': ['lib/libfoo.dylib','lib/libfoo.dll']},
include_package_data=True,
distclass=BinaryDistribution,
packages=['python_package'],
)
main.py :
#!/usr/bin/env python
import platform
from ctypes import CDLL, c_char_p
import pkg_resources
sysname = platform.system()
if sysname == 'Darwin':
lib_name = "libfoo.dylib"
elif sysname == 'Windows':
lib_name = "libfoo.dll"
else:
lib_name = "libfoo.so"
lib_path = pkg_resources.resource_filename('python_package', 'lib/{}'.format(lib_name))
foo = CDLL(lib_path)
bar = foo.bar
bar.restype = c_char_p
bar.argtypes = [c_char_p]
print(bar('hello'))
build wheel :
python setup.py bdist_wheel
It creates a specific plateform wheel rtfdoc-0.0.1-cp34-cp34m-macosx_10_10_x86_64.whl
and mac users can do a simple pip install rtfdoc-0.0.1-cp34-cp34m-macosx_10_10_x86_64.whl
This solution is not entirely satisfactory:
Thank you, I learned a lot, and I understand why setup.py in Pillow or Psycopg2 are huge
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With