I have a python module fully implemented in python. (For portability reasons.)
The implementation of a small part has been duplicated in a cython module. To improve perfomance where possible.
I know how to install the .c
modules created by cython with distutils
. However if a machine has no compiler installed, I suspect the setup will fail even though the module is still usable in pure python mode.
Is there a way to compile the .c
module if possible but fail gracefully and install without it if compiling is not possible?
Cython source file names consist of the name of the module followed by a . pyx extension, for example a module called primes would have a source file named primes. pyx . Cython code, unlike Python, must be compiled.
To compile the example. pyx file, select Tools | Run setup.py Task command from the main menu. In the Enter setup.py task name type build and select the build_ext task. See Create and run setup.py for more details.
I guess you will have to make some modification both in your setup.py
and in one __init__
file in your module.
Let say the name of your package will be "module" and you have a functionality, sub
for which you have pure python code in the sub
subfolder and the equivalent C code in c_sub
subfolder.
For example in your setup.py
:
import logging
from setuptools.extension import Extension
from setuptools.command.build_ext import build_ext
from distutils.errors import CCompilerError, DistutilsExecError, DistutilsPlatformError
logging.basicConfig()
log = logging.getLogger(__file__)
ext_errors = (CCompilerError, DistutilsExecError, DistutilsPlatformError, IOError, SystemExit)
setup_args = {'name': 'module', 'license': 'BSD', 'author': 'xxx',
'packages': ['module', 'module.sub', 'module.c_sub'],
'cmdclass': {'build_ext': build_ext}
}
ext_modules = [Extension("module.c_sub._sub", ["module/c_sub/_sub.c"])]
try:
# try building with c code :
setup(ext_modules=ext_modules, **setup_args)
except ext_errors as ex:
log.warn(ex)
log.warn("The C extension could not be compiled")
## Retry to install the module without C extensions :
# Remove any previously defined build_ext command class.
if 'build_ext' in setup_args['cmdclass']:
del setup_args['cmdclass']['build_ext']
# If this new 'setup' call don't fail, the module
# will be successfully installed, without the C extension :
setup(**setup_args)
log.info("Plain-Python installation succeeded.")
Now you will need to include something like this in your __init__.py
file (or at any place relevant in your case):
try:
from .c_sub import *
except ImportError:
from .sub import *
In this way the C version will be used if it was build, other-wise the plain python version is used. It assumes that sub
and c_sub
will provide the same API.
You can find an example of setup file doing this way in the Shapely
package. Actually most of the code I posted was copied (the construct_build_ext
function) or adapted (lines after) from this file.
Class Extension
has parameter optional
in constructor:
optional - specifies that a build failure in the extension should not abort the build process, but simply skip the extension.
Here is also a link to the quite interesting history of piece of code proposed by mgc.
The question How should I structure a Python package that contains Cython code
is related, there the question is how to fallback from Cython to the "already generated C code". You could use a similar strategy to select which of the .py
or the .pyx
code to install.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With