Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the most conventional way to integrate C code into a Python library using distutils?

Many well-known python libraries are basically written in C (like tensorflow or numpy) because this apparently speeds things up a lot. I was able to very easily integrate a C function into python by reading this. Doing so I can finally use distutils to access the functions of the source.c file:

# setup.py

from distutils.core import setup, Extension

def main():
    setup(
        # All the other parameters...
        ext_modules=[ Extension("source", ["source.c"]) ]
        )

if __name__ == "__main__":
    main()

so that when i run python setup.py install i can install my library. However, what if i want to create a python-coded wrapper object for the functions inside source.c? Is there a way to do this without polluting the installed modules?
Wandering around the internet I have seen some seemingly simple solutions using shared libraries (.so). However I would need a solution that does not involve attaching the compiled C code, but one that compiles it the first time the program is run.

like image 396
Giuppox Avatar asked Mar 05 '21 18:03

Giuppox


People also ask

What does Distutils do in Python?

The distutils package provides support for building and installing additional modules into a Python installation. The new modules may be either 100%-pure Python, or may be extension modules written in C, or may be collections of Python packages which include modules coded in both Python and C.

How are Python libraries written in C?

To write Python modules in C, you'll need to use the Python API, which defines the various functions, macros, and variables that allow the Python interpreter to call your C code. All of these tools and more are collectively bundled in the Python. h header file.

What is a .so file in Python?

The basics of using a Shared Library file A file with the . SO file extension is a Shared Library file. They contain information that can be used by one or more programs to offload resources so that the application(s) calling the SO file doesn't have to actually provide the file.


1 Answers

The shared libraries are the right way to go in this case. The distutils have ability to build the static libraries as follows:

from distutils.ccompiler import new_compiler
from distutils import sysconfig

c = new_compiler()
workdir = "."
c.add_include_dir( sysconfig.get_python_inc() )
c.add_include_dir("./include")
objects = c.compile(["file1.c", "file2.c"])
c.link_shared_lib(objects, "mylibrary", output_dir=workdir)

This will generate the .so library in the working directory.

For example how it's used in real setup see the following example

like image 81
jordanvrtanoski Avatar answered Sep 22 '22 21:09

jordanvrtanoski