Python C++ extension: compile only modified source files

Question

I'm developing a python package that contains a C++ extension. When I install the package using the setup.py script or using pip, the C++ source files are all compiled and linked to obtain a single .so library, which can then be imported in the Python source code. During development, I need to make multiple changes to the source code (testing, debugging, etc). I find that re-installing the package involves rebuilding all the C++ source files, even if only a small part of one file was changed. Obviously, this takes up quite a bit of time.

I'm aware of the development mode (python setup.py develop or pip install -e) that places a link to the source files, so that changes made are immediately seen when the module is re-imported. However, this applies only to the .py source files and not the C++ extension, which has to be re-compiled after every change.

Is there a way to have setup.py look at the .o files in the build directory (while in development mode) and use their timestamps to figure which ones need to be re-compiled? I'm thinking of the way GNU Make performs selective compilation based on timestamps. Thanks

Answer 1

I would recommend to use Make (other other build systems like CMake) for development and setup.py only for the final installation / deployment. I have done similar Python + C++ projects and it works great that way.