Translate F2PY compile steps into setup.py

Question

I've inherited a Fortran 77 code which implements several subroutines which are run through a program block which requires a significant amount of user-input via an interactive command prompt every time the program is run. Since I'd like to automate running the code, I moved all the subroutines into a module and wrote a wrapper code through F2PY. Everything works fine after a 2-step compilation:

gfortran -c my_module.f90 -o my_module.o -ffixed-form
f2py -c my_module.o -m my_wrapper my_wrapper.f90

This ultimately creates three files: my_module.o, my_wrapper.o, my_module.mod, and my_wrapper.so. The my_wrapper.so is the module which I import into Python to access the legacy Fortran code.

My goal is to include this code to use in a larger package of scientific codes, which already has a setup.py using distutils to build a Cython module. Totally ignoring the Cython code for the moment, how am I supposed to translate the 2-step build into an extension in the setup.py? The closes I've been able to figure out looks like:

from numpy.distutils.core import setup, Extension
wrapper = Extension('my_wrapper', ['my_wrapper.f90', ])
setup(
    libraries = [('my_module', dict(sources=['my_module.f90']],
                                    extra_f90_compile_args=["-ffixed-form", ])))],
    ext_modules = [wrapper, ]
)

This doesn't work, though. My compiler throws many warnings on the my_module.f90, but it still compiles (it throws no warnings if I use the compiler invocation above). When it tries to compile the wrapper though, it fails to find the my_module.mod, even though it is successfully created.

Any thoughts? I have a feeling I'm missing something trivial, but the documentation just doesn't seem fleshed out enough to indicate what it might be.

Answer 1

It might be a little late, but your problem is that you are not linking in my_module when building my_wrapper:

wrapper = Extension('my_wrapper', sources=['my_wrapper.f90'], libraries=['my_module'])

setup(
    libraries = [('my_module', dict(sources=['my_module.f90'],
                                    extra_f90_compile_args=["-ffixed-form"]))],
    ext_modules = [wrapper]
)

If your only use of my_module is for my_wrapper, you could simply add it to the sources of my_wrapper:

wrapper = Extension('my_wrapper', sources=['my_wrapper.f90', 'my_module.f90'],
                                  extra_f90_compile_args=["-ffixed-form"])
setup(
    ext_modules = [wrapper]
)

Note that this will also export everything in my_module to Python, which you probably don't want.

I am dealing with such a two-layer library structure outside of Python, using cmake as the top level build system. I have it setup so that make python calls distutils to build the Python wrappers. The setup.pys can safely assume that all external libraries are already built and installed. This strategy is advantageous if one wants to have general-purpose libraries that are installed system-wide, and then wrapped for different applications such as Python, Matlab, Octave, IDL,..., which all have different ways to build extensions.