Why doesn't setup_requires work properly for numpy?

Question

I wanted to create a setup.py file that automatically resolves a build-time dependency to numpy (for compiling extensions). My first guess was to use setup_requires and subclass a command class to import the numpy module:

from setuptools import setup, Extension
from distutils.command.build import build as _build

class build(_build):
    def run(self):
        import numpy
        print(numpy.get_include())
        _build.run(self)

setup(
    name='test',
    version='0.0',
    description='something',
    cmdclass={'build':build},
    setup_requires=['numpy'],
)

Now, running python setup.py build successfully compiles numpy but then fails (inside build.run) with:

AttributeError: 'module' object has no attribute 'get_include'

However, if the running the same command again, the command now succeeds (and doesn't need to recompile numpy).

I have tested this on python{2.6,2.7,3.3} with and without virtualenv on pretty recent versions setuptools.

I have seen a workaround using pkg_resources.resource_filename which seems to work just fine, if all we want is the include directory. EDIT: only works on python2!

But still, I am now curious. What caveats does the usage of setup_requires have? What could be the reasons that it doesn't work properly for numpy? For some more simple modules it seems to have no problems.

Answer 1

Figured out, that a proper initialization of the numpy module is prevented by a check for __NUMPY_SETUP__ inside numpy/__init__.py:

if __NUMPY_SETUP__:
    import sys as _sys
    _sys.stderr.write('Running from numpy source directory.\n')
    del _sys
else:
    # import subodules etc. (main branch)

This global state is not reset by setuptools after the installation. The following works:

...
def run(self):
    __builtins__.__NUMPY_SETUP__ = False
    import numpy
    ...