Set default compiler when using Cython and setuptools to compile multiple extensions

Question

I currently have a project, where we make heavy use of Cython to speed up all of our submodules / classes and I wish to compile them with clang instead of gcc on a Linux system. To this end I currently have a function that gathers all the extensions in the following way:

def extensions():
    try:
        from Cython.Build import cythonize
    except ImportError:
        def cythonize(*args, **kwargs):
            print("Hint: Wrapping import of cythonize in extensions()")
            from Cython.Build import cythonize
            return cythonize(*args, **kwargs)

    try:
        import numpy
        lstIncludes = [numpy.get_include()]
    except ImportError:
        lstIncludes = []

    extensionArguments = {
        'include_dirs':
        lstIncludes + ['package/core', 'package/inspect', 'util'],
        'extra_compile_args': compilerArguments,
        'extra_link_args': linkerArguments,
        'define_macros': defineMacros
    }

    return cythonize(
        [Extension("*", ["package/*.pyx"], **extensionArguments),
         Extension("*", ["package/algs/*.pyx"], **extensionArguments),
         Extension("*", ["package/core/*.pyx"], **extensionArguments)],
        compiler_directives=cythonDirectives)

Moreover, I set

os.environ["CC"] = "clang"
os.environ["CXX"] = "clang"

in setup.py and when I call python setup.py build_ext --inplace, the first Cython extension is built using clang but the second one is built with gcc. This behaviour does not change, if I invoke

export CC=clang; python setup.py build_ext --inplace

Then again, the first uncompiled of the cythonized extensions is built using clang but the second one and all following are built using gcc again.

How can I set and fix the compiler for all Cython modules during compilation?

Edit: After some weeks of consideration I still do not have a solution to above problem and it also occurs across multiple machines, so it is not a result of my specific setup.

Answer 1

I finally found out, that it is necessary to forcibly override stuff that disutils has extracted from our system configuration. To this end one has to set certain options for each extension separately by modifying above code as follows

from distutils import sysconfig

def extensions():
    '''
    Handle generation of extensions (a.k.a "managing cython compilery").
    '''
    try:
        from Cython.Build import cythonize
    except ImportError:
        def cythonize(*args, **kwargs):
            print("Hint: Wrapping import of cythonize in extensions()")
            from Cython.Build import cythonize
            return cythonize(*args, **kwargs)

    try:
        import numpy
        lstIncludes = [numpy.get_include()]
    except ImportError:
        lstIncludes = []

    extensionArguments = {
        'include_dirs':
        lstIncludes + ['package/core', 'package/inspect', 'util'],
        'extra_compile_args': compilerArguments,
        'extra_link_args': linkerArguments,
        'define_macros': defineMacros
    }

    # me make damn sure, that disutils does not mess with our
    # build process

    sysconfig.get_config_vars()['CFLAGS'] = ''
    sysconfig.get_config_vars()['OPT'] = ''
    sysconfig.get_config_vars()['PY_CFLAGS'] = ''
    sysconfig.get_config_vars()['PY_CORE_CFLAGS'] = ''
    sysconfig.get_config_vars()['CC'] = 'gcc'
    sysconfig.get_config_vars()['CXX'] = 'g++'
    sysconfig.get_config_vars()['BASECFLAGS'] = ''
    sysconfig.get_config_vars()['CCSHARED'] = '-fPIC'
    sysconfig.get_config_vars()['LDSHARED'] = 'gcc -shared'
    sysconfig.get_config_vars()['CPP'] = ''
    sysconfig.get_config_vars()['CPPFLAGS'] = ''
    sysconfig.get_config_vars()['BLDSHARED'] = ''
    sysconfig.get_config_vars()['CONFIGURE_LDFLAGS'] = ''
    sysconfig.get_config_vars()['LDFLAGS'] = ''
    sysconfig.get_config_vars()['PY_LDFLAGS'] = ''

    return cythonize(
        [Extension("*", ["package/*.pyx"], **extensionArguments),
         Extension("*", ["package/algs/*.pyx"], **extensionArguments),
         Extension("*", ["package/core/*.pyx"], **extensionArguments)],
        compiler_directives=cythonDirectives,
        nthreads=4
    )

This way one has the full control over anything happening while building the extensions.