How to include header file in Cython correctly (setup.py)?

Question

How can I include my_math/clog.h properly in the package my_math to let tox pass the py.test suite?

---

Project file structure:

my_math/__init__.py
my_math/clog.h
my_math/clog.pxd
my_math/integrate.pyx
setup.py
tests/__init__.py
tests/test_log.py
tox.ini

my_math/clog.h

#include <python.h>
#include <math.h>

#if PY_VERSION_HEX < 0x3050000 && (defined(_WIN32) || defined(_WIN64))
// Calculates log2 of number
double log2(double n);
{
    // log(n)/log(2) is log2
    double log_2 = 0.693147180559945309417232121458176568075500134360255254120680;
    return log(n) / log_2;
}
#endif

my_math/integrate.pyx

def extern from "clog.h":
    double log2(double)

def call_log2(n):
    return log2(n)

setup.py

import os

from setuptools import setup, Extension
from setuptools.dist import Distribution
from Cython.Distutils import build_ext

Distribution(dict(setup_requires='Cython'))

ext_modules = [
    Extension("my_math.integrate", ["my_math/integrate.pyx"],
              include_dirs=['my_math'],  # path to .h file(s)
              library_dirs=['my_math'],  # path to .a or .so file(s)
              # libraries=['clog'],
              extra_compile_args = ['-I/usr/local/include',
                                    '-L/usr/local/lib'],
              depends=['my_math/clog.h']
            )
]

setup(
    name='my_math',
    package_dir={'my_math': 'my_math'},
    packages=['my_math'],
    include_package_data=True,
    package_data={'': ['*.pyx', '*.pxd', '*.h', '*.c']},
    cmdclass = {'build_ext': build_ext},
    ext_modules=ext_modules
)

tests/test_log.py

from pytest import approx

from my_math.integrate import call_log2

def test_call_log2():
    assert approx(call_log2(100)) == 6.64386

tox.ini

[tox]
envlist = py36

[testenv]
deps = cython
       pytest
commands = python setup.py build_ext --inplace
           py.test tests

my_math/__init__.py and tests/__init__.py are empty.

---

I run python3 setup.py build_ext --inplace without any glitches.

I can call the function call_log2 directly:

python3 -c 'from my_math.integrate import call_log2; print(call_log2(100))'
6.643856189774724

But not with tox:

clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Imy_math -I/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c my_math/integrate.c -o build/temp.macosx-10.11-x86_64-3.6/my_math/integrate.o -I/usr/local/include -L/usr/local/lib
clang: warning: argument unused during compilation: '-L/usr/local/lib'
my_math/integrate.c:432:10: fatal error: 'clog.h' file not found
#include "clog.h"
         ^
1 error generated.
error: command 'clang' failed with exit status 1

If I uncomment # libraries = ['clog'], a fresh python3 setup.py build_ext --inplace will produce a ld: library not found for -lclog error.

Answer 1

Resolved through corrected MANIFEST.in:

global-include *.txt *.rst *.pyx *.pxd *.c *.h

setup.py:

'': ['data/*', '*.pyx', '*.pxd', '*.c', '*.h']

MANIFEST.in and setup.py must both cover all file patterns to support bdist and sdist.

Related: How to include package data with setuptools/distribute?