Use the Eigen library with cppyy

Question

I've been successfully using cppyy for automatic python bindings for a C++ project I'm working on. I recently included the Eigen library, but I'm having trouble using this together with cppyy. Does anyone have any experience doing this, or know how I should do this?

I have the following structure for the repo (only relevant parts shown):

.
├── CMakeLists.txt
├── build
├── external
   ── eigen
├── include
   ── all .hpp files
├── src
   ── all .cpp files
├── python
   ── qmc.py

Here the external/eigen is a copy of the Eigen GitHub repo. The qmc.py file is where the cppyy magic happens, and it looks like this (before trying to add Eigen, this works just fine)

import cppyy
import tempfile
import os
import glob

try:
    current_dir = os.path.dirname(__file__)
except NameError:
    current_dir = os.getcwd()
source_dir = os.path.dirname(current_dir)
install_dir = os.path.join(source_dir, 'build')
include_dir = os.path.join(source_dir, 'include')
eigen_dir = os.path.join(source_dir, 'external', 'eigen')
print(current_dir, source_dir, include_dir, install_dir)

def cmake_run(build_type='Release', c_compiler='gcc', cxx_compiler='g++'):
    os.environ['CC'] = c_compiler
    os.environ['CXX'] = cxx_compiler
    os.system('cd {} && cmake {} -DCMAKE_BUILD_TYPE={}'.format(install_dir, source_dir, build_type))

def load_library():
    os.system('cd {} && make engine'.format(install_dir))
    libraries = glob.glob(os.path.join(install_dir, 'libengine.*'))
    print('Found libraries: {}'.format(libraries))
    library = libraries[0]
    cppyy.load_library(library)
    for header in glob.glob(os.path.join(include_dir, '*.hpp')):
        print('Loading {}'.format(header))
        cppyy.include(header)

The build part works, but as soon as I try to load any header that uses Eigen, I get an error. I've tried just about everything I can think of (include needed headers manually one by one, copying the entire library into the build dir etc.) but regardless of what I do the same type of errors pop up. Something like

In file included from 
/path/to/repo/projects/include/myheader.hpp:3:10: fatal error: 'Eigen/Dense' file not found
#include <Eigen/Dense>
         ^~~~~~~~~~~~~

Any help with what to change here would be much appreciated!

Edit: To be clear, the build step works just fine, i.e. the code compiles, links and runs as it should. Loading the library with cppyy also works. The issue is that cppyy also needs to include the header files. Again, this works for my own headers, but it is unable to find the Eigen headers...

Answer 1

When calling help(), there is:

>>> import cppyy
>>> help(cppyy)
    """
    add_include_path(path)
        Add a path to the include paths available to Cling.
    """
>>>

So with eigen_dir being the path to Eigen, this should be the ticket:

cppyy.add_include_path(eigen_dir)

There are better ways, though, since you are using cmake already. See this repo: https://github.com/jclay/cppyy-knearestneighbors-example. With that, auto-loading should work. I.e. no need to deal with libraries and headers in your own code.