Python3, Boost-Python and Cpp linker errors

Question

So I'm about to throw my laptop through the window, out the window, and go and burn Apple HQ.

See Updates Below:

I can't get python3, boost-python and clang to work with each other. The error I'm stuck at is running:

clang++ <FLAGS/INCLUDES> -o hello.so hello.cpp 

Invokes the response:

Undefined symbols for architecture x86_64:
  "__Py_NoneStruct", referenced from:
      boost::python::api::object::object() in hello-0c512e.o
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [hello] Error 1

Any help would be greatly, greatly appreciated. I think I have included everything necessary. Let me know if you require extra information.

The setup:

• OSX 10.11.6 (El Capi-s#@%)
• Must use Xcode 7.3 (and appropriate CLT): Requirement of NVIDIA for CUDA programming (installed).
• Must use Python3 (Homebrew installed)
  • brew install python3
  • which python -> /usr/bin/python
  • /usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/include/python3.5m
  • I set up an alias for python3.5 (see below).
• Using Boost-Python (Homebrew installed)
  • brew install boost
  • brew install boost-python --with-python3 --without-python
  • /usr/local/Cellar/boost-python/1.62.0/
• Using LLVM 3.9.0 (Homebrew installed)
  • brew install llvm --universal

Now a few helpful things that you may ask for:

Clang++:

Apple LLVM version 7.3.0 (clang-703.0.29)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

FLAGS and INCLUDES in makefile:

CPPFLAGS  = -g -Wall -std=c++11 -stdlib=libc++
LDHEADERS = -I/usr/local/opt/llvm/include
LDLIBS = -L/usr/local/opt/llvm/lib
BOOSTHEADERS = -I/usr/local/Cellar/boost/1.62.0/include/boost
BOOSTLIBS = -L/usr/local/Cellar/boost-python/1.62.0/lib
PYTHONHEADERS = -I/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/include/python3.5m
PYTHONLIBS = -L/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib

And finally, the code I am trying to compile:

hello.cpp

#include <boost/python.hpp>

struct World
{
    void set(std::string msg) { this->msg = msg; }
    std::string greet() { return msg; }
    std::string msg;
};

using namespace boost::python;

BOOST_PYTHON_MODULE(hello)
{
    class_<World>("World")
        .def("greet", &World::greet)
        .def("set", &World::set)
    ;
};

Answer 1

After much heartache and pain, I HAVE THE ANSWER!

To all those using OSX and homebrew, here's how you do it.

1. brew install python3 Python3 has UCS4 native (Unicode) which is an essential part of this process. If you need Python2, then make sure it is configured for UCS4 as it is natively UCS2.
2. brew install boost Install general boost first.
3. brew install boost-python --with-python3 --without-python This installs boost-python for Python3 WITHOUT Python2. You can change these options if you need Python2.
4. brew install llvm --universal Make sure you have llvm installed, this should have clang++ included in it which is the compiler we will use (not the Xcode one).
5. Create a makefile (see below) that includes the directories for all your python and boost headers/libraries, AND includes the libraries you want to use. (This is what tripped me up, I had the directories but did not specify which library in that directory the compiler should use).

My makefile:

# compiler flags:
#  -g    adds debugging information to the executable file
#  -Wall turns on most, but not all, compiler warnings

COMPILER = /usr/local/Cellar/llvm/3.9.0/bin/clang++
CPPFLAGS  = -g -Wall -std=c++11 -stdlib=libc++

# Python and BoostPython links.
BOOSTHEADERS = -I/usr/local/Cellar/boost/1.62.0/include/boost
BOOSTLIBRARIES = -L/usr/local/Cellar/boost-python/1.62.0/lib/
# This is the boost library we want to use, there are also libraries for multithreading etc. 
# All we do is find the file libboost_python3.a and link it by taking away the 'lib' and '.a'.
BOOSTLIB = -lboost_python3
PYTHONHEADERS = -I/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/include/python3.5m
PYTHONLIBRARIES = -L/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib
# Link to python3.5 library, the same as we did for boost.
PYTHONLIB = -lpython3.5

# Collect links.
LIBRARIES = $(BOOSTLIBRARIES) $(PYTHONLIBRARIES) $(PYTHONLIB) $(BOOSTLIB)
HEADERS = $(BOOSTHEADERS) $(PYTHONHEADERS)

# Build target.
TARGET = hello


# BEGIN MAKE
all: $(TARGET)

$(TARGET): $(TARGET).cpp
    # Note that '-shared' creates a library that is accessible.
    $(COMPILER) -shared $(LIBRARIES) $(HEADERS) $(TARGET).cpp -o $(TARGET).so

clean:
    $(RM) $(TARGET)

Then all you need to do is run your makefile with all the inclusions and everything should be sweet :) I hope this helps someone and removes the pain I had of trying to get insertProfanity boost working.

Answer 2

Not enough points to comment so unfortunately I'll have to post as an answer since not too long ago I was going through this same linker error stuff with Boost.

My guess is that this appears to be a Python linker problem, as opposed to a Boost linker problem. I see that you've added the Python includes to your CXX_INCLUDE_PATH, but what about your Python library?

If the path you've mentioned above (that long path in your CXX_INCLUDE_PATH variable) is where your includes are then [that_long_path]/Versions/3.5/lib should be where your Python libraries are. Include this library when you run your build command by using

clang++ -g -v -std=c++11 -stdlib=libc++ -L/[that_long_path]/Versions/3.5/lib -lpython3.5m hello.cpp -o hello.so 

The -L flag tells the compiler to include that directory, while the -l flag tells the compiler to include the following library. Alternatively you can just append this lib path to your current CXX_INCLUDE PATH, which would then look like this:

export CXX_INCLUDE_PATH="/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/include/python3.5m:/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib"

Also make sure you are including any relevant boost include paths and library paths when you run the build command to avoid any more linker errors (if boost is not installed in a default location like usr/local/[some_place].

Then run this new build command (may be able to do without the -std and -stdlib flags:

clang++ -g -v -std=c++11 -stdlib=lib++ -lpython3.5m -o hello.so hello.cpp

In summary: 1) Include your python header files (which you seem to have done) 2) Include your python library file(s) (which you seem to be missing) 3) Include any relevant boost libraries and the boost include directory (which is a bit out of the scope of this question but still worth noting)

Hope this helps.