Installing custom builds heroku and issue with Library paths

Question

I'm attempting to install a custom build on heroku, so I'm using a variety of ways to attempt a third part installing using the buildpacks. In my .buildpacks file I have:

https://github.com/ddollar/heroku-buildpack-apt
https://github.com/heroku/heroku-buildpack-python.git

and in my Aptfile I have the following: libgeoip-dev which is a pre-requisite for geoip which is installed with the requirements.txt (GeoIP==1.3.2)

Here are my environment variables:

remote: C_INCLUDE_PATH is /app/.heroku/vendor/include:/app/.heroku/vendor/include:/app/.heroku/python/include
remote: CPATH is /tmp/build_xxxxx/.apt/usr/include:
remote: LD_LIBRARY_PATH is /app/.heroku/vendor/lib:/app/.heroku/vendor/lib:/app/.heroku/python/lib

The error message I am getting is:

remote:        building 'GeoIP' extension
remote:        creating build
remote:        creating build/temp.linux-x86_64-2.7
remote:        gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c py_GeoIP.c -o build/temp.linux-x86_64-2.7/py_GeoIP.o -fno-strict-aliasing
remote:        creating build/lib.linux-x86_64-2.7
remote:        gcc -pthread -shared build/temp.linux-x86_64-2.7/py_GeoIP.o -lGeoIP -o build/lib.linux-x86_64-2.7/GeoIP.so
remote:        /usr/bin/ld: cannot find -lGeoIP
remote:        collect2: error: ld returned 1 exit status
remote:        error: command 'gcc' failed with exit status 1

What is the smartest way to fix this? I.e. I guess I cannot change where the package manager installs. Is there a way around this?

Answer 1

https://github.com/heroku/heroku-buildpack-python/blob/master/bin/compile#L99-L107

# Prepend proper path buildpack use.
export PATH=$BUILD_DIR/.heroku/python/bin:$BUILD_DIR/.heroku/vendor/bin:$PATH
export PYTHONUNBUFFERED=1
export LANG=en_US.UTF-8
export C_INCLUDE_PATH=/app/.heroku/vendor/include:$BUILD_DIR/.heroku/vendor/include:/app/.heroku/python/include
export CPLUS_INCLUDE_PATH=/app/.heroku/vendor/include:$BUILD_DIR/.heroku/vendor/include:/app/.heroku/python/include
export LIBRARY_PATH=/app/.heroku/vendor/lib:$BUILD_DIR/.heroku/vendor/lib:/app/.heroku/python/lib
export LD_LIBRARY_PATH=/app/.heroku/vendor/lib:$BUILD_DIR/.hero ku/vendor/lib:/app/.heroku/python/lib
export PKG_CONFIG_PATH=/app/.heroku/vendor/lib/pkg-config:$BUILD_DIR/.heroku/vendor/lib/pkg-config:/app/.heroku/python/lib/pkg-config

vs.

https://github.com/ddollar/heroku-buildpack-apt/blob/master/bin/compile#L75-L81

export PATH="$BUILD_DIR/.apt/usr/bin:$PATH"
export LD_LIBRARY_PATH="$BUILD_DIR/.apt/usr/lib/x86_64-linux-gnu:$BUILD_DIR/.apt/usr/lib/i386-linux-gnu:$BUILD_DIR/.apt/usr/lib:$LD_LIBRARY_PATH"
export LIBRARY_PATH="$BUILD_DIR/.apt/usr/lib/x86_64-linux-gnu:$BUILD_DIR/.apt/usr/lib/i386-linux-gnu:$BUILD_DIR/.apt/usr/lib:$LIBRARY_PATH"
export INCLUDE_PATH="$BUILD_DIR/.apt/usr/include:$INCLUDE_PATH"
export CPATH="$INCLUDE_PATH"
export CPPPATH="$INCLUDE_PATH"
export PKG_CONFIG_PATH="$BUILD_DIR/.apt/usr/lib/x86_64-linux-gnu/pkgconfig:$BUILD_DIR/.apt/usr/lib/i386-linux-gnu/pkgconfig:$BUILD_DIR/.apt/usr/lib/pkgconfig:$PKG_CONFIG_PATH"

The heroku-buildpack-python buildpack is not playing nice with heroku-buildpack-apt buildpack as it is clobbering important variables for gcc to link your python extension with the geoip lib. File a bug on the issue tracker.

Issue tracker: https://github.com/heroku/heroku-buildpack-python/issues

Answer 2

I naively solved a similar problem by modifying heroku-buildpack-python's compile script to not unset LD_LIBRARY_PATH and INCLUDE_PATH, as well as to append the current LD_LIBRARY_PATH and INCLUDE_PATH variables during their export so as to not overwrite them.

Here is my fork: https://github.com/jasrusable/heroku-buildpack-python