Can I incorporate system libraries (e.g. libxml2) I compile against into a gem (e.g. nokogiri) that I can deploy to Heroku?

Question

Nokogiri has a problem with translating to and from UTF-8 characters that turns out to come from libxml2, specifically version 2.7.6, which is the highest supported version on Ubuntu 10.04 LTS. The bug is fixed in version 2.7.7 and up, but since our app is hosted on Heroku (bamboo-ree-1.8.7 stack, based on Ubuntu 10.04), we have to use version 2.7.6, and continue to experience the bug, unless:

1. Someone can/has hacked nokogiri to get around the problem
2. Canonical bumps the supported libxml2 version for Ubuntu 10.04 (and/or Heroku updates libxml2 in their stack)
3. I can come up with a way for nokogiri to use a version of libxml2 which I can bundle with the app in such a way that it can be deployed to Heroku.

Happy to hear any feedback on 1 or 2 of course, but curious whether 3 is feasible. Here is what I know to be possible:

• Nokogiri can be linked against a library located in a non-standard place - http://nokogiri.org/tutorials/installing_nokogiri.html
• The gem created this way can be installed to vendor/gems, and thus take precedence over system gems

My question is, can I include the higher versioned libxml2 with the app in such a way that the compiled, vendor'ed gem uses it when I have committed it and pushed it to Heroku?

Answer 1

It's not the most user friendly solution, but here's what I did once to get a custom version of libpq. I did it on cedar, but it'll probably work for bamboo if you go into tmp first

1. heroku run bash
2. curl the source of what you want to build
3. build it
4. send the binaries out of the dyno
5. vendor the built binaries in vendor/whatever
6. heroku config:add LD_LIBRARY_PATH=/app/vendor/whatever

Sorry there isn't a better way to get custom binaires yet. Good luck.

Answer 2

I'm not sure of the exact answer but I have found an issue raised for Nokogiri which sounds like the issue you've just described: https://github.com/tenderlove/nokogiri/issues/458

One of the responses talks about how they are hosted on Heroku and are locked to version 2.7.6 of libxml2 and have pleaded their case to upgrade to 2.7.8 to fix their issue: https://github.com/tenderlove/nokogiri/issues/458#issuecomment-2600583

Although unlikely, it might be worth contacting Heroku to see what they can do (if anything), or if they possibly might have any suggestions? I would hazard a guess they won't change the libxml2 version as it's on a locked down stack (Bamboo) but who knows?