Native extensions fallback to pure Ruby if not supported on gem install

Question

I am developing a gem, which is currently pure Ruby, but I have also been developing a faster C variant for one of the features. The feature is usable, but sometimes slow, in pure Ruby. The slowness would only impact some of the potential users (depends which features they need, and how they use them), so it makes sense to have the gem available with graceful fallback to Ruby-only functions if it cannot compile on a target system.

I would like to maintain the Ruby and C variants of the feature in a single gem, and provide the best (i.e. fastest) experience from the gem on installation. That would allow me to support the widest set of potential users from a single project of mine. It would also allow other people's dependent gems and projects to use the best available dependency on a target system, as opposed to a lowest-common-denominator version for compatibility.

I would expect the require to fallback at runtime to appear in the main lib/foo.rb file simply like this:

begin
  require 'foo/foo_extended'
rescue LoadError
  require 'foo/ext_bits_as_pure_ruby'
end

However, I don't know how to get the gem installation to check (or try and fail) for native extension support so that the gem installs correctly whether or not it can build 'foo_extended'. When I researched how to do this, I mainly found discussions from a few years back e.g. http://permalink.gmane.org/gmane.comp.lang.ruby.gems.devel/1479 and http://rubyforge.org/pipermail/rubygems-developers/2007-November/003220.html that imply Ruby gems do not really support this feature. Nothing recent though, so I am hoping someone on SO has some more up-to-date knowledge?

My ideal solution would be a way to detect, prior to attempting a build of the extension, that the target Ruby did not support (or perhaps simply not want, at the project level) C native extensions. But also, a try/catch mechanism would be OK if not too dirty.

Is this possible, if so how? Or is the advice to have two gem variants published (e.g. foo and foo_ruby), that I am finding when I search, still current best practice?

Answer 1

This is my best result attempting to answer my own question to date. It appears to work for JRuby (tested in Travis and on my local installation under RVM), which was my main goal. However, I would be very interested in confirmations of it working in other environments, and for any input on how to make it more generic and/or robust:

---

The gem installation code expects a Makefile as output from extconf.rb, but has no opinion on what that should contain. Therefore extconf.rb can decide to create a do nothing Makefile, instead of calling create_makefile from mkmf. In practice that might look like this:

ext/foo/extconf.rb

can_compile_extensions = false
want_extensions = true

begin
  require 'mkmf'
  can_compile_extensions = true
rescue Exception
  # This will appear only in verbose mode.
  $stderr.puts "Could not require 'mkmf'. Not fatal, the extensions are optional."
end


if can_compile_extensions && want_extensions
  create_makefile( 'foo/foo' )

else
  # Create a dummy Makefile, to satisfy Gem::Installer#install
  mfile = open("Makefile", "wb")
  mfile.puts '.PHONY: install'
  mfile.puts 'install:'
  mfile.puts "\t" + '@echo "Extensions not installed, falling back to pure Ruby version."'
  mfile.close

end

As suggested in the question, this answer also requires the following logic to load the Ruby fallback code in the main library:

lib/foo.rb (excerpt)

begin
  # Extension target, might not exist on some installations
  require 'foo/foo'
rescue LoadError
  # Pure Ruby fallback, should cover all methods that are otherwise in extension
  require 'foo/foo_pure_ruby'
end

Following this route also requires some juggling of rake tasks, so that the default rake task doesn't attempt to compile on Rubies that we're testing on that don't have capability to compile extensions:

Rakefile (excerpts)

def can_compile_extensions
  return false if RUBY_DESCRIPTION =~ /jruby/
  return true
end 

if can_compile_extensions
  task :default => [:compile, :test]
else
  task :default => [:test]
end

Note the Rakefile part doesn't have to be completely generic, it just has to cover known environments we want to locally build and test the gem on (e.g. all the Travis targets).

I have noticed one annoyance. That is by default you will see Ruby Gems' message Building native extensions. This could take a while..., and no indication that the extension compilation was skipped. However, if you invoke the installer with gem install foo --verbose you do see the messages added to extconf.rb, so it's not too bad.