Continue multi-host tests even on failure

Question

I've built some serverspec code to run a group of tests on multiple hosts. The trouble is that the testing stops at the current host when any test fails. I want it to continue to all hosts even if a test fails.

The Rakefile:

namespace :spec do
  task :all => hosts.map {|h| 'spec:' + h.split('.')[0] }
  hosts.each do |host|
    begin
      desc "Run serverspec to #{host}"
      RSpec::Core::RakeTask.new(host) do |t|
        ENV['TARGET_HOST'] = host
        t.pattern = "spec/cfengine3/*_spec.rb"
      end
    rescue
    end
  end
end

Complete code: https://gist.github.com/neilhwatson/1d41c696102c01bbb87a

Answer 1

This behaviour is controlled by RSpec::Core::RakeTask#fail_on_error so in order to have it continue on all hosts you need to add t.fail_on_error = false. I also think that you don't need to rescue.

namespace :spec do
  task :all => hosts.map {|h| 'spec:' + h.split('.')[0] }
  hosts.each do |host|
    desc "Run serverspec to #{host}"
    RSpec::Core::RakeTask.new(host) do |t|
      ENV['TARGET_HOST'] = host
      t.pattern = "spec/cfengine3/*_spec.rb"
      t.fail_on_error = false
    end
  end
end