How can you run a single RSpec example many times in parallel for deflaking?

Question

I have an RSpec test that is flaky. To reproduce the failure I want to run the test hundreds of times as fast as I can. How can I run a single example many times in parallel?

Is there a better way to reproduce a flaky test failure?

Answer 1

I debug flaky tests by wrapping the flaky code in

begin
  ... some code that I think is flaky
rescue => e
  require 'debug'; binding.break
end

and then running the spec a bunch of times. e.g.

# bash
for run in {1..10}; do
  bundle exec rspec spec/some/file/path.rb
done

Alternatively you can call the test 100 times in RSpec by wrapping the appropriate context or it block. e.g.

100.times do
  it '...' do
    # some test code
  end
end

If you have Ruby 3.0 you could try something with Ractor?

VIRTUAL_CPU_COUNT = Etc.nprocessors
RUN_COUNT = 100

ractors = []

VIRTUAL_CPU_COUNT.times do |i|
  ractors << Ractor.new(i) do |i|
    puts "In Ractor #{i}"
    (RUN_COUNT / VIRTUAL_CPU_COUNT).times do |t|
    `bundle exec rspec spec/some/file/path.rb`
    end
    puts "Finished Ractor #{i}"
    "i: #{i}" # implicit return value, or use Ractor.yield
  end
end

values = ractors.map(&:take)

I don't know how to get debugging to work though.

There's also the gem https://rubygems.org/gems/parallel_tests, that might help?