Performance testing with RSpec

Question

I'm trying to incorporate performance tests into a test suite for non-Rails app and have a couple problems.

1. I don't need to run perf tests every time, how can I exclude them? Commenting and uncommenting config.filter_run_excluding :perf => true seems like a bad idea.
2. How do I report benchmark results? I think RSpec has some mechanism for that.

Answer 1

I created rspec-benchmark Ruby gem for writing performance tests in RSpec. It has many expectations for testing speed, resources usage, and scalability.

For example, to test how fast your code is:

expect { ... }.to perform_under(60).ms

Or to compare with another implementation:

expect { ... }.to perform_faster_than { ... }.at_least(5).times

Or to test computational complexity:

expect { ... }.to perform_logarithmic.in_range(8, 100_000)

Or to see how many objects get allocated:

expect {
  _a = [Object.new]
  _b = {Object.new => 'foo'}
}.to perform_allocation({Array => 1, Object => 2}).objects

To filter your tests you can separate the specs into a performance directory and add a rake task

require 'rspec/core/rake_task'

desc 'Run performance specs'
RSpec::Core::RakeTask.new(:perf) do |task|
  task.pattern = 'spec/performance{,/*/**}/*_spec.rb'
end

Then run them whenever you need them:

rake perf