Test Speed: ActiveRecord use_transactional_fixtures vs. DatabaseCleaner.strategy = :transaction

Question

Judging from the source(database_cleaner, active_record), it appears that they should be equally fast. But there are claims that using database_cleaner's transaction strategy slows down controller and model specs (for example). I don't have a large test suite in hand for benchmarking. Anyone has any insight or has compared the two?

Answer 1

I spent a little time comparing the two on a mid-sized codebase that makes extensive use of ActiveRecord fixtures. When I switched it to use DatabaseCleaner instead of use_transactional_fixtures, the model specs started taking about twice the time.

After making the same comparisons you did, I realized that there are two things that use_transactional_fixtures controls:

1. Whether each spec is wrapped in a transaction
2. Where in the process the fixture data is loaded by ActiveRecord
   • When use_transactional_fixtures is true, the fixture data is loaded once, before the first test
   • When it is false, the fixture data is loaded before each spec

DatabaseCleaner's transaction strategy can substitute for (1). But the downside is that your suite will repeatedly load the same fixture data.

So, if you don't use AR fixtures (or only use a couple), use_transactional_fixtures and DatabaseCleaner's transaction strategy will have similar performance. But if you use a lot of AR fixtures, turning use_transactional_fixtures off will have a big performance impact.