Rails 3.0.7 -> How do you get your tests to run faster?

Question

I am running mysql, database_cleaner, Rspec, etc. I have about 518 tests so far and they take 88 seconds to run. This is unacceptable to me as my app development is just beginning.

So before going further, I'd like to try and find ways to reduce the time it takes to run these tests - hopefully without having to actually change the tests.

In most cases, I am trying to use stubs. However, when I am testing models and queries, I do use the database.

I think database_cleaner is slowing them down, but I don't know how to test queries and stuff without it.

Using sqlite3 with the ":memory:" option only seems to shave off about 10 seconds (kind of disappointing result...)

What can I do to really speed up my tests?

Answer 1

There's a variety of strategies you can use to speed up your test times. If you're just starting, and you're seeing an 88 second run time, I would imagine a good number of these apply to you:

• Use spork - Spork will do all the bootstrapping and environment requires once, keep that in memory, and only reload your tests. It can be a huge help with running tests quickly.
• Be smart about how you test - Personally, my workflow is to develop tests, run the full test suite to see what fails, and then only run the new / failed tests until I can get those green. Finally, when I'm all done I run the suite one last time to see if I regressed something else.
• Clean up your Gemfile - The majority of Rails boot time is spent in requires, and if you have gems you're not using anymore, you're adding load time for no benefit. Take out anything you're not using, and consider placing things only used in one or two spots in a named group so you can require them manually during execution (be careful with this - you're trading initial load speed for request performance, which is great for dev, but crappy for production)
• Be smart about what to stub and mock out - If you're truly doing unit tests, for instance, you should avoid touching the database althogether, or at least for your controller tests. Think about what the responsibility of the class really is. A controller isn't responsible for saving records, it's responsible for telling the models to save records. Even models aren't responsible for saving things in databases, they're responsible for telling ActiveRecord to.
• If you're stubbing things out well, consider not including Rails. You'll need to have nearly all ActiveRecord functionality stubbed out in your tests, but if you can do this, you'll see a massive decrease in your test time (probably more than an order of magnitude).

Answer 2

Ryan Brunner offered a lot of great advice. Everything he said is true in general, yet did not apply to me.

I didn't mention Factory Girl because I didn't think to mention it (don't ask). It turned out to be very relevant detail because it was responsible for the tests running so slow.

By simply removing Factory girl completely from my controller tests (I was using Factory.build), I have managed to get them down from 50 seconds to something like 5.

The reason is that Factory.build calls Factory.create for associations, which causes a database hit... so if you have a lot of associations, it will take awhile to create a new model object. But even more, that only accounted for 30-35% of the overhead in my case. Factory_girl was actually spending 65-70% of its time doing non-database stuff. I have no idea why, but after forcing every call to be Factory.build, it will still taking quite awhile to build my objects. Going with basic MyClass.new ended up being MUCH faster.

My entire test suite now takes a little under 30 seconds instead of up to 90 seconds. That is a 300% speed increase in general by making these changes... but when it came to the controller tests, I got a 2000% speed increase - and I was already stubbing! All of that performance overhead was due to Factory.build! That is where most of the gains came from.

Of course, I went back into my models and used Factory.build or simply MyClass.new wherever I could.

I also added :default_strategy => :build in factories.rb too whenever I could, to prevent Factory Girl from hitting the database. If you ask me, this should be the default as only 1 test failed as a result of this change, but I managed to get 10 entire seconds out of my model tests by this change alone.

If you're having problems like I am, follow these steps and you should notice a 2-3x speed improvement with not much drawback.