Running Rails unit tests on Heroku

Question

I've deployed an app to Heroku, and it all works fine. The problem is I can't get my unit tests running remotely. I've tried:

heroku rake test:units

and

heroku rake db:test:prepare

but for both I get a massive stack trace, ending with:

rake aborted!
undefined method `[]' for nil:NilClass
/app/[id]/home/.bundle/gems/ruby/1.9.1/gems/activerecord-3.0.3/lib/active_record/railties/databases.rake:429:in `block (3 levels) in <top (required)>'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:634:in `call'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:634:in `block in execute'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:629:in `each'
/usr/ruby1.9.2/lib/ruby/1.9.1/rake.rb:629:in `execute'

I'm running the bamboo-mri-1.9.2 stack.

The closest I've got to an answer so far is this blog post from 2009.

Answer 1

Heroku doesn't provide a testing database, so there's no straightforward way to do that. You could theoretically create a new heroku instance and hack up the rake tasks to just use the 'production' database possibly, but I doubt the effort required would be worth it.

Answer 2

Heroku devcenter provides an explanation about how managing a staging environment here : http://devcenter.heroku.com/articles/multiple-environments

I don't know if it possible to run unit tests but I perfectly understand why you want to do this.

Your Heroku application:

• may have different gems from you local system (as far as I know, if you develop under Windows, your Gemfile.lock is ignored)
• may have a different OS (some Ruby libraries as IO does not behave exactely the same way on Windows and Linux)
• may have a different ruby version
• may have a different database system (it seems not so easy to use a local database with the exact same configuration as the one you use on Heroku)
• more generally speaking, will run on a different system (Linux version, ImageMagick, etc.)

You can test all that with a staging environment but we could say that for every unit test, so I think it would be great if we could run unit test on Heroku.

Answer 3

The normal way is to do the testing locally, then deploy to a hidden Heroku instance ("staging"). Then you can test that hidden staging app with your beta testers, run stress tests on it and so on. If you are satisfied with it, deploy your app to your "production" Heroku instance.

So testing on Heroku is possible and usual, but not with unit tests.

Answer 4

Heroku does not have the test suite, you really should be testing on development side before deployment to heroku.