Why are my cucumber scenarios failing when steps are run together, but pass when run singularly?

Question

When I run my cucumber scenarios as a whole, or with the command: cucumber I get 7 failing steps. When I run them individually with the work in progress tag they pass fine.

I don't think it's a database state issue.. I'm running with transactions and I also tried running without and cleaning the database with database cleaner.... still does not help.

I tried to run the debugger but it does not seem to work when I run the command cucumber. It only works when I run with the work in progress tag: cucumber -p wip

I thought it might be that things are running too fast and capybara is not checking things properly?

Any ideas?

Answer 1

Eureka! I've been having this same problem for awhile now - my tests got slower and slower the more I added - also, some tests would fail randomly, but only when run as a whole suite - after my tests would finish I would just run the feature again and viola! all passing. Very frustrating - but the MOST frustrating part was the speed - recently I upgraded to snow leopard and compiled everything to 64bits. The result? My tests went from taking 7 minutes to 32!

There's a clue in that however - 64 bit apps use more memory to do do the same thing, apparently - however, when I was running my tests the memory on my machine was never coming close to maxing out. Hint #2? Webrat was going fast, it was only when using culerity/celerity to test javascript that things were really slowing down.

After poking around I found out that jruby tells java to give it a maximum 'heap size' of 512 mbs. JRuby allows you to set java options when it is invoked, and culerity allows an environment variable to invoke jruby any way you like. Sure enough, around that time, java would stop consuming memory and the processor would try to set itself on fire. So are you ready? Here it is:

JRUBY_INVOCATION="jruby -J-Xmx1024m" cucumber

That increased my heap size to a gigabyte and my test time dropped down to 7 minutes! Is that it? Did I get it? I sure hope it helps!