unit-tests karma-runner/jasmine profiling

Question

For unit-tests we use grunt/karma-runner/jasmine/phantom.js. Because we try to cover any new functionality, a number of unit-tests grows up rapidly. And, unfortunately, time for tests also increasing. Now it is not critical and takes 10 seconds for 1000 tests, but it goes worse and worse.

And questions:

1. I know, some of the tests are bad-written (a lot of time consuming), but which one I should optimize? Is it exist some karma/jasmine profiler which measure time execution for each test?

2. Can I start more karma-runner threads/processes, because CPU used only for 5%-10%? Unit-tests really independent.

3. Each time when I save file, karma:watch starts all tests, may be exist some option for karma-runner, which re-starts only tests for current folder (we use rule: unit tests file.spec.js stored in the same folder as source file.js)?

Thanks,

update1: Someone suggest me use iit/ddescribe for jasmine (the same as .only for mocha) and it is great option for develop/debugging, but may be exist some other way ?

I post the question to karma-user forum here.

Answer 1

Finding slow tests

• reportSlowerThan (dots or progress reporter will report all the tests that are slower than given number in ms),

• you can jump to the browser and profile your tests in the same way you would do it with an app (eg. flamecharts in Chrome are awesome).

Making the tests faster

• parallelisation:

  We are working on this. You will be able to parallelize the tests across multiple browsers. Of course, these browsers don't have to run on the same machine so you will be able to use multiple machines too.

  I don't think running multiple instances of Karma itself would help that much, as it is written in a very "non-blocking" way and I haven't seen it to be the bottleneck. We should however optimize the CPU/memory usage. A lot of improvements in this area is coming in v0.12.

• running only a subset of tests:

  You should totally use iit/ddescribe (Jasmine). Any reason why this does not work for you? Btw. Mocha's implementation of it.only() and describe.only() has issues, so I recommend Jasmine (or fixing Mocha).

  There is also a prototype of Google Closure plugin which only loads the files you really need (say you iit one test, it will only load/parse/evaluate the files you really need for that iit). This improves start time on huge projects significantly - we should do similar tricks for other module loaders (RequireJS, CommonJS).

• write faster code:

  You probably know this already... ;-)

  DOM is slow. A good separation of logic/DOM will allow you to test more stuff without actually touching DOM. If you use Angular, you might check out the testing directives example.

  Using Dependency Injection pattern helps a lot as you basically minimize the environment you need to prepare (and also tear down) for each test. Here's an example.

  In general, lower lower tests are faster. I think it's good to cover each functionality/issue with lowest possible test.

• eliminate memory leaks:

  If your tests leak memory, the browser will get slower and slower and so the test number 1000 will be pretty slow even though it's well written. Make sure each test frees up all the references. You can use http://localhost:9876/debug.html and profile the memory. Check the memory before executing (after Jasmine executed all the describe() blocks and collected the tests) and then after executing the tests - it should be the same.

  Finding some memory leaks is harder than others, but it can significantly improve the speed - I've seen stuff like from ~5mins to under a minute just by eliminating memory leaks.